DisastRAG: An Integrated System for Disaster Information Access

Effective disaster response hinges on swift access to a variety of information, which is often scattered across different formats, including structured records and unstructured documents. Traditional systems typically rely on a singular method for information retrieval, which can hinder their effectiveness in meeting diverse and urgent needs during crises. DisastRAG addresses this challenge by integrating a multi-source approach that leverages retrieval-augmented large language models. This innovative framework allows for the retrieval of documents from a carefully curated database of hazards, structured access to relational disaster records, and a fallback to external web sources for additional data. The system also features advanced capabilities such as query understanding, strategic routing, and contextual memory, all within a cohesive structure. Performance evaluations demonstrated that incorporating retrieval augmentation significantly enhances outcomes, with improvements of 12-23 percentage points in multiple-choice tasks and up to 10.5 percentage points in open-ended scenarios. Notably, the findings indicate that larger candidate pools benefit less robust models, while more powerful models are affected by retrieval inaccuracies. The hybrid retrieval method excels in providing comprehensive coverage for open-ended queries, while vector retrieval tends to favor precise factual answers.