https://arxiv.org/html/2403.14403v2

🌈 Abstract

The article discusses the development of Adaptive-RAG, a novel adaptive question-answering (QA) framework that can dynamically select the most suitable strategy for retrieval-augmented large language models (LLMs) based on the complexity of the input query. The key points are:

• Retrieval-augmented LLMs, which incorporate external knowledge into LLMs, have shown promise in enhancing QA accuracy. However, existing approaches either handle simple queries with unnecessary overhead or fail to adequately address complex multi-step queries.
• The proposed Adaptive-RAG framework uses a classifier to predict the complexity level of incoming queries and selects the most appropriate retrieval-augmented or non-retrieval strategy accordingly.
• The training dataset for the complexity classifier is automatically collected by leveraging the predicted outcomes of different models and the inherent biases in existing QA datasets.
• Adaptive-RAG is shown to significantly improve the overall efficiency and accuracy of QA systems compared to relevant baselines, by balancing the complexity and simplicity for diverse queries.

🙋 Q&A

[01] Introduction

1. What are the key limitations of existing retrieval-augmented LLMs for question-answering?

• Existing retrieval-augmented LLMs focus primarily on single-hop queries, which can be answered by retrieving a single relevant document.
• They struggle to effectively handle complex multi-step queries that require connecting and aggregating information from multiple documents.
• Approaches that handle complex multi-step queries through iterative retrieval and reasoning incur heavy computational overhead, while simpler single-step approaches are insufficient for complex queries.

2. What is the key idea behind the proposed Adaptive-RAG framework? The key idea is to dynamically select the most suitable retrieval-augmented or non-retrieval strategy based on the predicted complexity of the input query, in order to balance the efficiency and accuracy for diverse query complexities.

3. How does Adaptive-RAG operationalize the process of determining query complexity? Adaptive-RAG uses a classifier, which is a smaller language model trained to predict the complexity level of incoming queries. The training dataset for this classifier is automatically collected by leveraging the predicted outcomes of different models and the inherent biases in existing QA datasets.

[02] Related Work

1. What are the key components of open-domain question-answering (QA) systems? Open-domain QA systems typically consist of two main components:

• A retriever module that sources relevant documents for the given query
• A reader module that interprets the retrieved documents to provide the final answer

2. How have retrieval-augmented LLMs advanced open-domain QA? The integration of LLMs and retrieval modules has led to significant advancements in open-domain QA by:

• Mitigating the hallucination problem in LLMs through strengthened reasoning abilities
• Leveraging the retrieved external documents to enhance the overall QA performance

3. What are the key approaches in multi-hop QA? Multi-hop QA approaches generally involve:

• Decomposing the complex multi-hop queries into simpler single-hop sub-queries
• Iteratively accessing the LLMs and retriever modules to solve the sub-queries
• Merging the solutions to the sub-queries to formulate the final answer

Some recent studies have also explored interleaving Chain-of-Thought reasoning within this iterative process.