An Exploratory Tour of Retrieval Augmented Generation (RAG) Paradigm

🌈 Abstract

The article provides an exploratory tour of the Retrieval Augmented Generation (RAG) paradigm, covering its methodologies, functionality, architecture, types, benefits, limitations, and future directions. It also discusses the evaluation of RAG-based applications.

🙋 Q&A

[01] What is Retrieval Augmented Generation (RAG)?

• RAG is a technique that enhances large language models (LLMs) by supplementing them with external context, such as databases and customer records, to improve the accuracy and credibility of their generative responses, particularly for knowledge-intensive tasks.
• RAG merges retrieval methods with deep learning advancements to address the static limitations of LLMs by enabling the dynamic integration of up-to-date external information.

[02] Why use RAG?

• RAG addresses the limitations and challenges of LLMs, such as their reliance on limited training data, the risk of hallucination, and the lack of private data context.
• RAG improves LLM responses by retrieving pertinent document segments from external sources using semantic similarity, mitigating the issue of generating inaccurate content.
• Other reasons to use RAG include improving knowledge cutoff, reducing hallucination risks, enhancing contextual limitations, and improving auditability of generative AI responses.

[03] What are the types of RAG?

• Naive RAG: The earliest and simplest type, involving indexing, retrieving, and generating.
• Advanced RAG: Improves the retrieval phase through pre-retrieval and post-retrieval strategies, such as fine-grained segmentation, metadata usage, re-ranking, and shortening of retrieved chunks.
• Modular RAG: Adopts a versatile, modular approach, allowing for the integration of efficient strategies like better similarity search, query rewriting, and routing searches across various data sources.

[04] How does RAG work?

• RAG consists of two main phases: retrieval and generation.
• In the retrieval phase, efficient search strategies fetch the most relevant semantic similar documents from a vector store or knowledge base that match the user prompt or query.
• In the generation phase, the coalesced and condensed context are provided to the LLM to generate the final response.

[05] What are the common use cases of RAG?

• Customer question-answering systems: RAG-enhanced chatbots can provide accurate answers by accessing and interpreting relevant information from internal knowledge bases.
• Content recommendation systems: RAG improves user interaction and content engagement by leveraging advanced retrievers for personalized recommendation.
• Educational and legal research analysis: RAG simplifies and creates condensed study materials and legal analysis by accessing and summarizing relevant materials.
• Content creation and summarization: RAG models can assist with finding useful information from different sources and creating condensed reports and summaries.

[06] How does RAG compare to prompt engineering and fine-tuning?

• Prompt engineering involves minimal changes to the model or external knowledge base, utilizing the inherent parametric-knowledge capabilities of LLMs.
• Fine-tuning requires additional training of the model with domain-specific datasets, resulting in a niche model for a specific use case.
• RAG initially had little need for model adjustments, but as the research advances, Modular and Advanced RAG will increasingly incorporate fine-tuning methods.

[07] How are RAGs evaluated?

• Traditional metrics like Exact Match (EM) and F1 scores are used to evaluate RAG model performance on downstream tasks.
• Recent research has focused on developing comprehensive frameworks and benchmarks to evaluate RAGs, considering both retrieval-based and generation-based aspects.
• Retrieval-based frameworks and benchmarks focus on the effectiveness of retrieving relevant information, while generation-based aspects assess language quality, coherence, relevancy, and accuracy.

[08] What are the future developments in RAG?

• Pluggable and modular search methods in Advanced and Modular RAG approaches
• Multi-model based RAG
• Simplifying retrieval methods, especially during multi-hop retrieval and re-ranking
• Production-ready RAG
• RAGs in long-context length