> For the complete documentation index, see [llms.txt](https://riteshs4hu.gitbook.io/infosec-notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://riteshs4hu.gitbook.io/infosec-notes/artificial-intelligence/ai-agents/retrieval-augmented-generation-rag.md). # Retrieval-Augmented Generation (RAG) Retrieval-Augmented Generation (RAG) is an architecture that improves the output of a large language model (LLM) by retrieving relevant information from external data sources before generating a response. Instead of relying only on its training data, the model is provided with additional context retrieved from documents, databases, or knowledge bases. *** ### Core Concept RAG separates responsibilities into two parts: * Retrieval system selects relevant information * Language model interprets that information and generates an answer This approach ensures that responses are grounded in real data. *** ### How RAG Works #### Indexing Phase ``` Documents → Chunking → Embeddings → Vector Storage ``` * Documents are divided into smaller chunks * Each chunk is converted into an embedding (numerical representation) * Embeddings and original text are stored for retrieval *** #### Query Phase ``` Query → Embedding → Similarity Search → Relevant Chunks → LLM → Answer ``` * User query is converted into an embedding * Similar chunks are retrieved using vector similarity * Retrieved text is passed to the LLM as context * LLM generates the final response *** ### What is Semantic Search? Semantic search retrieves information based on meaning rather than exact keyword matching. It uses embeddings to identify relationships between words and phrases. #### Example ``` Query: "car repair" Document: "how to fix a vehicle" ``` Semantic search identifies similarity in meaning and retrieves the document. *** ### Semantic Search vs Keyword Search | Aspect | Keyword Search | Semantic Search | | ------------------------ | -------------- | --------------- | | Matching | Exact words | Meaning-based | | Synonyms | Not supported | Supported | | Natural language queries | Limited | Effective | | Accuracy on large data | Low | High | #### Example ``` Query: "Who runs the archive?" Document: "Ishan manages the archive" ``` Keyword search fails due to word mismatch.\ Semantic search retrieves the correct result. *** ### Why RAG is Required Large language models have inherent limitations: * Knowledge is static and limited to training data * Responses may be incorrect or fabricated * No access to private or domain-specific information RAG addresses these issues by retrieving relevant external data and grounding responses. *** ### Role of RAG vs LLM | Component | Responsibility | | --------- | ----------------------------- | | RAG | Retrieve relevant information | | LLM | Generate and explain answer | RAG does not perform reasoning or generate text.\ The LLM does not retrieve external data without RAG. *** ### Token Usage in RAG RAG does not always reduce tokens per request. Adding retrieved context increases the total input size. #### Example ``` Query only → ~50 tokens Query + context → ~300 tokens ``` *** ### When RAG Reduces Token Usage RAG becomes efficient in scenarios involving large data. #### Example ``` Without RAG → Send full document (5000 tokens) With RAG → Send relevant chunks (300 tokens) ``` RAG reduces unnecessary data transfer and improves efficiency over time. *** ### When RAG is Not Required RAG is not necessary when: * The query involves general knowledge * The dataset is small * Exact keyword search is sufficient #### Example ``` "What is FTP?" ``` The LLM can answer directly without external retrieval. *** ### Grep vs RAG #### Keyword-Based Approach ``` Documents → Keyword Search → Results → LLM ``` #### RAG-Based Approach ``` Documents → Embeddings → Semantic Search → Relevant Chunks → LLM ``` #### Example ``` Document: "Employees are entitled to 20 days of paid leave" Query: "How many vacation days?" ``` Keyword search fails due to missing exact terms.\ RAG retrieves the correct information using semantic similarity. *** ### Real-World Applications #### Internal Knowledge Systems Used for querying company documents such as policies and procedures. ``` Query: "What is the leave policy?" ``` *** #### Developer Tools Used for searching codebases and technical documentation. ``` Query: "Where is authentication implemented?" ``` *** #### Customer Support Used to provide answers from manuals and FAQs. ``` Query: "Why is my device not connecting to WiFi?" ``` *** #### Personal Knowledge Bases Used to query notes, research, and documentation. ``` Query: "Explain RAG from my notes" ``` *** ### Limitations of RAG * Incorrect retrieval leads to incorrect answers * Additional latency due to retrieval step * Requires tuning of chunk size and retrieval parameters * Dependent on data qualityRelevant retrieval → Better context → Accurate response *** *