{"notes":[{"id":"retrieval-augmented-generation/rag-failure-points","category":"retrieval-augmented-generation","legacyPath":"","legacySlug":"","legacyPaths":[],"title":"Seven Failure Points","subtitle":"The Seven Deadly Sins","author":"Z. Yuan","paperDate":"2024-01-11","publishedOn":"2026-05-28","paperTitle":"Seven Failure Points When Engineering a Retrieval Augmented Generation System","paperUrl":"https://arxiv.org/abs/2401.05856","bibliography":{"title":"Seven Failure Points When Engineering a Retrieval Augmented Generation System","authors":["Scott Barnett","Stefanus Kurniawan","Srikanth Thudumu","Zach Brannelly","Mohamed Abdelrazek"],"url":"https://arxiv.org/abs/2401.05856","pdfUrl":"https://arxiv.org/pdf/2401.05856v1","codeUrl":null,"preprint":{"id":"2401.05856","submittedOn":"2024-01-11","url":"https://arxiv.org/abs/2401.05856"},"publication":null,"citation":"@misc{retrievalaugmentedgenerationragfailurepoints2024,\n  title = {{Seven Failure Points When Engineering a Retrieval Augmented Generation System}},\n  author = {Scott Barnett and Stefanus Kurniawan and Srikanth Thudumu and Zach Brannelly and Mohamed Abdelrazek},\n  year = {2024},\n  eprint = {2401.05856},\n  archivePrefix = {arXiv},\n  url = {https://arxiv.org/abs/2401.05856}\n}\n","citationPath":"/content/papers/citations/retrieval-augmented-generation/rag-failure-points.40ca0c32c91e.bib"},"thumbnail":{"src":"/content/papers/assets/bd756f315ebe65964e8195f5120c63e86e709d677c39b319fecacd74ea197d71.png","width":1422,"height":578},"locale":"en","path":"/content/papers/en/retrieval-augmented-generation/rag-failure-points.ea3adc527c55.md","headings":[{"id":"defining-the-problem","text":"Defining the Problem","depth":2},{"id":"solving-the-problem","text":"Solving the Problem","depth":2},{"id":"three-case-studies","text":"Three Case Studies","depth":3},{"id":"rags-two-processes","text":"RAG's Two Processes","depth":3},{"id":"one-question-seven-ways-to-go-wrong","text":"One Question, Seven Ways to Go Wrong","depth":3},{"id":"discussion","text":"Discussion","depth":2},{"id":"lessons-from-the-three-cases","text":"Lessons from the Three Cases","depth":3},{"id":"research-directions-proposed-by-the-authors","text":"Research Directions Proposed by the Authors","depth":3},{"id":"what-to-keep-in-mind-while-reading","text":"What to Keep in Mind While Reading","depth":3},{"id":"conclusion","text":"Conclusion","depth":2}],"anchorAliases":{},"guide":null,"relatedReading":[]}],"text":"In 2023, putting together a RAG demo was easy enough.\n\nSplit the documents into chunks, compute embeddings, and put them in a vector database. When a question arrives, retrieve the closest chunks and send them to ChatGPT alongside the question. In an afternoon, you could show your boss that “AI understands our documents.”\n\nThe hard part came afterward.\n\nThis paper comes from the Applied Artificial Intelligence Institute at Deakin University in Australia. It appeared at CAIN 2024, a conference on software engineering for AI systems. It introduces no new model and sets no new benchmark record. Instead, it is an **experience report**: the authors built three RAG systems and organized the failures they encountered into seven categories.\n\n## Defining the Problem\n\nThe authors begin with three things RAG is meant to accomplish:\n\n1. **Reduce hallucinations**: have the model answer from documents rather than improvise.\n2. **Provide sources**: let users see which document an answer comes from.\n3. **Avoid annotation**: use semantic search directly, without manually adding metadata to every document.\n\nBut RAG also inherits weaknesses from both sides. Information retrieval systems retrieve the wrong material or miss relevant material; large language models misread it or give bad answers.\n\nThe authors therefore ask two questions:\n\n- **Where can a RAG system fail during engineering?**\n- **What should engineers watch for?**\n\n## Solving the Problem\n\n### Three Case Studies\n\nThe report draws on three systems:\n\n| Case | Domain | Documents | Scale | Example question |\n| --- | --- | --- | --- | --- |\n| Cognitive Reviewer | Research | PDFs | Unrestricted | What are this paper's key points? |\n| AI Tutor | Education | Videos, HTML, PDFs | 38 documents | What topics were taught in week six? |\n| BioASQ | Biomedicine | Scientific papers in PDF | 4,017 documents | What is pseudotumor cerebri, and how is it treated? |\n\n*Adapted from Table 1 of the paper.*\n\n**Cognitive Reviewer** helps researchers read the literature. A user enters a research question and uploads a set of papers; the system ranks them against the research objective and supports questions across the entire collection. Users upload the documents at runtime, so developers cannot check their quality in advance.\n\n**AI Tutor** acts as a teaching assistant. Students ask questions, and the system answers from the course materials with citations. Videos are transcribed with Whisper before being chunked. A rewriter uses the conversation history to turn vague follow-ups such as “Explain that concept again” into complete queries. The system was developed from August to November 2023 and piloted from October 30 in a course with about 200 students.\n\nBoth systems were used at Deakin University. Confidentiality constraints meant that only the third case released its data and code.\n\n**BioASQ** was built to test a larger collection. The authors downloaded 4,017 open-access papers from the BioASQ biomedical question-answering dataset, used GPT-4 to answer 1,000 questions, and scored the answers automatically with OpenAI's evals tool. They manually checked answers marked incorrect and a subset of those marked correct.\n\nIn this domain, they found automatic scoring stricter than human scoring. They also identified a validity concern: the human reviewers were not biomedical experts, and the language model might know more than they did.\n\n\n> [!NOTE]\n> \n> The introduction and conclusion say 15,000 documents, while §4.3 and Table 1 say 4,017. This note uses the latter figure.\n\n\n### RAG's Two Processes\n\nBefore looking at the failures, let us read the authors' pipeline diagram. The red boxes mark the seven failure points.\n\n[![The indexing and query processes of a RAG system, with red boxes marking the seven failure points](/content/papers/assets/bd756f315ebe65964e8195f5120c63e86e709d677c39b319fecacd74ea197d71.png)](/content/papers/assets/bd756f315ebe65964e8195f5120c63e86e709d677c39b319fecacd74ea197d71.png)\n\n*Figure 1 of the paper, from [**page 3 of the v1 PDF**](https://arxiv.org/pdf/2401.05856v1#page=3), CC BY 4.0. Indexing is shown above, querying below; red boxes mark failure points.*\n\nThe upper **Index Process** usually runs during development. A Chunker splits documents into smaller pieces, each piece is turned into an embedding, and the embeddings are stored in a database.\n\nEven here, the decisions are difficult. Chunks that are too small cannot answer some questions; chunks that are too large bring noise into the answer. Changing the embedding model requires reindexing all documents.\n\nThe lower **Query Process** runs at runtime and passes through five stages:\n\n1. **Rewriter**: a language model rewrites the question into a generalized query, optionally using the conversation history.\n2. **Retriever**: the query embedding retrieves the top $K$ most similar chunks.\n3. **Reranker**: the chunks are reordered to bring those containing the answer toward the front.\n4. **Consolidator**: the content that will enter the prompt is assembled. This stage exists because the APIs of the time had token and rate limits. When all snippets cannot fit, the system must select, combine, or chain multiple prompts.\n5. **Reader**: the model filters noise from the prompt, follows formatting instructions, and generates the final answer.\n\n### One Question, Seven Ways to Go Wrong\n\nTake AI Tutor's example question, “What topics were taught in week six?” We can follow it through the pipeline to see where each failure occurs.\n\n\n> [!TIP]\n> \n> The scenarios below are illustrations devised for this note. The paper does not trace this question through all seven failures; the definitions of the failure points follow §5 of the paper.\n\n\n**FP1: Missing Content.** The week-six materials were never uploaded. Ideally, the system should answer, “Sorry, I don't know.” But when a question is close to existing content, it may be fooled into inventing an answer.\n\n**FP2: Missed the Top Ranked Documents.** The week-six slides are in the database, but rank eighth in similarity while the system retrieves only the top five. In principle, all documents can be ranked; in practice, only the top $K$ proceed downstream.\n\n**FP3: Not in Context.** The slides were retrieved, but the Consolidator dropped them to fit the token limit.\n\n**FP4: Not Extracted.** The slides really are in the prompt, but the model fails to extract the answer. The authors say this commonly happens when the context is noisy or contradictory—for example, when materials from both the old and new semesters are retrieved.\n\n**FP5: Wrong Format.** The student asks for a bulleted list, but the model writes a paragraph and ignores the formatting instruction.\n\n**FP6: Incorrect Specificity.** The response contains the answer, but is too general or too detailed. The authors give an educational example: teachers want the answer accompanied by relevant teaching materials. This can also happen when users are unsure how to ask and make the question too broad.\n\n**FP7: Incomplete.** Week six covered three topics, but the model lists only two, even though the third is in the context. The authors suggest splitting questions such as “What are the key points of documents A, B, and C?” into separate requests.\n\nThe first three failures happen **before the answer reaches the model**: the material is absent, is not retrieved, or is left out of the prompt. The last four happen **after the model receives it**: failure to extract, wrong format, wrong specificity, or a missing part.\n\n## Discussion\n\n### Lessons from the Three Cases\n\nThe authors collect their lessons in Table 2, linking each to the relevant failure points:\n\n1. **A larger context improved results.** AI Tutor answered more accurately with an 8K context than with 4K. The authors explicitly note that this contradicts earlier findings on GPT-3.5. (FP4)\n2. **Semantic caching can reduce cost and latency.** Rate limits and fees become concerns with concurrent users; common questions can be placed in a semantic cache. (FP1)\n3. **Jailbreak attacks can bypass RAG.** Research suggests that fine-tuning can weaken a model's safety training, so fine-tuned models used in RAG should be tested. (FP5–7)\n4. **Metadata helps retrieval.** Adding filenames and chunk numbers to retrieved content helps the Reader extract the required information. (FP2, FP4)\n5. **On short texts, open-source embedding models can match commercial models.** (FP2, FP4–7)\n6. **RAG systems need continuous calibration.** Unexpected inputs arrive at runtime, so ongoing monitoring is necessary. (FP2–7)\n7. **Make the RAG pipeline configurable.** Chunk size, embeddings, chunking strategy, retrieval strategy, consolidation strategy, context size, and prompts all need adjustment. (FP1, FP2)\n8. **A pipeline assembled piecemeal is not optimal.** The authors cite research showing that end-to-end training can improve domain adaptation. (FP2, FP4)\n9. **Performance can only be measured at runtime.** Offline evaluation methods such as G-Eval look promising, but require annotated question-and-answer data. (FP2–7)\n\nThe final point echoes the abstract's two conclusions:\n\n- **RAG systems can only be validated during operation.**\n- **Robustness in a RAG system develops over time; it cannot be fully designed in at the outset.**\n\nThe reason is practical. RAG works with documents that have not been curated into evaluation material, and developers usually lack ready-made question-and-answer test sets. The actual questions become clear when real users start asking them.\n\n### Research Directions Proposed by the Authors\n\n1. **Chunking and embeddings**: should chunks follow punctuation and paragraphs, or semantic boundaries? Their effects on retrieval need systematic comparison. Embedding multimodal content such as tables, images, and formulas remains an open problem too.\n2. **RAG and fine-tuning**: fine-tuning trains information into a model and must address who can access what. RAG can add documents at any time and control which snippets users can see. The trade-offs in accuracy, latency, cost, and robustness need systematic comparison.\n3. **Testing and monitoring**: generating test questions and answers that reflect a real domain remains an open problem. Each model update can also change latency and performance characteristics.\n\n### What to Keep in Mind While Reading\n\nThis is an experience report. Its seven failure points are categories distilled from three cases. The paper **does not report how frequently each failure occurs**, nor compare how much different designs reduce them. The data from the two deployed systems remain confidential; only the BioASQ case is available for examination.\n\nThese are also experiences from early 2024, in the GPT-4 era. The Consolidator behind FP3 existed mainly because of token limits; later increases in context length changed the pressure on that stage. That is a subsequent development rather than a finding of the paper, but the timing is useful to bear in mind.\n\n## Conclusion\n\nThis paper offers a map rather than a new method.\n\nIt separates a RAG system into indexing and querying, then marks seven places where it can fail: missing content, missing the top-ranked results, omission from context, failure to extract, wrong format, wrong specificity, and an incomplete answer. The first three concern getting the material in front of the model; the last four concern how well the model uses it once it arrives.\n\nThe map is useful because, when your RAG system gets an answer wrong, you can inspect the stages one by one. Does the material exist? Was it retrieved? Was it included? Did the model find the answer?\n\nA better embedding model can fix only some of these problems.\n\nFor the rest, you may have to wait until launch and meet them one at a time.\n","imageSizes":{"/content/papers/assets/bd756f315ebe65964e8195f5120c63e86e709d677c39b319fecacd74ea197d71.png":{"width":1422,"height":578}}}