In the landscape of 2023, Retrieval-Augmented Generation (RAG) became a prominent technique for embedding knowledge into large language models (LLMs). The limitations of context windows often led to hallucinations, making RAG a go-to solution for extracting information through natural language queries. However, the RAG pipeline can be cumbersome and resource-intensive. Organizations seeking to implement chat functionalities based on their documentation face a daunting task. Developers typically require a comprehensive pipeline involving chunking, embeddings, vector databases, and rerankers, which can take months to establish and necessitate ongoing maintenance, especially for extensive documentation. With many modern models capable of handling over a million tokens, it is often feasible to fit entire documentation within the context window. But if it doesn't fit, is it essential to build a complete RAG system from scratch? This article will explore the reasons behind RAG's default status, examine the quality decline associated with Long Context, compare different methodologies, and provide insights on when and how to utilize these approaches effectively.
RAG, Search, and LongContext: Why Complex Pipelines Aren't Always Necessary
In 2023, RAG emerged as a primary method for integrating knowledge into LLMs, but its complexity raises questions about necessity.
4 min readTechnology