M-RAG: Enhancing Retrieval-Augmented Generation

Retrieval-Augmented Generation (RAG) has gained traction as a method to boost the dependability of large language models (LLMs). However, traditional RAG systems face challenges due to their reliance on text chunking, which can lead to fragmented information, increased noise in retrieval, and overall inefficiency. Some recent studies suggest that advanced long-context LLMs might render multi-stage retrieval unnecessary by directly handling entire documents. Yet, simply increasing context capacity does not address issues like relevance filtering and prioritizing evidence. To tackle these problems, we introduce M-RAG, a new retrieval strategy that eliminates the need for text chunks. M-RAG utilizes structured k-v decomposition meta-markers, incorporating a streamlined retrieval key aligned with user intent and a context-rich information value for generation. This approach facilitates efficient query-key similarity matching while maintaining expressive capabilities. Experimental findings from LongBench subtasks indicate that M-RAG surpasses traditional chunk-based RAG methods, especially in low-resource scenarios. Further analysis shows that M-RAG retrieves more relevant evidence efficiently, confirming the advantages of separating retrieval representation from generation and positioning this method as a scalable and effective alternative to existing chunk-based techniques.