Enhancing Reading Recommendations with Retrieval-Augmented Generation and LLMs

This article discusses a newly developed system aimed at generating tailored reading materials by leveraging Large Language Models (LLMs) in conjunction with Retrieval-Augmented Generation (RAG). The architecture is structured into four key components: Input, RAG, Generation, and Judging. Users can input specific questions and indicate the desired complexity of the reading material. The RAG component plays a crucial role by sourcing relevant information from the web, which enhances the content generated by three advanced LLMs: Meta LLaMA 4 Scout, LLaMA 3.1 8B Instant, and Google Gemma2 9B. The system employs various prompting methods, including Chain-of-Thought, zero-shot, and few-shot prompting, to produce reading content. Additionally, the LLM-as-a-Judge component assesses the quality of the generated responses and their alignment with the requested readability level. Experimental findings indicate that the integration of RAG significantly boosts the performance of the system across all models and prompting strategies, improving relevance and grounding by as much as 26-35 percentage points. The results affirm that this RAG-enhanced framework effectively generates reading content that meets user inquiries and complexity preferences.