Innovative Bayesian Alignment for Enhanced LLM Recommendations

The landscape of recommendation systems has been transformed by Large Language Models (LLMs), which utilize their generative strengths to better understand user preferences. However, many current methods in LLM-based recommendations focus on token-level objectives, which complicates the optimization of list-level metrics that truly reflect recommendation effectiveness, such as NDCG and fairness. Best-of-N (BoN) methods aim to directly optimize these metrics during the recommendation process, but their computational demands often limit practical applications. To tackle these issues, BoN Alignment seeks to embed search capabilities within the model. Yet, this approach faces two significant challenges: the problem of Indiscriminate Supervision, where static references fail to accurately assess the quality of candidates beyond their empirical limits, and Gradient Decay, where the supervision signal weakens as the model improves, hindering optimization efficiency. To address these limitations, we introduce BLADE (Bayesian List-wise Alignment via Dynamic Estimation). BLADE employs a Bayesian framework that continuously refines the target distribution by integrating historical data with real-time evidence from the model's performance. This adaptive mechanism ensures that the training feedback remains relevant and effective. Our extensive evaluations across three real-world datasets reveal that BLADE surpasses existing benchmarks, achieving notable improvements in ranking accuracy and complex list-wise metrics.