Understanding LLM Memorization in Generative Recommendations: Insights and Training Approaches

Generative recommendation (GR) has gained traction as a forward-looking approach in the realm of recommendation systems. Large language models (LLMs) have been increasingly utilized for GR due to their extensive pretrained knowledge, which is anticipated to enhance their ability to generalize beyond the typical user behavior patterns captured by traditional memorization-focused models. However, a significant oversight in current LLM-based GR research is the models' propensity to memorize, which can hinder their effective use of pretrained knowledge. This study delves into the phenomenon of one-hop memorization, where models suggest items that are immediate successors to those found in the training dataset. Our findings indicate that LLMs exhibit this behavior more frequently than their non-LLM counterparts, with a substantial portion of their performance improvements stemming from users whose target items can be predicted through one-hop memorization. To address this limitation, we introduce IIRG, a novel training framework designed to help LLMs learn deeper item-item relationships that extend beyond one-hop transitions. IIRG focuses on collaborative relationships from item co-occurrences across multiple user sequence hops and semantic relationships among thematically similar items. Our results demonstrate that IIRG significantly enhances LLM performance, particularly for users whose test items are not represented by one-hop transitions during training.