TokenMinds: Enhancing User Representation in Recommender Systems

In the realm of industrial recommender systems, traditional user modeling often relies on dense embeddings that are limited by their fixed dimensions. A new approach has emerged, utilizing large language models (LLMs) to create text-based user tokens. While this method captures topical relationships, it struggles with grounding these representations to specific item attributes. On the other hand, Semantic ID (SID) tokenization has shown promise in enhancing generative recommendations but has not yet been fully applied to user representations. TokenMinds aims to bridge this gap by expanding the PLUM framework to user modeling, generating both discrete SID-based user tokens and dense embeddings through an encoder-decoder setup derived from pre-trained LLMs. This innovative dual-output mechanism allows for semantically rich user representations while ensuring compatibility with existing models that depend on dense embeddings. Furthermore, the shared SID vocabulary facilitates cross-scenario modeling, effectively merging long and short video behaviors into a unified system, which significantly lowers training and operational costs. Extensive offline testing and real-world applications on various YouTube platforms demonstrate the effectiveness of TokenMinds, confirming the complementary advantages of SID-based user tokens alongside dense embeddings in large-scale ranking systems.