Exploring Negative Sampling Techniques in Information Retrieval

Information Retrieval (IR) plays a crucial role in numerous contemporary natural language processing (NLP) tasks. The advent of dense retrieval (DR) has transformed IR by leveraging neural networks to create semantic vector representations, leading to notable improvements in retrieval effectiveness. A key aspect of training robust dense retrievers is the identification of informative negative samples through contrastive learning. This survey synthesizes insights from 35 influential studies, offering a thorough and current examination of negative sampling strategies in dense IR. A distinctive feature of this work is its emphasis on modern NLP applications and the incorporation of recent methodologies driven by Large Language Models (LLMs), which have not been extensively covered in earlier literature. We introduce a classification system that organizes these techniques into categories such as random sampling, static and dynamic mining, and synthetic dataset generation. Furthermore, we evaluate these methods based on their effectiveness, computational efficiency, and ease of implementation. The survey concludes by highlighting existing challenges and exploring promising avenues for utilizing LLM-generated synthetic data in the future.