Investigating Biases in Large Language Models for AI Search Manipulation

Large language models (LLMs) have become integral to various business applications, particularly in enhancing web search functionalities. These models are utilized in LLM Overview systems, which sift through search results to identify the most pertinent sources and formulate responses to user inquiries. Research indicates that LLMs exhibit various biases, which can influence both the selection of sources and the generation of answers in these systems. This paper primarily focuses on the selection phase, exploring how biases can be exploited to manipulate outcomes in LLM Overview systems. To investigate this, we developed a compact language model trained through reinforcement learning to modify search snippets, thereby increasing their appeal to LLM Overviews. Our experimental design constrains the model to operate solely on snippets while limiting reward manipulation tactics, mirroring the practical constraints found in web search contexts. Findings reveal that biases exist within LLM Overview systems, and reinforcement learning can often enhance snippet content to sway results. Additionally, we discovered that selections are more influenced by comparative advantages among sources rather than absolute merits. We also assess the safety implications of manipulating LLM Overviews, highlighting that context poisoning can yield misleading or detrimental outcomes.