Why Language Models Hallucinate

Large language models often guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such "hallucinations" persist even in state-of-the-art systems. We analyze this phenomenon from a mathematical perspective and find that the statistical pressures of the next-word prediction training pipeline induce hallucinations and current evaluation procedures reward guessing over acknowledging uncertainty. We also propose "open rubric" evaluations with explicit error penalties, providing a practical path to reliable LLMS by aligning their incentives. The talk will be fact-based, and the speaker will readily admit ignorance. Joint work with Adam T. Kalai, Ofer Nachum and Eddie Zhang.