0:00 / 0:00
#hallucinations#next-wordprediction#languagemodels
Why Pretrained Models Hallucinate
Simons Institute for the Theory of Computing
77K subscribers • 30 videos on Hallway • University channel
- One of the most-watched uploads on Simons Institute, a conference or university channel Hallway lists. Nobody has recommended it in the communities yet, so it ranks below community picks.
- Presented by Santosh Vempala, Georgia Institute of Technology
Advanced. Deep dive for people who know the field.
17K views3 weeks ago
AI and machine learningComputer scienceTransformers and LLMsComputation and complexityMath for machine learning
Santosh Vempala analyzes hallucination in language models from a mathematical perspective, showing how next-word prediction training induces hallucinations.
Santosh Vempala (Georgia Institute of Technology) simons.berkeley.edu/talks/santosh-vempala-georgia-institute-… Trustworthy AI: From Hallucinations to Reliable Autonomy We analyze hallucination in language models from a mathematical perspective and find that the statistical pressures of the next-word prediction training pipeline induce hallucinations under fairly general conditions even when trained on clean data. The core proof is a simple reduction from a natural classification task ("fact or hallucination") to a generation/prompt completion task. The talk is based on facts and on work with Adam T. Kalai (STOC 2025) and with Kalai, Ofer Nachum and Eddie Zhang (Nature 2026).Why it's here
A top upload from a listed channelSimons Institute is on Hallway's list of conference and university channels, and this upload is one of its most-watched talks or lectures. Nobody has recommended it in the communities yet, so it ranks below community picks.