EdwardCoffinon Hacker News • Recommends
On “Fitting an elephant with four non-zero parameters”
#freemandyson#enricofermi#overfitting
65K subscribers • 2 videos on Hallway
Beginner. No background in the topic needed.
257K views10 years ago2 recommendations
AI and machine learningComputer scienceClassic machine learningMath for programmers
Freeman Dyson recounts how Enrico Fermi rejected his team's complex physical model, famously quoting John von Neumann on overfitting with too many parameters.
To listen to more of Freeman Dyson’s stories, go to the playlist: youtube.com/playlist?list=PLVV0r6CmEsFzDA6mtmKQEgWfcIu49J4nN Freeman Dyson (1923-2020), who was born in England, moved to Cornell University after graduating from Cambridge University with a BA in Mathematics. He subsequently became a professor and worked on nuclear reactors, solid state physics, ferromagnetism, astrophysics and biology. He published several books and, among other honours, was awarded the Heineman Prize and the Royal Society's Hughes Medal. [Listener: Sam Schweber; date recorded: 1998] TRANSCRIPT: So I was in a high state of enthusiasm and I decided I would go to Chicago and show these results to Fermi and tell him how well we were doing. We wanted to have Fermi's blessing on our efforts as he was really the prime mover in this whole subject, and it was a good opportunity for me to get to know Fermi. Anyway, I arranged through Hans Bethe to go to Chicago and tell him about what we were doing. So I arrived at Chicago and knocked on Fermi's door, and he was very polite. I came in, and he said, 'Yes?' and I showed him the graphs on which our experiment, our theoretical numbers were plotted and Fermi's experimental numbers were plotted, and the agreement was on the whole pretty good. And Fermi hardly looked at these graphs, he just put them on the desk, just glanced at them very briefly and he said, 'I am not very impressed with what you've been doing.' And he said, 'When one does a theoretical calculation, you know, there are two ways of doing it. Either you should have a clear physical model in mind, or you should have a rigorous mathematical basis. You have neither.' So that was it - in about two sentences he disposed of the whole subject. Well then I asked him, well what does he think about the numerical agreement, and he said, 'How many parameters did you use for the fitting? How many free parameters are there in your method?' So I counted up. It turned out there were four. And he said, 'You know, Johnny von Neumann always used to say, "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk." So I don't find the numerical agreement very impressive either.' So I said, 'Thank you very much for you help,' and I said goodbye. There was nothing more to be said. The whole discussion took maybe 10 or 15 minutes. And I came back to Cornell to tell the team the bad news. So that was another watershed in my life, and I think it was profoundly useful what Fermi did. He had this amazing intuition. He could spot what was good and what was bad right away. I mean, we might have worked on these calculations for five years if Fermi hadn't given us the red light and, as it was, Fermi was absolutely right because in the end of course it turned out that the theory on which we based the whole calculation was an illusion. There is really no such thing as a pseudo-scalar theory of pions. In reality 10 years later, or whenever it was, quarWhat people say. Frequently cited by engineers and data scientists as the origin of the famous quote about fitting an elephant with four parameters.
EdwardCoffinon Hacker News • Recommends
On “Fitting an elephant with four non-zero parameters”
Calavaron Hacker News • Recommends
On “Fitting an elephant with four non-zero parameters”
magicalhippoon Hacker News • Mentions
On “Study: Dark matter does not exist and the universe is 27B years old”
kayo_20211030on Hacker News • Mentions
On “What Is a Particle? (2020)”
godelskion Hacker News • Mentions
On “A statistical approach to model evaluations”
godelskion Hacker News • Mentions
On “The impact of competition and DeepSeek on Nvidia”
godelskion Hacker News • Mentions
On “Understanding Reasoning LLMs”
godelskion Hacker News • Mentions
On “ARC-AGI without pretraining”
godelskion Hacker News • Mentions
On “The cultural divide between mathematics and AI”
godelskion Hacker News • Mentions
On “Pen and Paper Exercises in Machine Learning (2022)”
godelskion Hacker News • Mentions
On “The "AI 2027" Scenario: How realistic is it?”
godelskion Hacker News • Mentions
On “Chomsky on what ChatGPT is good for (2023)”
godelskion Hacker News • Mentions
On “Chomsky on what ChatGPT is good for (2023)”
godelskion Hacker News • Mentions
On “Persona vectors: Monitoring and controlling character traits in language models”
godelskion Hacker News • Mentions
On “Andrej Karpathy – It will take a decade to work through the issues with agents”
godelskion Hacker News • Mentions
On “Spatial intelligence is AI’s next frontier”