Skip to content
Hallway
0:00 / 0:00

#p-hacking#reproducibilitycrisis#p-value

Is Most Published Research Wrong?

Veritasium

21M subscribers • 25 videos on Hallway

YouTube
  • Recommended by 8 people: 8 on Hacker News.
  • A classic. People kept recommending it from 2017 to 2025.
  • Presented by Derek Muller, creator of Veritasium
  • May be out of date. Uploaded 10 years ago, in an area where tools change fast.

Beginner. No background in the topic needed.

6.2M views10 years ago8 recommendations

Computer scienceAI and machine learningComputation and complexityMath for programmers

Derek Muller explains the statistical and systemic reasons why a large portion of published scientific research findings may be false.

Mounting evidence suggests a lot of published research is false. Check out Audible: bit.ly/AudibleVe Support Veritasium on Patreon: bit.ly/VePatreon Patreon supporters: Bryan Baker, Donal Botkin, Tony Fadell, Jason Buster, Saeed Alghamdi More information on this topic: wke.lt/w/s/z0wmO The Preregistration Challenge: cos.io/prereg/ Resources used in the making of this video: Why Most Published Research Findings Are False: journals.plos.org/plosmedicine/article?id=10.1371/journal.pm… Trouble at the Lab: economist.com/news/briefing/21588057-scientists-think-scienc… Science isn't broken: fivethirtyeight.com/features/science-isnt-broken/#part1 Visual effects by Gustavo Rosa

Why it's here

8 people recommend this

What people say. Recommended as an excellent explanation of p-hacking, publication bias, and the replication crisis in science.

r_singhon Hacker News • Recommends

On “Neural nets more accurate than humans at detecting sexual orientation in images”

If the test data does not have a 50/50 split (or something around that ratio), the headline is straight up lies. Skewed test data is the most common problem with research, reminds me of this great video by Veritasium (Is most published research wrong?): link to this video
View on Hacker News

sillysaurusxon Hacker News • Recommends

On “Actively speaking two languages protects against cognitive impairment”

Cue “p = 0.05 is a random choice” and a big thread about p-hacking. Any takers? It feels like everything to say has already been said about the bias of small sample sizes and the dangers of p-hacking. But it also seems like the discussion is inevitable here. I wonder if there’s a new insight to glean, somehow... For a background on why p-values are misleading, see link to this video which is excellent. He points out that 0.05 was selected by Ronald Fischer in 1925, and people have gone with it since.
View on Hacker News

scotty79on Hacker News • Recommends

On “The Coronavirus Is Here Forever”

Please watch this video. It neatly shows why should you treat small and less rigorous studies with suspicion, even if there are many giving some results: link to this video
View on Hacker News

KK7NILon Hacker News • Recommends

On “I don't think Lindley's paradox supports p-circling”

Wikipedia is infamously bad at teaching math. This Veritasium video does a great job at explaining how such skewed priors can easily appear in our current academic system and the paradox in general: link to this video
View on Hacker News

burntrelish1273on Hacker News • Recommends

On “Receding glacier causes Canadian river to vanish in four days”

Definitely. Another good idea is to show the difference between reality and data can be overwhelmingly significant. Videos demonstrating this with examples: - "The Bayesian Trap" link - "Is Most Research Wrong?" link to this video
View on Hacker News

ReligiousFlameson Hacker News • Recommends

On “Science to Retract Study on Fish and Microplastics”

There's an interesting video by Veratasium "Is Most Published Research Wrong?" (Clickbaity title to be sure.) It shows, on the overall statistics of valid/invalid vs. published/unpublished papers, the arbitrary p-value threshold is wholy inadequate to prevent a sizable fraction of published papers from being false positives (invalid but published). link to this video
View on Hacker News

wolfram74on Hacker News • Recommends

On “The US government is removing scientific data from the Internet”

It's not 'these days' it's the intrinsic nature of false hypotheses outnumbering true hypotheses by an enormous ratio. Even if everything were working perfectly you'd expect false results to outnumber true results. It's like the common example of disease screening, where even if you only get a false positive 1% of the time, but there are only 1% of true positives in your sample population, you're going get many more inaccurate results than accurate results. This video walks through an example calculation and what kinds of steps are being taken to improve on things. link to this video
View on Hacker News

hannasanarionon Hacker News • Recommends

On “Dead sperm whale found in Indonesia had ingested '6kg of plastic'”

My dude. Please go get yourself some education on how science works. I feel embarrassed for you. The reproducibility crisis is about the prevalence of false positive results in the scientific literature. This video is a good primer: link to this video A false positive is when a paper says "this works" when it actually doesn't, because of a statistical fluke in their sample. Even if citing the reproducibility crisis as an excuse to believe something that science has repeatedly found to be false isn't a really bad reason for belief (which it definitely is)... it doesn't even apply because there can't be false positive results regarding neurolinguistic programming because there aren't any positive results…
View on Hacker News

u/ztherionin r/compsci • Mentions

On “What evidence is Turing referring to when he writes this about extrasensory perception?: "These disturbing phenomena seem to deny all our usual scientific ideas. How we should like to discredit them ! Unfortunately the statistical evidence, at least for telepathy, is overwhelming."”

Failed to reproduce. That happens a lot, even with well-designed experiments.
7View on Reddit

DiThion Hacker News • Mentions

On “High refined sugar intake linked to a 23% higher risk of mental disorders”

…videos are old, where most researchers could get away with hacking the P values 0. A lot of studies done in the last decade contradict the older ones (the article I linked earlier is full of sources). Also correlation is not causation. In how many of those studies controlled the amount of carbs? Because the combination is very important 1 0 link to this video 1 link Edit: Looking at the last link of your first video, it acknowledges low carb diets are beneficial short and medium term, but there's very little literature for long term. It concludes low carbs are bad long term... but you must see the table on page 13 to see how they reached that conclusion. The "low carb score" group still had…
View on Hacker News

trapperkeeper74on Hacker News • Mentions

On ““Peer review” is younger than we think”

There are many, many BS papers because of the mass-production death-march researchers are told to produce, low p-value and sample size standards and ”Is Most Published Reseach Wrong?” (Veritasium, 2016) link to this video Which is about an essay from 2005: link This doesn’t mean adversarial and critical feedback are bad, it means dis/incentives, politics and standards need adjustments.
View on Hacker News

ehsankiaon Hacker News • Mentions

On “Fraud Scandals Sap China’s Dream of Becoming a Science Superpower”

I mean, doesn't the west have the exact same issues? p-Hacking, lack of repeat studies, focusing on experiments that will make the biggest headlines, etc. link to this video link
View on Hacker News

nukemandanon Hacker News • Mentions

On “Consumption of a dark roast coffee blend reduces DNA damage in humans”

All I can say... "Is Most Published Research Wrong?" link to this video TL;DR: Likely yes. Spinning statistics is not that hard, even with the best of intentions.
View on Hacker News

sillysaurusxon Hacker News • Mentions

On “'I am mine': Alzheimer's at 41 (2018)”

Yeah, sure. Just don't eat, and that'll prevent alzheimer's. If you study history, you have to reconcile these views with the fact that we've been wrong about every medical development until recent history. At one point we were killing cats to curtail the black plague, when in fact it was the fleas that were the cause. We just don't know. And it's fine not to know. link to this video
View on Hacker News

dharmabon Hacker News • Mentions

On “Ask HN: Where do you get your health news?”

Most published research is wrong: link to this video I usually check the UK NHS website for general information that has been reproduced consistently, and talk to a doctor for anything more specific. This about how poorly technology and law news is reported. Health news is _even worse_.
View on Hacker News

d0mineon Hacker News • Mentions

On “Lancet journal retracts article on hearing aids and dementia after prodding”

> most science withstands the scrutiny If "most" means "most published papers" and "scrutiny" means "can replicate" then no, it doesn't withstand scrutiny: most published research is wrong. link to this video (2016) (the exact percentages of non-replicatable articles may be different in different fields)
View on Hacker News