Live Science7%
Neuroscience findings often can't be replicated — and it's a big problem for what we know about the brain 23%
By RJ Mackenzie14%
7/21/2026, 12:00:00 PM
BS Summary: This article contains 24 faulty reasoning types, including Negativity Bias, Halo Effect, and Overconfidence Bias, with Hasty Generalization as the most egregious example at 13.7% saturation with 169 hits. Analysis detected 1,116 faulty-reasoning hits from 1,238 analyzed words, generating a BS Score of 36.5% and a BS Rank of 23% (16,301 of 21,176 articles). This article is better (less manipulative) than 77.00% of the article peer group.
Neuroscience findings often can't be replicated — and it's a big problem for what we know about the brain
One of the central assumptions of modern neuroscience research is that the brain's shape and structure affect behavior.
Scientists have linked a thicker cortex (the brain's outer layer) to higher intelligence, certain brain wave patterns to better volleyball ability, and higher connectivity between parts of the brain to chess-playing skills.
There's a vast number of these so-called brain-wide association studies (BWAS).
But when experts repeat these studies, they can't replicate the results.
Some of these issues aren't unique to neuroscience.
They begin with what Ellis called the "original sin" of science: Academics are put under huge pressure to publish positive research findings.
But some of these factors do apply specifically to this field.
For example, a landmark study in 2007 found that the brains of kids with attention-deficit/hyperactivity disorder took longer to mature.
But a study published earlier this year found that this link vanished once the different rates of aging between boys and girls were taken into account.
"That's what made the whole house of cards topple," Matthew Albaugh, co-author of the replication paper and a clinical neuroscientist at the University of Vermont, previously told Live Science.
In 2011, researchers at University College London published a study that found that the density of gray matter, the part of the brain where brain cell bodies are found, in several brain areas was linked to participants' number of Facebook friends.
(In 2011, this was a vital social metric.)
A 2015 paper failed to replicate this finding, but the authors of the original paper argued that was because the follow-up study didn't follow the same protocol.
So, in 2019, Genon and her colleagues tried a different type of replication analysis.
They used a vast database and conducted more than 10,000 analyses of MRI scans from hundreds of volunteers.
The median neuroimaging study sample size was around 25, so they divided this larger dataset into smaller ones and then looked for significant associations within each.
If one of these micro-studies reported a significant association between a behavior and a brain structure, the team tried to reach the same findings using a different subset of the data.
Very few of the repeat studies replicated the results of the first.
"It was clear evidence that the replicability of those brain-behavior associations is relatively weak," Genon said.
Several brain studies that could not be reproduced in follow-up work show how problems can creep in.
Each of the original papers has over 500 citations, with the number of citations reflecting how much these studies may influence thinking in the field.
One of the key problems is that in healthy volunteers, variation among individual brains is relatively small, which means you need a large number of participants to detect average differences that tie to behavior, Genon said.
That's similar to genome-wide association studies, which pick up the very tiny effects of individual genes, and they must sample tens or hundreds of thousands of people to find robust effects.
To detect meaningful differences in brain-wide imaging studies, only sample sizes in the thousands would do the trick, a 2022 paper suggested.
Because brain scans are expensive and time-consuming to capture, most studies have relied on much smaller samples.
"It's hard to do exactly what was done in the original studies," said Martin Hebart, a group leader at the Max Planck Institute for Human Cognitive and Brain Sciences in Germany, who was not involved in either the Facebook study or its replication.
Additionally, the behavioral tests used to establish psychological variables can be inconsistent, Genon said.
What's more, some of these studies may be operating on the outdated assumption that small brain regions control complex characteristics like intelligence, she added.
In reality, however, "those types of abilities are usually relatively distributed across the brain," she said.
When researchers analyze multiple small brain areas with that assumption in mind using separate statistical tests, it can raise the risk of producing mirage associations.
Some studies may be flawed at the outset.
A 2017 brain imaging study by researchers at Weill Cornell Medical College explored whether MRI data could be used to stratify patients with depression.
The researchers found that brain activity clustered into four "biotypes" of depression, each of which featured distinct, unusual activity patterns in key brain networks.
But a follow-up study found that the differences among the clusters weren't statistically significant, meaning they could have occurred by chance.
Richard Dinga, a neuroscientist at the Friedrich Schiller University Jena in Germany who worked on the follow-up study, said the main problems lay in its design.
Its statistical approach, he said, essentially guaranteed that significant links would be detected.
That's because of a problem called overfitting, in which a model is so well tuned to one set of data that it struggles to find real patterns in other datasets.
This design correlated 17 clinical features of depression with 30,000 brain imaging features, but Dinga said the authors could have used "30,000 coin flips" and still produced the same strong correlation.
Conor Liston, a psychiatrist at Weill Cornell Medicine and co-author of the original paper, acknowledged the model's susceptibility to overfitting.
"That was not at all our intention, but that was the result," Liston told Live Science.
The results of many studies that share the same issues as these papers will never be replicated and yet will be left to stand.
For one thing, there's no money in replication efforts.
"Grant agencies wouldn't be really excited" to fund a study whose goal was to confirm past work, Genon said.
And even when these studies get done, they aren't as likely to get published when they undermine prior findings.
For Dinga, the way forward is to collect more data.
"That's what solved the same irreproducibility problem in genetics," he said, referring to genome-wide association studies that have produced important findings over the past two decades by using sample sizes that have reached the millions.
But it's also important to improve study design, he added, since bigger datasets won't fix studies with incorrect analysis approaches.
"The methods just don't have a chance to produce something useful," he said.
Some researchers are leveraging large brain imaging datasets.
Martin Hebart, a group leader at the Max Planck Institute for Human Cognitive and Brain Sciences in Germany, and Luca Kämmer, a doctoral student in Hebart's lab, are developing re:vision, a project that is collecting imaging data — the largest initiative yet to capture how the brain responds to visual stimuli.
They plan to make this information available to other researchers, who will then explore whether hypotheses tested in other papers can be replicated in re:vision's dataset.
Hebart and Kämmer said they have already received over a dozen applications for their project.
Reception has been particularly positive among younger researchers in the field.
But convincing senior scientists to retest their data might take more work because they have more to lose if their long-standing finding is ultimately not replicated, Kämmer said.
"Everybody always says that replication is great.
But then when I suggest that their specific studies could be replicated, often they are a bit more skeptical," he told Live Science.
See how much you know about the most complex organ in the human body with our brain quiz!
Speakers
8speakers41%attributed speech729writer words
Voice mapSelect a segment to jump to its words
Selected voice
100%flagged-word coverageLuca Kämmer
58 attributed words11% of attributed speech66% writer coverage
Attribution is sentence-level. Pattern percentages are calculated only from words assigned to that voice.
Loading…
Loading…
Loading…
Analysis
Hover over highlighted words in the article to view the associated bias or fallacy analysis.