Everything is a Fight Over the Null Hypothesis
The real battle is over who holds null, and only secondarily over making the empirical case.
The null hypothesis is the default that competing theories are trying to disprove. Science only falsifies claims; it cannot prove them permanently.
In modern science, a result is significant when it’s demonstrated with 95% confidence. This favors the incumbent, which is considered right until it’s disproven to a high standard. The benefit of this approach is that crazy new theories are not accepted just because they happened to get slightly lucky with a few results.
For example, if trying to show that dietary fiber affects health, the null you would disprove is that dietary fiber has no effect (believed until ~1972).
But where does the incumbent come from? Sometimes it’s a long-held theory that has made accurate predictions and withstood the test of time. Other times, though, the rationale is murky. Dietary fiber, which made up 10-15% of ancestral human diets, had to have its effects proven over the course of the 20th century, rather than those stripping it from foods with newly invented industrial processes needing to prove that it was unnecessary.
Here, it was a historical contingency, the result of modern chemistry emerging at the end of the 19th century. Chemically “inert” fiber was thought not to interact with the body and thus must simply take up space (despite its benefits recorded as far back as the ancient physician Galen).
It was only in the 70’s, when Denis Burkitt noticed that African villagers didn’t suffer from common Western gastrointestinal diseases, despite being much poorer, that the benefit of fiber was again appreciated.1
The side that “won” the null hypothesis has the defender’s advantage, but didn’t necessarily pass any evidentiary bar. This means the real battle is over who holds null, and only secondarily over making the empirical case. It’s led to the aphorism heard across economics departments, “everything is a fight over the null.”
Culture is another reason a null hypothesis can win,2 as seen in defamation lawsuits in the US vs the UK. There’s more criticism of public figures in the US because the plaintiff has to prove a defamatory statement is untrue, compared to the UK, where the defendant must prove what they’ve said is true.
US null: “defamatory statement is true.”
UK null: “defamatory statement is false.”
It’s not for empirical reasons that the UK and the US differ. There were no studies in those countries that determined if defamatory statements were more or less likely to be true on average. It’s due to cultural differences where the US prioritizes freedom of speech and limited government, while the UK values the protection of reputation and a regulated commons.
Mercifully, I won’t get further into cultural debates here. Rather, in my next post, I’ll explore epistemological nulls we’ve previously gotten wrong, and how we got there:
Everything is Independent - Assuming events or phenomena are independent of each other and do not interact, even when they do: rogue waves, the 2008 crash, and the wrongful conviction of Sally Clark.
Things are Static - Incorrectly assuming we’re fixed in place, e.g., the continents on the Earth, the Earth in the solar system, and the universe itself.
One of a Kind - It was believed that the Sun was the only “sun” (before ~1850s), and then that our galaxy was the only galaxy (before ~1920s), and then that our solar system’s planets were (nearly) the only planets (before ~1940s).
History is littered with examples, and once you understand them, you’ll start to see them everywhere.
His claims turned out to be somewhat overstated, but he was correct to recognize that fiber has health benefits and is not inert.
For example, the Blank Slate is (ironically) considered a culturally defined null and is more common in certain places and academic departments than in others.