3 ms·

How are people supposed to do science without running statistical models?

by dash2 3y ago

mike_hearn 3y ago
This is asked in good faith of course, but that question really gets to the heart of what's been corrupting science. Statistical techniques can be very useful (ChatGPT!) but they aren't by themselves science. Science is about building a theoretical understanding of the natural world, where that theory can be expressed in precise language and used to produce new and novel hypotheses. A big part of why so much science doesn't replicate is that parts of academia have lost sight of that. Downloading government datasets and regressing them against each other isn't actually science even though it's an easy way to get published papers, because it doesn't yield a theoretical understanding of the domain. It often doesn't even let you show causality, let alone the mechanisms behind that causality. If you look at epidemiology, part of why it's lost its way is that it's become dominated by what the media calls "mathematicians"; on HN we'd call them data scientists. Their papers are essentially devoid of theorizing beyond trivial everyday understandings of disease (people get sick and infect each other). Thousands of papers propose new models which are just a simple equation overfitted to a tiny dataset, often just a single city or country. The model's predictions never work but this doesn't invalidate any hypothesis because there weren't any to begin with. How do you even make progress in a field if there's nothing to be refuted or refined? You can fit curves forever and get nowhere. In psychology this problem has at least been recognized. "A problem in theory" discusses it: https://www.nature.com/articles/s41562-018-0522-1 https://www.nature.com/articles/s41562-018-0522-1
- dash2 3y ago
  Right, statistical models are not sufficient for science. I agree. But they are necessary. So I recur to my original question.

How are people supposed to do science without running statistical models?

by dash2 3y ago

How are people supposed to do science without running statistical models?

This is asked in good faith of course, but that question really gets to the heart of what's been corrupting science. Statistical techniques can be very useful (ChatGPT!) but they aren't by themselves science. Science is about building a theoretical understanding of the natural world, where that theory can be expressed in precise language and used to produce new and novel hypotheses. A big part of why so much science doesn't replicate is that parts of academia have lost sight of that. Downloading government datasets and regressing them against each other isn't actually science even though it's an easy way to get published papers, because it doesn't yield a theoretical understanding of the domain. It often doesn't even let you show causality, let alone the mechanisms behind that causality. If you look at epidemiology, part of why it's lost its way is that it's become dominated by what the media calls "mathematicians"; on HN we'd call them data scientists. Their papers are essentially devoid of theorizing beyond trivial everyday understandings of disease (people get sick and infect each other). Thousands of papers propose new models which are just a simple equation overfitted to a tiny dataset, often just a single city or country. The model's predictions never work but this doesn't invalidate any hypothesis because there weren't any to begin with. How do you even make progress in a field if there's nothing to be refuted or refined? You can fit curves forever and get nowhere. In psychology this problem has at least been recognized. "A problem in theory" discusses it: https://www.nature.com/articles/s41562-018-0522-1 https://www.nature.com/articles/s41562-018-0522-1

dash2 3y ago
Right, statistical models are not sufficient for science. I agree. But they are necessary. So I recur to my original question.