4 ms·

Never Trust the Science - On the need to identify bias & interpret data yourself

27 points by Luc 7mo ago

afpx 7mo ago
How about a big vetted database like arxiv of all hypotheses, all proposed experiments to test them, and all experimental results?
- lithocarpus 7mo ago
  Vetted by who? To be clear I'd be very much in favor of scientific studies and their data having to be publicly available. But on any controversial area, which is most of the areas anyone cares about, there will be 2+ sides of the issue and any vetting body will be compromised to some degree for one of those sides.
  - tracker1 7mo ago
    That's the rub, isn't it... who watches the watchmen? In times past, journalism at least had the veil of impartiality, but modern journalism is far more of an editorial activist activity than simply answering the 6 W's of a given story. I'm not sure it was ever actually much better... and it may just be my pessimistic Gen X nature. But I've personally seen too many misrepresentations about too many studies where the body and available data in fact don't match the headlines or the numbers themselves are deceptive in a way that is much less significant than represented. 200% the risk of X... when in sample A of 10000, 1 had X, and in sample b of the same size, 2 had X... while it's a real relative stat, the absolute values are all but meaningless in context.
    - thephyber 7mo ago
      Yellow journalism existed generations before you and I. The institution was always sullied by the worst and has always contained some of the most dogged pestering fact finders. It’s not even clear that journalists of the 1960s-1980s were as impartial or brutally honest as we remember. That is most likely a halo effect from having a few highly trusted very visible personalities (eg. Walter Cronkite), but even they were slow to realize (by a decade) how much of a morass the Vietnam War was.
      - jltsiren 7mo ago
        It was always about independence, not impartiality. Instead of having a big boss on the top issuing correct opinions, reputable news outlets gave their reporters a lot of freedom in their work. Each reporter had their own biases, and the variation within each outlet was usually greater than the variation between outlets.
- habinero 7mo ago
  arxiv is an open-access journal that checks for spam. It is very much not "vetted" lol
- lunatuna 7mo ago
  Ya, I want it even bigger. All commercial claims should be accessible for your own determination. Fastest, biggest, longest, widest, shortest, most liked, doctor recommended, any empirical claim must have the data used and calculations to make the claim available for examination. Data storage is so cheap now. I don't see it as a dent to anyone's profit.
- JR1427 7mo ago
  This would just be impractical. Nothing would ever get done. Too many potential experiments.
RcouF1uZ4gsC 7mo ago
And if a result is surprising to you, you should trust it less and look into it more deeply. And if you do so, one of two good outcomes will hopefully happen: 1. You find the result is bogus 2. You learn something new and update your internal model of the world.
- thephyber 7mo ago
  This ignores a huge blind spot that humans have: confirmation bias. There is no accurate heuristic which is a good short.
like_any_other 7mo ago
> For example, failing to control for obvious confounders in observational data is likely to produce biased results. If we like the direction of this bias, we can do less adjustment for confounders. For example, the study showing that having a white doctor increased mortality of black babies didn't correct for birth weight - once that was done, the effect disappeared (and media interest waned): https://www.wsj.com/opinion/justice-jacksons-incredible-statistic-black-newborns-doctors-math-flaw-mortality-4115ff62 https://www.wsj.com/opinion/justice-jacksons-incredible-stat...
_aavaa_ 7mo ago
> Finally, if you want to simply know which Science™ you can trust, I’d recommend finding and following individuals who repeatedly demonstrate competence in statistical methods and scientific interpretation. So like, the scientists themselves? > If in doubt, read the study critically yourself. I cannot believe the author manages to say this with a straight face. “Hey you average person (with maybe a college degree), go read the original academic paper yourself. Doesn’t matter that you don’t have the background, struggle with basic math (much less statistics), can’t evaluate the claims, and don’t know which questions to ask.” The age of the polymath is long dead, we’re living in The Great Endarkenment. You trust your pilot to do their job, you trust your civil engineers with the bridge you driver over, and the mechanical engineers with the controlled explosions happening in your car, but when it comes to cutting edge scientific articles, here is where you, average Joe, will be able to know better that the experts in the field who specialized in this and do it every day.
- like_any_other 7mo ago
  > You trust your pilot to do their job In this case, the "pilot" (the combined media and researcher science communication system) is deliberately steering the plane into the side of a mountain. A coin flip would do a better job. They've burned their credibility to the ground, and you're trying to repair it by invoking other professions that haven't done so.
  - _aavaa_ 7mo ago
    Okay, I’ll bite. Who exactly is “they”? How have they burned their credibility to the ground? And how does reading scientific papers yourself address this issue if in your telling it was created by people who are no better than a coin flip?
    - like_any_other 7mo ago
      Well, this study and the BBC's reporting on it (ignoring the misleading title) isn't quite as bad as worse-than-coin-flip, but the study from my other comment [1] is worse-than-coin flip - not only did they fail to adjust for birth weight, they even cut out data they didn't like [2]. So "they" varies by field, institute, and researcher. [1] https://news.ycombinator.com/item?id=47431120 https://news.ycombinator.com/item?id=47431120 [2] https://dailycaller.com/2025/03/31/exclusive-researchers-axed-data-point-undermining-narrative-that-white-doctors-are-biased-against-black-babies/ https://dailycaller.com/2025/03/31/exclusive-researchers-axe... (every claim the article makes is backed up by attached FOIA'd documents, so you don't have to take the Daily Caller at their word if you don't trust them)
thephyber 7mo ago
This assumes that everyone is currently sufficiently informed enough to make the same expert observations about methodology affecting bias. This is flatly untrue for the vast majority of the population. And nobody has enough time or desire (or likely money to subscribe to the journals) to read the details of the papers and grok the nuances. Humans think in simple narratives for a reason. We shouldn’t have blind faith in science, but we also shouldn’t have to go back to first principles and do our own version of every experiment. The repeatability crisis is a thing and we know about it. P value hacking is a thing we know about. The problem described in the article is that we shouldn’t believe headlines or short summaries created by writers who aren’t incentivized to add the nuance. And nobody should believe a headline anyway - in addition to necessarily being lossy, for any for profit organization they are likely written by someone other than the writer and probably A/B tested for clicks.
- cyanydeez 7mo ago
  You haven't even mentioned how LLMs can absolutely mimic any opinion anywhere at any time! The trust in Science is about the system the produces it; not a single paper or whatever, but that's being erradicated because of the needle in haystack problem. So even if you think going to original sources makes you safe, think again.
JR1427 7mo ago
Something people often don't consider is the limited resources for doing science - time, money, etc. The positive side to "bias" is intuition. This is where a bias ("I'm pretty sure it'll turn out to work like XYZ, so I'll do this experiment next, rather than getting bogged down in some other area.") massively shortcuts the amount of resources required to come to a scientific conclusion. During my PhD, I made many such shortcuts, following my nose. If I didn't, and tried to do everything objectively, I'd still be optimising buffers, and other such things.

27 points by Luc 7mo ago

afpx 7mo ago

How about a big vetted database like arxiv of all hypotheses, all proposed experiments to test them, and all experimental results?

lithocarpus 7mo ago
Vetted by who? To be clear I'd be very much in favor of scientific studies and their data having to be publicly available. But on any controversial area, which is most of the areas anyone cares about, there will be 2+ sides of the issue and any vetting body will be compromised to some degree for one of those sides.
- tracker1 7mo ago
  That's the rub, isn't it... who watches the watchmen? In times past, journalism at least had the veil of impartiality, but modern journalism is far more of an editorial activist activity than simply answering the 6 W's of a given story. I'm not sure it was ever actually much better... and it may just be my pessimistic Gen X nature. But I've personally seen too many misrepresentations about too many studies where the body and available data in fact don't match the headlines or the numbers themselves are deceptive in a way that is much less significant than represented. 200% the risk of X... when in sample A of 10000, 1 had X, and in sample b of the same size, 2 had X... while it's a real relative stat, the absolute values are all but meaningless in context.
  - thephyber 7mo ago
    Yellow journalism existed generations before you and I. The institution was always sullied by the worst and has always contained some of the most dogged pestering fact finders. It’s not even clear that journalists of the 1960s-1980s were as impartial or brutally honest as we remember. That is most likely a halo effect from having a few highly trusted very visible personalities (eg. Walter Cronkite), but even they were slow to realize (by a decade) how much of a morass the Vietnam War was.
    - jltsiren 7mo ago
      It was always about independence, not impartiality. Instead of having a big boss on the top issuing correct opinions, reputable news outlets gave their reporters a lot of freedom in their work. Each reporter had their own biases, and the variation within each outlet was usually greater than the variation between outlets.
habinero 7mo ago
arxiv is an open-access journal that checks for spam. It is very much not "vetted" lol
lunatuna 7mo ago
Ya, I want it even bigger. All commercial claims should be accessible for your own determination. Fastest, biggest, longest, widest, shortest, most liked, doctor recommended, any empirical claim must have the data used and calculations to make the claim available for examination. Data storage is so cheap now. I don't see it as a dent to anyone's profit.
JR1427 7mo ago
This would just be impractical. Nothing would ever get done. Too many potential experiments.

RcouF1uZ4gsC 7mo ago

And if a result is surprising to you, you should trust it less and look into it more deeply. And if you do so, one of two good outcomes will hopefully happen: 1. You find the result is bogus 2. You learn something new and update your internal model of the world.

thephyber 7mo ago
This ignores a huge blind spot that humans have: confirmation bias. There is no accurate heuristic which is a good short.

like_any_other 7mo ago

> For example, failing to control for obvious confounders in observational data is likely to produce biased results. If we like the direction of this bias, we can do less adjustment for confounders. For example, the study showing that having a white doctor increased mortality of black babies didn't correct for birth weight - once that was done, the effect disappeared (and media interest waned): https://www.wsj.com/opinion/justice-jacksons-incredible-statistic-black-newborns-doctors-math-flaw-mortality-4115ff62 https://www.wsj.com/opinion/justice-jacksons-incredible-stat...

_aavaa_ 7mo ago

> Finally, if you want to simply know which Science™ you can trust, I’d recommend finding and following individuals who repeatedly demonstrate competence in statistical methods and scientific interpretation. So like, the scientists themselves? > If in doubt, read the study critically yourself. I cannot believe the author manages to say this with a straight face. “Hey you average person (with maybe a college degree), go read the original academic paper yourself. Doesn’t matter that you don’t have the background, struggle with basic math (much less statistics), can’t evaluate the claims, and don’t know which questions to ask.” The age of the polymath is long dead, we’re living in The Great Endarkenment. You trust your pilot to do their job, you trust your civil engineers with the bridge you driver over, and the mechanical engineers with the controlled explosions happening in your car, but when it comes to cutting edge scientific articles, here is where you, average Joe, will be able to know better that the experts in the field who specialized in this and do it every day.

like_any_other 7mo ago
> You trust your pilot to do their job In this case, the "pilot" (the combined media and researcher science communication system) is deliberately steering the plane into the side of a mountain. A coin flip would do a better job. They've burned their credibility to the ground, and you're trying to repair it by invoking other professions that haven't done so.
- _aavaa_ 7mo ago
  Okay, I’ll bite. Who exactly is “they”? How have they burned their credibility to the ground? And how does reading scientific papers yourself address this issue if in your telling it was created by people who are no better than a coin flip?
  - like_any_other 7mo ago
    Well, this study and the BBC's reporting on it (ignoring the misleading title) isn't quite as bad as worse-than-coin-flip, but the study from my other comment [1] is worse-than-coin flip - not only did they fail to adjust for birth weight, they even cut out data they didn't like [2]. So "they" varies by field, institute, and researcher. [1] https://news.ycombinator.com/item?id=47431120 https://news.ycombinator.com/item?id=47431120 [2] https://dailycaller.com/2025/03/31/exclusive-researchers-axed-data-point-undermining-narrative-that-white-doctors-are-biased-against-black-babies/ https://dailycaller.com/2025/03/31/exclusive-researchers-axe... (every claim the article makes is backed up by attached FOIA'd documents, so you don't have to take the Daily Caller at their word if you don't trust them)

thephyber 7mo ago

This assumes that everyone is currently sufficiently informed enough to make the same expert observations about methodology affecting bias. This is flatly untrue for the vast majority of the population. And nobody has enough time or desire (or likely money to subscribe to the journals) to read the details of the papers and grok the nuances. Humans think in simple narratives for a reason. We shouldn’t have blind faith in science, but we also shouldn’t have to go back to first principles and do our own version of every experiment. The repeatability crisis is a thing and we know about it. P value hacking is a thing we know about. The problem described in the article is that we shouldn’t believe headlines or short summaries created by writers who aren’t incentivized to add the nuance. And nobody should believe a headline anyway - in addition to necessarily being lossy, for any for profit organization they are likely written by someone other than the writer and probably A/B tested for clicks.

cyanydeez 7mo ago
You haven't even mentioned how LLMs can absolutely mimic any opinion anywhere at any time! The trust in Science is about the system the produces it; not a single paper or whatever, but that's being erradicated because of the needle in haystack problem. So even if you think going to original sources makes you safe, think again.

JR1427 7mo ago

Something people often don't consider is the limited resources for doing science - time, money, etc. The positive side to "bias" is intuition. This is where a bias ("I'm pretty sure it'll turn out to work like XYZ, so I'll do this experiment next, rather than getting bogged down in some other area.") massively shortcuts the amount of resources required to come to a scientific conclusion. During my PhD, I made many such shortcuts, following my nose. If I didn't, and tried to do everything objectively, I'd still be optimising buffers, and other such things.