6 ms·

The higher-level problem is that there are tons of scientific papers with falsified data and very few people who care about this. When falsified data is discove

by kens 2y ago

The higher-level problem is that there are tons of scientific papers with falsified data and very few people who care about this. When falsified data is discovered, journals are very reluctant to retract the papers. A small number of poorly-supported people examine papers and have found a shocking number of problems. (For instance, Elisabeth Bik, who you should follow: @elisabethbik.bsky.social) My opinion is that the rate of falsified data is a big deal; there should be an order of magnitude more people checking papers for accuracy and much more action taken. This is kind of like the replication crisis in psychology but with more active fraud.

nextos 2y ago
This is why funding replication studies and letting people publish null results and reproductions of important results is fundamental. It will introduce a strong incentive to be honest. Liars will get caught rather quickly. Right now, it often takes decades to uncover fraud.
- ZYbCRq22HbJ2y7 2y ago
  > Right now, it often takes decades to uncover fraud. Where? Many in this thread are talking as if there is monoculture in academia.
- mike_hearn 2y ago
  Unfortunately as you spend more time investigating this problem it becomes clear that replication studies aren't the answer. They're a bandage over the bleeding but don't address the root causes, and would have nearly no impact even if funded at a much larger scale. Because this suggestion comes up in every single HN thread about scientific fraud I eventually wrote an essay on why this is the case: https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science... (press escape to dismiss the banner). If you're really interested in the topic please read it but here's a brief summary: • Replication studies don't solve many of the most common types of scientific fraud. Instead, you just end up replicating the fraud itself. This is usually because the methodology is bad, but if you try to fix the methodology to be scientific the original authors just claim you didn't do a genuine replication. • Many papers can't be replicated by design because the methodology either isn't actually described at all, or doesn't follow logically from the hypothesis. Any attempt would immediately fail after the first five minutes of reading the paper because you wouldn't know what to do. It's not clear what happens to the money if someone gets funded to replicate such a paper. Today it's not a problem because replicators choose which papers to replicate themselves, it's not a systematic requirement. • The idea implicitly assumes that very few researchers are corrupt thus the replicators are unlikely to also be. This isn't the case because replication failures are often due to field-wide problems, meaning replications will be done by the same insiders who benefit from the status quo and who signed off on the bad papers in the first place. This isn't an issue today because the only people who do replication studies are genuinely interested in whether the claims are true, it's not just a procedural way to get grant monies. • Many papers aren't worth replicating because they make trivial claims. If you punish non-replication without fixing the other incentive problems, you'll just pour accelerant on the problem of academics making obvious claims (e.g. the average man would like to be more muscular), and that just replaces one trust destroying problem with another. Replication failure is a symptom not a cause. The cause is systematically bad incentives.
  - boxed 2y ago
    Incentives are themselves dangerous. We should treat incentives like guns. Instead we apply incentives to all manner of problems and are surprised they backfire and destroy everything. You have to give people less incentives and more just time to do their basic job.
    - foldr 2y ago
      I think this hits the nail on the head. Academics have been treated like assembly line workers for decades now. So they’ve collectively learned how to consistently place assembled product on the conveyor belt. The idea that scientific output is a stack of publications is pretty absurd if you think about it for a few minutes. But try telling that to the MBA types who now run universities.
      - mike_hearn 2y ago
        You do need to incentivize something. If you incentivize nothing that's the same thing as an institution not existing and science being done purely as a hobby. You can get some distance that way - it's how science used to work - but the moment you want the structure and funding an institution can provide you must set incentives. Otherwise people could literally just stop turning up for work and still get paid, which is obviously not going to be acceptable to whoever is funding it.
        foldr 2y ago
        I think it’s an interesting feature of current culture that we take it as axiomatic that people need to be ‘incentivized’. I’m not sure I agree. To me that axiom seems to be at the root of a lot of the problems we’re talking about in this thread. (Yes, everyone is subject to incentives in some broad sense, but acknowledging that doesn’t mean that we have to engineer specific incentives as a means to desired outcomes.)
        krisoft 2y ago
- hwillis 2y ago
  That's a tremendously expensive way to detect fraud. There's funding, but also the people replicating need to have at least near the expertise of the original authors, and the rate of false positives is clearly going to be high. Mistakes on the part of the replicator will look the same as scientific fraud. Maybe most worryingly, the negative impact of human studies would be doubled. More than doubled, probably- look at how many people claimed to successfully replicate LK-99. A paper may need to be replicated many time to identify unknown. Maybe first we could just try specifically looking for fraud? Like recording data with 3rd parties (ensuring that falsified data will at least be recorded suspiciously), or a body that looks for fabricated data or photoshopped figures?
ZYbCRq22HbJ2y7 2y ago
> My opinion is that the rate of falsified data is a big deal Have anything that backs that up? Other than what you shared here? I would be very interested in the rate on a per author level, if you have some evidence. Fraud "impact" vs "impact" of article would be interesting as well.
- kens 2y ago
  See, for example, a paper mill that churned out 400 papers with potentially fabricated images: https://www.science.org/content/article/single-paper-mill-appears-have-churned-out-400-papers-sleuths-find https://www.science.org/content/article/single-paper-mill-ap... Influential microbiologist Didier Raoult had 7 papers retracted in 2024 due to faking the ethics approvals on the research. https://www.science.org/content/article/failure-every-level-how-science-sleuths-exposed-massive-ethics-violations-famed-french https://www.science.org/content/article/failure-every-level-... Fazlul Sarkar, a cancer researcher at Wayne State University, had 40 articles retracted after evidence of falsified data and manipulated and duplicated images. https://www.liebertpub.com/doi/10.1089/genbio.2024.29132.ebi https://www.liebertpub.com/doi/10.1089/genbio.2024.29132.ebi Overall, Elisabeth Bik has found thousands of studies that appear to have doctored images. https://www.newyorker.com/science/elements/how-a-sharp-eyed-scientist-became-biologys-image-detective https://www.newyorker.com/science/elements/how-a-sharp-eyed-...
  - someothherguyy 2y ago
    All of those examples have no relative meaning. If there are millions of papers published per year, then 1000 cases over a decade isn't very prevalent (still bad).
    - mike_hearn 2y ago
      Here's some numbers from insiders with relative meaning. https://blog.plan99.net/fake-science-part-i-7e9764571422 https://blog.plan99.net/fake-science-part-i-7e9764571422 ‘It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgement of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine.’ — Marcia Angell 0.04% of papers are retracted. At least 1.9% of papers have duplicate images “suggestive of deliberate manipulation”. About 2.5% of scientists admit to fraud, and they estimate that 10% of other scientists have committed fraud. “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue.” — Richard Horton, editor of the Lancet The statcheck program showed that “half of all published psychology papers…contained at least one p-value that was inconsistent with its test”. The GRIM program showed that of the papers it could verify, around half contained averages that weren’t possible given the sample sizes, and more than 20% contained multiple such inconsistencies. The fact that half of all papers had incorrect data in them is concerning, especially because it seems to match Richard Horton’s intuitive guess at how much science is simply untrue. And the GRIM paper revealed a deeper problem: more than half of the scientists refused to provide the raw data for further checking, even though they had agreed to share it as a condition for being published. After some bloggers exposed an industrial research-faking operation that had generated at least 600 papers about experiments that never happened, a Chinese doctor reached out to beg for mercy: “Hello teacher, yesterday you disclosed that there were some doctors having fraudulent pictures in their papers. This has raised attention. As one of these doctors, I kindly ask you to please leave us alone as soon as possible … Without papers, you don’t get promotion; without a promotion, you can hardly feed your family … You expose us but there are thousands of other people doing the same.“
ProjectArcturis 2y ago
There is no reward for finding falsified data. If there were, we'd find a lot more.
- jessekv 2y ago
  Are you saying publications need a bug bounty?
  - ProjectArcturis 2y ago
    Yes! Labs should escrow $10k for a year for each paper they publish. If anyone finds fraud, they get to keep it.

by kens 2y ago

nextos 2y ago

This is why funding replication studies and letting people publish null results and reproductions of important results is fundamental. It will introduce a strong incentive to be honest. Liars will get caught rather quickly. Right now, it often takes decades to uncover fraud.

ZYbCRq22HbJ2y7 2y ago
> Right now, it often takes decades to uncover fraud. Where? Many in this thread are talking as if there is monoculture in academia.
mike_hearn 2y ago
Unfortunately as you spend more time investigating this problem it becomes clear that replication studies aren't the answer. They're a bandage over the bleeding but don't address the root causes, and would have nearly no impact even if funded at a much larger scale. Because this suggestion comes up in every single HN thread about scientific fraud I eventually wrote an essay on why this is the case: https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science... (press escape to dismiss the banner). If you're really interested in the topic please read it but here's a brief summary: • Replication studies don't solve many of the most common types of scientific fraud. Instead, you just end up replicating the fraud itself. This is usually because the methodology is bad, but if you try to fix the methodology to be scientific the original authors just claim you didn't do a genuine replication. • Many papers can't be replicated by design because the methodology either isn't actually described at all, or doesn't follow logically from the hypothesis. Any attempt would immediately fail after the first five minutes of reading the paper because you wouldn't know what to do. It's not clear what happens to the money if someone gets funded to replicate such a paper. Today it's not a problem because replicators choose which papers to replicate themselves, it's not a systematic requirement. • The idea implicitly assumes that very few researchers are corrupt thus the replicators are unlikely to also be. This isn't the case because replication failures are often due to field-wide problems, meaning replications will be done by the same insiders who benefit from the status quo and who signed off on the bad papers in the first place. This isn't an issue today because the only people who do replication studies are genuinely interested in whether the claims are true, it's not just a procedural way to get grant monies. • Many papers aren't worth replicating because they make trivial claims. If you punish non-replication without fixing the other incentive problems, you'll just pour accelerant on the problem of academics making obvious claims (e.g. the average man would like to be more muscular), and that just replaces one trust destroying problem with another. Replication failure is a symptom not a cause. The cause is systematically bad incentives.
- boxed 2y ago
  Incentives are themselves dangerous. We should treat incentives like guns. Instead we apply incentives to all manner of problems and are surprised they backfire and destroy everything. You have to give people less incentives and more just time to do their basic job.
  - foldr 2y ago
    I think this hits the nail on the head. Academics have been treated like assembly line workers for decades now. So they’ve collectively learned how to consistently place assembled product on the conveyor belt. The idea that scientific output is a stack of publications is pretty absurd if you think about it for a few minutes. But try telling that to the MBA types who now run universities.
    - mike_hearn 2y ago
      You do need to incentivize something. If you incentivize nothing that's the same thing as an institution not existing and science being done purely as a hobby. You can get some distance that way - it's how science used to work - but the moment you want the structure and funding an institution can provide you must set incentives. Otherwise people could literally just stop turning up for work and still get paid, which is obviously not going to be acceptable to whoever is funding it.
      - foldr 2y ago
        I think it’s an interesting feature of current culture that we take it as axiomatic that people need to be ‘incentivized’. I’m not sure I agree. To me that axiom seems to be at the root of a lot of the problems we’re talking about in this thread. (Yes, everyone is subject to incentives in some broad sense, but acknowledging that doesn’t mean that we have to engineer specific incentives as a means to desired outcomes.)
        krisoft 2y ago
hwillis 2y ago
That's a tremendously expensive way to detect fraud. There's funding, but also the people replicating need to have at least near the expertise of the original authors, and the rate of false positives is clearly going to be high. Mistakes on the part of the replicator will look the same as scientific fraud. Maybe most worryingly, the negative impact of human studies would be doubled. More than doubled, probably- look at how many people claimed to successfully replicate LK-99. A paper may need to be replicated many time to identify unknown. Maybe first we could just try specifically looking for fraud? Like recording data with 3rd parties (ensuring that falsified data will at least be recorded suspiciously), or a body that looks for fabricated data or photoshopped figures?

ZYbCRq22HbJ2y7 2y ago

> My opinion is that the rate of falsified data is a big deal Have anything that backs that up? Other than what you shared here? I would be very interested in the rate on a per author level, if you have some evidence. Fraud "impact" vs "impact" of article would be interesting as well.

kens 2y ago
See, for example, a paper mill that churned out 400 papers with potentially fabricated images: https://www.science.org/content/article/single-paper-mill-appears-have-churned-out-400-papers-sleuths-find https://www.science.org/content/article/single-paper-mill-ap... Influential microbiologist Didier Raoult had 7 papers retracted in 2024 due to faking the ethics approvals on the research. https://www.science.org/content/article/failure-every-level-how-science-sleuths-exposed-massive-ethics-violations-famed-french https://www.science.org/content/article/failure-every-level-... Fazlul Sarkar, a cancer researcher at Wayne State University, had 40 articles retracted after evidence of falsified data and manipulated and duplicated images. https://www.liebertpub.com/doi/10.1089/genbio.2024.29132.ebi https://www.liebertpub.com/doi/10.1089/genbio.2024.29132.ebi Overall, Elisabeth Bik has found thousands of studies that appear to have doctored images. https://www.newyorker.com/science/elements/how-a-sharp-eyed-scientist-became-biologys-image-detective https://www.newyorker.com/science/elements/how-a-sharp-eyed-...
- someothherguyy 2y ago
  All of those examples have no relative meaning. If there are millions of papers published per year, then 1000 cases over a decade isn't very prevalent (still bad).
  - mike_hearn 2y ago
    Here's some numbers from insiders with relative meaning. https://blog.plan99.net/fake-science-part-i-7e9764571422 https://blog.plan99.net/fake-science-part-i-7e9764571422 ‘It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgement of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine.’ — Marcia Angell 0.04% of papers are retracted. At least 1.9% of papers have duplicate images “suggestive of deliberate manipulation”. About 2.5% of scientists admit to fraud, and they estimate that 10% of other scientists have committed fraud. “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue.” — Richard Horton, editor of the Lancet The statcheck program showed that “half of all published psychology papers…contained at least one p-value that was inconsistent with its test”. The GRIM program showed that of the papers it could verify, around half contained averages that weren’t possible given the sample sizes, and more than 20% contained multiple such inconsistencies. The fact that half of all papers had incorrect data in them is concerning, especially because it seems to match Richard Horton’s intuitive guess at how much science is simply untrue. And the GRIM paper revealed a deeper problem: more than half of the scientists refused to provide the raw data for further checking, even though they had agreed to share it as a condition for being published. After some bloggers exposed an industrial research-faking operation that had generated at least 600 papers about experiments that never happened, a Chinese doctor reached out to beg for mercy: “Hello teacher, yesterday you disclosed that there were some doctors having fraudulent pictures in their papers. This has raised attention. As one of these doctors, I kindly ask you to please leave us alone as soon as possible … Without papers, you don’t get promotion; without a promotion, you can hardly feed your family … You expose us but there are thousands of other people doing the same.“

ProjectArcturis 2y ago

There is no reward for finding falsified data. If there were, we'd find a lot more.

jessekv 2y ago
Are you saying publications need a bug bounty?
- ProjectArcturis 2y ago
  Yes! Labs should escrow $10k for a year for each paper they publish. If anyone finds fraud, they get to keep it.