4 ms·

> Retractions remain exceedingly rare for scientific papers: Just eight out of every 10,000 are retracted, according to a Retraction Watch database. Reading HN

> Retractions remain exceedingly rare for scientific papers: Just eight out of every 10,000 are retracted, according to a Retraction Watch database. Reading HN, one might get the impression that it's much higher.

ekianjo 3y ago
Its not because most of stuff is not retracted that its valid. Most studies are never replicated...
- wolverine876 3y ago
  What data do you have to support that?
  - ekianjo 3y ago
    first, I have a functioning brain that makes it possible for me to imagine that incentives to cheat and the complete lack of replication process makes it easy to push the balance towards publishing fake results. Then, I am well aware that the peer process is not about replication at all and is mostly useless in its current shape. Then people actually tried it which confirmed everything I mentioned: > This certainly does not only happen in cancer research. In a study from Nature, out of 1,576 researchers, more than 70% have tried and failed to reproduce another scientist’s experiments, and over half failed to reproduce their own. And even though 52% agree there is a reproducibility crisis, only 31% think that failing to reproduce published results means that the result is wrong. Even if replication is not necessary for results to be valid and true, any further work based on these results requires confirmation of valid results beyond the initial study. source: https://vce.usc.edu/weekly-news-profile/the-replication-crisis-issues-in-scientific-research/ https://vce.usc.edu/weekly-news-profile/the-replication-cris... Current Science is mostly cargo cult.
    - bsder 3y ago
      > Current Science is mostly cargo cult. Such statements are mostly cargo cult. Experiments with known results are finicky. See: "Electron Band Structure In Germanium, My Ass" https://pages.cs.wisc.edu/~kovar/hall.html https://pages.cs.wisc.edu/~kovar/hall.html Experiments with unknown results are ridiculously finicky. You would be stunned at the number of experiments that got lucky and hinge on things like "how strongly stoppered the bottle is" (this normally manifests as "give it to grad student so-and-so as he somehow always makes it work even though the rest of you suck"). Unless you actively attempt to replicate an experiment you simply will not trip over this. This isn't to say that there isn't real fraud--there is. However, even when everybody is fully aboveboard and cooperating, replication is ridiculously hard. This isn't limited to science. Any company that tries to replicate a manufacturing line trips over this, too. Suddenly, a whole bunch of variables that you didn't even conceive of affect your results. Go to an injection molding conference and prepare to laugh until you cry over how uncontrolled everything is.
      - ekianjo 3y ago
        I am an engineer so I know very well that industrial replication is far from easy, but that's a whole different topic here. That's why when manufacturing at scale you implement numerous controls for each step to ensure you remain into "whats known" territory. Now, when you publish a discovery in a paper, common sense demands that you are sure that you results are not random, and the only 100% reliable way to do that is by replicating your results. There's no way around that. The fact that most people who publish stuff and the system that allows it never ever do that undermines any credibility in said "science". Hence this is cargo cult. This is typically something that mimics the look of Science by using the scientific method, but it has none of its properties, because the incentives to cheat are obvious and there is nothing in place to prevent them.
        bsder 3y ago
        "Difficult to replicate" does NOT mean "wrong". The fact that your experimental technique sucks is not my problem, thanks. Go get some better grad students. And this is before we start getting into medicine which has both ethical and expense issues with running multiple experiments. Your position is both unrealistic and untenable. Medicine and science would literally grind to a halt. > the incentives to cheat are obvious and there is nothing in place to prevent them. The real issue is that Marc Tessier-Lavigne has caused an entire industry to waste millions (possibly billions) of dollars,d elayed Alzheimer's research for decades and will suffer no real punishment. He's 65 years old and has made a shitload of money based on the fraud. The real question is "How do you punish this in order to disincentivize it?"
andrewflnr 3y ago
What data do you have to support that? Because I certainly don't get that impression at all, and it doesn't seem like anyone else in this thread does either, so do you have some polling, or maybe NLP analysis of comments? Of course you're entitled to your impressions, but so are the rest of us.
- wolverine876 3y ago
  The data is right there at the top of my comment.
  - andrewflnr 3y ago
    You made a claim about an "impression" on HN. You cite data about the actual phenomenon. Interesting.
    - wolverine876 3y ago
      I didn't understand your prior comment. I don't feel the need to backup my impression of HN by doing personal research. I wrote, > Reading HN, one might get the impression that it's much higher [than eight out of every 10,000 are retracted] You wrote, > I certainly don't get that impression at all, and it doesn't seem like anyone else in this thread does either People in the thread seem to disagree with the data, not my impression. They think the fraud rate is much higher than the data implies, which supports my impression.
      - andrewflnr 3y ago
        Are you serious? I can see people say explicitly in replies to you what they actually disagree with. Are you not capable of distinguishing between the claims "this stat is false" and "it's bad that this stat is true", or "this stat doesn't imply what you think it does"?

ekianjo 3y ago

Its not because most of stuff is not retracted that its valid. Most studies are never replicated...

wolverine876 3y ago
What data do you have to support that?
- ekianjo 3y ago
  first, I have a functioning brain that makes it possible for me to imagine that incentives to cheat and the complete lack of replication process makes it easy to push the balance towards publishing fake results. Then, I am well aware that the peer process is not about replication at all and is mostly useless in its current shape. Then people actually tried it which confirmed everything I mentioned: > This certainly does not only happen in cancer research. In a study from Nature, out of 1,576 researchers, more than 70% have tried and failed to reproduce another scientist’s experiments, and over half failed to reproduce their own. And even though 52% agree there is a reproducibility crisis, only 31% think that failing to reproduce published results means that the result is wrong. Even if replication is not necessary for results to be valid and true, any further work based on these results requires confirmation of valid results beyond the initial study. source: https://vce.usc.edu/weekly-news-profile/the-replication-crisis-issues-in-scientific-research/ https://vce.usc.edu/weekly-news-profile/the-replication-cris... Current Science is mostly cargo cult.
  - bsder 3y ago
    > Current Science is mostly cargo cult. Such statements are mostly cargo cult. Experiments with known results are finicky. See: "Electron Band Structure In Germanium, My Ass" https://pages.cs.wisc.edu/~kovar/hall.html https://pages.cs.wisc.edu/~kovar/hall.html Experiments with unknown results are ridiculously finicky. You would be stunned at the number of experiments that got lucky and hinge on things like "how strongly stoppered the bottle is" (this normally manifests as "give it to grad student so-and-so as he somehow always makes it work even though the rest of you suck"). Unless you actively attempt to replicate an experiment you simply will not trip over this. This isn't to say that there isn't real fraud--there is. However, even when everybody is fully aboveboard and cooperating, replication is ridiculously hard. This isn't limited to science. Any company that tries to replicate a manufacturing line trips over this, too. Suddenly, a whole bunch of variables that you didn't even conceive of affect your results. Go to an injection molding conference and prepare to laugh until you cry over how uncontrolled everything is.
    - ekianjo 3y ago
      I am an engineer so I know very well that industrial replication is far from easy, but that's a whole different topic here. That's why when manufacturing at scale you implement numerous controls for each step to ensure you remain into "whats known" territory. Now, when you publish a discovery in a paper, common sense demands that you are sure that you results are not random, and the only 100% reliable way to do that is by replicating your results. There's no way around that. The fact that most people who publish stuff and the system that allows it never ever do that undermines any credibility in said "science". Hence this is cargo cult. This is typically something that mimics the look of Science by using the scientific method, but it has none of its properties, because the incentives to cheat are obvious and there is nothing in place to prevent them.
      - bsder 3y ago
        "Difficult to replicate" does NOT mean "wrong". The fact that your experimental technique sucks is not my problem, thanks. Go get some better grad students. And this is before we start getting into medicine which has both ethical and expense issues with running multiple experiments. Your position is both unrealistic and untenable. Medicine and science would literally grind to a halt. > the incentives to cheat are obvious and there is nothing in place to prevent them. The real issue is that Marc Tessier-Lavigne has caused an entire industry to waste millions (possibly billions) of dollars,d elayed Alzheimer's research for decades and will suffer no real punishment. He's 65 years old and has made a shitload of money based on the fraud. The real question is "How do you punish this in order to disincentivize it?"

andrewflnr 3y ago

What data do you have to support that? Because I certainly don't get that impression at all, and it doesn't seem like anyone else in this thread does either, so do you have some polling, or maybe NLP analysis of comments? Of course you're entitled to your impressions, but so are the rest of us.

wolverine876 3y ago
The data is right there at the top of my comment.
- andrewflnr 3y ago
  You made a claim about an "impression" on HN. You cite data about the actual phenomenon. Interesting.
  - wolverine876 3y ago
    I didn't understand your prior comment. I don't feel the need to backup my impression of HN by doing personal research. I wrote, > Reading HN, one might get the impression that it's much higher [than eight out of every 10,000 are retracted] You wrote, > I certainly don't get that impression at all, and it doesn't seem like anyone else in this thread does either People in the thread seem to disagree with the data, not my impression. They think the fraud rate is much higher than the data implies, which supports my impression.
    - andrewflnr 3y ago
      Are you serious? I can see people say explicitly in replies to you what they actually disagree with. Are you not capable of distinguishing between the claims "this stat is false" and "it's bad that this stat is true", or "this stat doesn't imply what you think it does"?