4 ms·

Is incompetence fraud? Or just incompetence? I'm asking because a fair number of the molecular biologists who get caught by Elizabeth Bik for copy/pasting ima

by dekhn 2y ago

Is incompetence fraud? Or just incompetence? I'm asking because a fair number of the molecular biologists who get caught by Elizabeth Bik for copy/pasting images of gels insist they just made honest mistakes (with some commentary about the atrocious nature of record-keeping in modern biology). I alter Ionnides's conclusion to be instead: "Roughly 50% of papers in quantitative biological sciences contain at least one error serious enough to invalidate the conclusion" and "Roughly 75% of really interesting papers are missing at least one load-bearing method detail that reproducers must figure out on their own" (my own personal observations of the literature are consistent with these rates; I was always flabbergasted at people who just took Figure 3 as correct).

kelipso 2y ago
There is no one hovering over scientists all the time ready to stick a hot poker in them when they make a mistake or get careless. I was in academia and my impression is there is a reluctance to double and triple check results to make sure they are right as long as the results match your instincts, whether it's time pressure, laziness, bias, or just being human.
- dekhn 2y ago
  At least in my own mental model of publishing a paper (I've published only a few), I'd want my coauthors to stick hot pokers in my if I made a mistake or got careless. But then, my entire thesis was driven by a reproducible Makefile that downloaded the latest results from a supercomputer, re-ran the whole analysis, and wrote the latex necessary (at least partly to avoid making trivial mistakes). It was clear everything I was doing was just getting in the way of publishing high prestige papers.
  - robwwilliams 2y ago
    All too easy to understand your situation. NIH is finally but slowly waking up and is imposing more “onerous” (aka: essential and correct) data management and sharing (DMS) document. Every grant applicant now submits following these guidelines: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-24-175.html https://grants.nih.gov/grants/guide/notice-files/NOT-OD-24-1... Unfortunately, not all NIH institutes understand how to evaluate and moderate this key new policy. Oddly enough the peer reviewers do NOT have access to DMS plans as of this year.
    - IG_Semmelweiss 2y ago
      Is this a process whereby the researcher is forced to submit the thesis (null, etc) of the research, ahead of the study and its findings?
      - robwwilliams 2y ago
        I think you are referring to clinical trial registration. Different idea and process (a QA-like step). The NIH DMS mandates are about the data generated by an award.
llamaimperative 2y ago
> I'm asking because a fair number of the molecular biologists who get caught by Elizabeth Bik for copy/pasting images of gels insist they just made honest mistakes You're talking about (almost certainly) fraudsters denying they committed fraud. The vast majority of non-replicable results have nothing to do with these types of errors, purposeful or not.
kelnos 2y ago
> Is incompetence fraud? Or just incompetence? Fraud requires intent; it's a word that describes what happened, but also the motivations of the people involved. Incompetence doesn't assume any intent at all; it's merely a description of the (lack of) ability of the people involved. Incompetent people can certainly commit fraud (perhaps to try to cover up their incompetence), but that's by no means required. > ...insist they just made honest mistakes If they're lying about that, it's fraud; they're either covering up their unrealized incompetence with fraud, or trying to cover up their intended fraud with protestations of mere incompetence. If they really did make honest mistakes, then it's just garden-variety incompetence. (Or just... mistakes. To me, incompetence is when someone consistently makes mistakes often. One-time or few-time mistakes are just things that happen to people, no matter how good the are at what they do.)
- pfdietz 2y ago
  The legal phrase I like is "knew or should have known". If there is a situation where you should have known something was wrong, it's as bad as if you really knew it was wrong. To hold otherwise incentivizes willful blindness and plausible deniability.
  - dllthomas 2y ago
    I don't think the fact that the law (rightly, I will grant) unified two things when determining whether to punish means that we should always unify those things in our reasoning in other contexts.
- dataviz 2y ago
  People often use incompetence as an excuse for what were actually intentional bad decisions. Never attribute to malice that which is adequately explained by stupidity. Maybe someone was incompetent but also knew they were cutting corners. Should they get a pass because they claim they didn't mean to do it? We should hold people accountable regardless of intent.
  - veunes 2y ago

by dekhn 2y ago

kelipso 2y ago

There is no one hovering over scientists all the time ready to stick a hot poker in them when they make a mistake or get careless. I was in academia and my impression is there is a reluctance to double and triple check results to make sure they are right as long as the results match your instincts, whether it's time pressure, laziness, bias, or just being human.

dekhn 2y ago
At least in my own mental model of publishing a paper (I've published only a few), I'd want my coauthors to stick hot pokers in my if I made a mistake or got careless. But then, my entire thesis was driven by a reproducible Makefile that downloaded the latest results from a supercomputer, re-ran the whole analysis, and wrote the latex necessary (at least partly to avoid making trivial mistakes). It was clear everything I was doing was just getting in the way of publishing high prestige papers.
- robwwilliams 2y ago
  All too easy to understand your situation. NIH is finally but slowly waking up and is imposing more “onerous” (aka: essential and correct) data management and sharing (DMS) document. Every grant applicant now submits following these guidelines: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-24-175.html https://grants.nih.gov/grants/guide/notice-files/NOT-OD-24-1... Unfortunately, not all NIH institutes understand how to evaluate and moderate this key new policy. Oddly enough the peer reviewers do NOT have access to DMS plans as of this year.
  - IG_Semmelweiss 2y ago
    Is this a process whereby the researcher is forced to submit the thesis (null, etc) of the research, ahead of the study and its findings?
    - robwwilliams 2y ago
      I think you are referring to clinical trial registration. Different idea and process (a QA-like step). The NIH DMS mandates are about the data generated by an award.

llamaimperative 2y ago

> I'm asking because a fair number of the molecular biologists who get caught by Elizabeth Bik for copy/pasting images of gels insist they just made honest mistakes You're talking about (almost certainly) fraudsters denying they committed fraud. The vast majority of non-replicable results have nothing to do with these types of errors, purposeful or not.

kelnos 2y ago

> Is incompetence fraud? Or just incompetence? Fraud requires intent; it's a word that describes what happened, but also the motivations of the people involved. Incompetence doesn't assume any intent at all; it's merely a description of the (lack of) ability of the people involved. Incompetent people can certainly commit fraud (perhaps to try to cover up their incompetence), but that's by no means required. > ...insist they just made honest mistakes If they're lying about that, it's fraud; they're either covering up their unrealized incompetence with fraud, or trying to cover up their intended fraud with protestations of mere incompetence. If they really did make honest mistakes, then it's just garden-variety incompetence. (Or just... mistakes. To me, incompetence is when someone consistently makes mistakes often. One-time or few-time mistakes are just things that happen to people, no matter how good the are at what they do.)

pfdietz 2y ago
The legal phrase I like is "knew or should have known". If there is a situation where you should have known something was wrong, it's as bad as if you really knew it was wrong. To hold otherwise incentivizes willful blindness and plausible deniability.
- dllthomas 2y ago
  I don't think the fact that the law (rightly, I will grant) unified two things when determining whether to punish means that we should always unify those things in our reasoning in other contexts.
dataviz 2y ago
People often use incompetence as an excuse for what were actually intentional bad decisions. Never attribute to malice that which is adequately explained by stupidity. Maybe someone was incompetent but also knew they were cutting corners. Should they get a pass because they claim they didn't mean to do it? We should hold people accountable regardless of intent.
- veunes 2y ago