11 ms·

When a postdoc in my lab committed fraud, I had to face my own culpability

85 points by Stratoscope 3y ago

anatnom 3y ago
The premise of this article is better than its execution. The author even points in the direction of open and honest sharing, saying "others could surely have learned from the mistakes I made". But the only mistake pointed to is "I stopped reviewing his original data". Is that the root cause and only way to prevent these problems? If so, drive the point home more firmly.
- flobosg 3y ago
  The analysis in the article is very shallow and raises more questions than answers. What motivated the postdoc to fabricate the data? What kind of project were they working on? A postdoctoral position can be a perilous journey, especially if the project involved is a high-risk one (which would be my guess, given the author’s urge to get recognized in her area). If they don’t generate enough data and publications within those few years, a postdoc can leave the lab effectively empty-handed. There is a tremendous pressure for productivity, at all costs.
  - shusaku 3y ago
    It was so shallow that to me it read like an editor took an axe to a shaky draft.
- adr1an 3y ago
  While I'd love to see bullet points on takeaways, I wouldn't put the 'load' for the lack of them on the author. Even if he expressed the intention to do such thing and then couldn't deliver... I mean, seems like revictimization in a way. He was a victim (even compares the situation to a theft). In these cases, maybe there's not much you can really do to be safe. It's really up to the other person, who should not be dishonest. That being said, on crucial factor on such behavior is paperianism (a.k.a. publish or perish) and the lack of interest of the biggest journals to publish negative results at all (it could be anything, from short communications to just a database)..
  - light_hue_1 3y ago
    In my neck of science/AI/ML we've been talking about negative results being important as a community for a long time. But it never really happens. It's so much harder to judge negative results than positive results. And it's so incredibly hard to attribute blame. Why was this result negative? Did you screw something up? Something very trivial you don't normally even report on? It's totally possible and that makes negative papers hard to swallow. Anyone can produce an unlimited number of negative results by being incompetent.
    - throwup238 3y ago
      > Anyone can produce an unlimited number of negative results by being incompetent. This is well said, I'm surprised I haven't heard it expressed before. Assuming competence is rarer than incompetence, rewarding negative results will probably just drive the incentives even more to the extremes. Instead of outright fraud, everyone will get to hide behind "oops."
    - renonce 3y ago
      I can see the reason for not rewarding individual negative results, but not for not publishing them at all. Any single positive result is almost certainly the cumulation of lots of work, most of them being negative. You can’t pretend that you arrived at this positive result with zero failed attempts. Why can’t we ask for past negative results along with every positive result, just for the sake of completeness? The authors could have just released a bunch of Jupyter notebooks or whatever original format the negative experiment results are in - why everything in academia has to be written in a certain format where every word is carefully scrutinized to be considered useful?
  - nextos 3y ago
    Another crucial factor is the principal investigator (PI) culture which has, at least in most sciences, turned professors into rent seekers. Nowadays, professors tend to play a middlemen role. Apply for grants, advertise results, and claim credit. Nothing else. Most of the time, they do not come up with ideas, nor care about them or do any of the hard work. Places like the Arc Institute have been born to cut PI middlemen out and get research out of this tar pit.
- renonce 3y ago
  It’s already very nice of the author to reflect and share his past mistakes aloud, as this would be a good start for others to raise the same question towards the science community in general. Pressing the author for detail into this specific mistake that happened years ago is nothing compared to lots of existing labs whose integrity has not been questioned at all.
daemonk 3y ago
“I had depended on the idea that scientific research is a sacred trust” And that is part of the problem. Having gone through a phd, this unspoken religiosity that underlies scientific research where you are expected to be monks for science contributes to stuff like this.
- anigbrowl 3y ago
  But selfless dedication to the truth is actually the standard you want. ISTM a bigger issue is the practice of grading on a curve, which incentivizes cheating; and once a student gets away with that as an undergrad, it's a pattern that's likely to persist. Some architect grads I knew once mentioned hiding their building models in the classroom ceiling in the leadup tp their final exams. When I asked why, they said that there was a non-trivial risk that if you left your model out in the classroom other students would damage it to let a leg up on the competition, and the more ambitious the final project the greater the risk. In my innocence I was quite shocked, but as so often, once you become aware of such behavior you start noticing it in many other contexts.
  - cozzyd 3y ago
    My wife was briefly premed and tells me she suspects someone(s) in orgo lab were hiding reagents from others
    - hiddencost 3y ago
      My mom, in law school, said that students would rip key passages out of the law school books in the library to hurt other students.
  - hiddencost 3y ago
    The incentive structure doesn't support doing science for the purpose of discovering the truth.
- deleted 3y ago
  [deleted]
- 3y ago
RcouF1uZ4gsC 3y ago
> John’s fraud left me feeling personally violated, as if a thief had come to my home and rifled through my drawers and closets. Scientific fraud is, indeed, a kind of theft. Apart from the wasted time and ill-spent money, it is a blow to one’s reputation, and it damages a lab’s credibility. I love how the author perpetrated scientific fraud and is playing the victim. Part of the responsibility of a PI is supervision. The author put their name to the paper and the presentation. It is his responsibility to verify this. Also what kind of lab culture did they create. Was there a super focus on discovery at all costs? Was the author just exploiting the post docs instead of actually properly supervising and mentoring them? It kind of reminds me of the Wells-Fargo CEO who created a culture of high demands and no supervision then acted all surprised and innocent when fraud occurred. I think the people at the top of the hierarchy need to receive more blame, rather than push blame to the overworked, underpaid lowest person on the hierarchy.
- maegul 3y ago
  In the abstract this is an excessive take. The point is that trust is a major component of scientific work and how it functions collectively. A effect being that when that is violated a lot breaks down with a god amount of collateral damage. With increasing complexity in research and pressure to produce and publish more, it’s a growing weakness. Bottom line is that team work and trust are now indelible parts of research in a culture predicated on individual success and contributions. Honestly not sure how science adjusts.
  - RcouF1uZ4gsC 3y ago
    I see it as a symptom of our culture where benefits flow uphill and blame flows downhill. When writing papers and presenting at conferences, the PI was all about how he was an integral part of the research and if you asked why he was presenting instead of the post-doc he would have talked about his responsibilities to supervise and guide. Once the fraud came out, he is just a poor victim who can’t be expected to know the post-doc wasn’t even using the equipment and was making up data completely.
    - 3y ago
deleted 3y ago
[deleted]
alexfromapex 3y ago
So glad these types of conversations are happening. There are often difficult conversations but the fraud in research seems rampant and can cause so much damage to humanity it’s important to guard against.
donatj 3y ago
[flagged]
pseudo0 3y ago
> I had already presented John’s data at several national conferences, and we were writing a paper I thought would have a huge impact on our field. Sounds like this person was part of the problem if their idea of "supervision" was flying around taking credit for their postdoc's fraudulent research while being so out of the loop that they didn't notice the postdoc fabricating data whole cloth.
- RcouF1uZ4gsC 3y ago
  I bet that if the government goes after PIs who take federal money and then run a paper mill with inadequate supervision of postdocs, the situation vastly improves. If you know you can go to jail if your postdocs fabricate data, I bet you will be a lot more thorough in checking all the original data. The problem of scientific research is misaligned incentives. There is tremendous upside for publishing novel ideas and very little downside for fraud, especially for the PI who can pawn it off on the overworked, underpaid post-doc.
  - dmckeon 3y ago
    Rather than an post-hoc investigation model, how about promoting reproducibility in publication, with an expectation that others can run the same study with identical equipment, models, codes, playbooks, etc. For PIs getting government funding, true reproducibility should be a requirement, and other researchers should be funded to report pro or con on the reproducibility of studies, and then ethical lapses could be much more visible. Science based solely on faith in the ethics of other researchers is just religion with more complicated rituals.
    - c0pium 3y ago
      Reproduction is not valued by the community in any way that matters. There’s no prize for reproducing, or failing to reproduce, others work. Getting a grant to do reproduction is much harder than getting the initial grant. Nobody is citing reproduction papers, so their authors can’t get that sweet sweet citation count. We as a society don’t actually value the discipline of science or the pursuit of knowledge. We value being first.
knotadev 3y ago
I must admit I was surprised to see the specific study and falsified research wasn’t discussed. I can appreciate this may have been more of a reflection than reprisal (with remediations no doubt made at the time) but if that was the case, would have been good to read it ‘unredacted’. Had the falsified data not made it out of the lab I would understand this article’s approach - the author clearly uninterested in a fruitless smear campaign - but given “lies” are out there, would have been interesting to see exactly what it was and to what extent the findings proliferated. Maybe the author didn’t want to give any more weight to incorrect information. I can only make assumptions. A challenging and nuanced situation all round no doubt.
- c0pium 3y ago
  The explanation is pretty simple; she’s a coward and seeking closure by unburdening herself without doing the hard thing and being fully transparent about what happened. If you helped build someone’s reputation by not bothering to verify their work and just taking credit for your involvement with them, you have an uncomfortable but undeniable responsibility to be just as strident in getting out the truth.
  - jmcgough 3y ago
    Easy to say that behind a computer screen, but we're in this mess because from a game theory perspective there is no gain to be had and a huge risk to your career and/or finances by publicly naming someone. This also happened several years ago, when there was even less honesty and discussion around this topic.
    - globalnode 3y ago
      sounds to me like the author was naive and idealistic, nothing wrong with that, it hurts everyone when people do the wrong thing by others. maybe what they are doing by publishing the story is trying to warn others not to just take people at face value and be aware of your own biases. its a good lesson. no further naming or shaming required.
nothercastle 3y ago
The PI was asleep at the wheel. When my father worked in a lab he knew who was fabricating data amongst his peers in the other adjacent labs not just his colleagues in the same labs. There was no was no way stop them but he knew and it made for frequent dinner conversation.
- EnigmaFlare 3y ago
  Why no way to stop them? Couldn't he have reported it to the university? Sounds like he was complicit if he knew and did nothing.
  - sgt101 3y ago
    That's all very well, but whistleblowers can find themselves out of a job without getting any result if they aren't in the right place at the right time. What if the OP's dad was of a minority ethnicity in a tenuous position in a racist environment? What if the family were poor with a sick child? What if he was an only parent? What if not everyone has the courage to do the right thing all the time because they are just ordinary people and not the star of a movie? What if they are scared and we don't judge them for that?
    - EnigmaFlare 3y ago
      "Because I like money" is how corrupt practices generally sustain themselves. That doesn't mean the participants aren't complicit.
      - sgt101 3y ago
        There's a big difference between a corporate executive with (say) $2m in the bank that keeps their head down and lets corruption ride, and a college professor with a family that keeps his beak out of things. A really big difference.
        EnigmaFlare 3y ago
bunnie 3y ago
Really minor nit, but the comments in this thread refer to the PI as a 'he' or 'him'. The PI is Rosalind A. Coleman. I did not thoroughly investigate the preferred pronouns of this individual but canonically Rosalind would be assumed a 'she' and the top Google images returned for the name present as female.
pvaldes 3y ago
> I was devastated. What should I do? Require the postdoc to repeat the experiments in her presence could be one of the several solutions available
perihelions 3y ago
I strongly disagree with the other comments. This PI seemed fully involved in, did the appropriate amount of oversight over, their lab's work. They studied the data their postdoc gave them, discussed its implications, and understood it. That's adequate. It's unreasonable to expect any team to be robust against a malicious insider attack—to be able to audit things with sufficient aggressiveness, that they could catch sophisticated liars who manipulate raw instrument data. I don't have a *clue* what the solution is; but, asking ordinary people to turn into FBI detectives is not a viable one. It won't work in the real world, with real humans. It's a fantasy. I sympathize with the outrage in this thread. But, go past your initial hot-takes: would you apply the same standards to yourself? Suppose you're in software, and one of your coworkers is sneaking subtle, obfuscated security bugs into your production code. Is it your fault if you fail to catch that? Would you consider yourself culpable if you did a code review and failed to notice some "Underhanded C Contest"-tier sophisticated backdoor? Because IMHO that is utterly impossible; and I think it's similarly impossible to catch data manipulation, reliably, if it's sophisticated enough. Solutions need to be grounded in what's achievable.
- flobosg 3y ago
  > that they could catch sophisticated liars who manipulate raw instrument data There was no raw data manipulation, but fabrication of derived data. These kind of too-good-to-be-true results are easily auditable, if anything by replicating the experiment. The PI’s blind enthusiasm prevented her to do her due diligence. > would you apply the same standards to yourself? While I was in academia I checked basically every kind of raw biological data I was getting. Routine QC allowed me, for instance, to find contaminants in some of the reagents the lab was using.
  - achileas 3y ago
    For many labs, there are no resources, time, or ability to rerun every bit of data collection that happens. When I was in academia, our lab had a hard enough time recruiting participants (we were a cog neuro lab) and our experiments were several hours long per participant with enough individual variation to be useless without several months of data collection and analysis to find any sort of signal in the noisy data. And that’s a single lab, with PIs overseeing several and collaborating with even more, an element of trust is going to be needed if anything is to ever get done.
2devnull 3y ago
“I had depended on the idea that scientific research is a sacred trust, not only to the other individuals in one’s group, but also to the research enterprise as a whole.” The concern of the author is ultimately one of status, preeminence. Of course people motivated by prestige will not always be trustworthy. Her sin isn’t being too trusting, it’s a lack of self awareness.

85 points by Stratoscope 3y ago

The premise of this article is better than its execution. The author even points in the direction of open and honest sharing, saying "others could surely have learned from the mistakes I made". But the only mistake pointed to is "I stopped reviewing his original data". Is that the root cause and only way to prevent these problems? If so, drive the point home more firmly.

flobosg 3y ago
The analysis in the article is very shallow and raises more questions than answers. What motivated the postdoc to fabricate the data? What kind of project were they working on? A postdoctoral position can be a perilous journey, especially if the project involved is a high-risk one (which would be my guess, given the author’s urge to get recognized in her area). If they don’t generate enough data and publications within those few years, a postdoc can leave the lab effectively empty-handed. There is a tremendous pressure for productivity, at all costs.
- shusaku 3y ago
  It was so shallow that to me it read like an editor took an axe to a shaky draft.
adr1an 3y ago
While I'd love to see bullet points on takeaways, I wouldn't put the 'load' for the lack of them on the author. Even if he expressed the intention to do such thing and then couldn't deliver... I mean, seems like revictimization in a way. He was a victim (even compares the situation to a theft). In these cases, maybe there's not much you can really do to be safe. It's really up to the other person, who should not be dishonest. That being said, on crucial factor on such behavior is paperianism (a.k.a. publish or perish) and the lack of interest of the biggest journals to publish negative results at all (it could be anything, from short communications to just a database)..
- light_hue_1 3y ago
  In my neck of science/AI/ML we've been talking about negative results being important as a community for a long time. But it never really happens. It's so much harder to judge negative results than positive results. And it's so incredibly hard to attribute blame. Why was this result negative? Did you screw something up? Something very trivial you don't normally even report on? It's totally possible and that makes negative papers hard to swallow. Anyone can produce an unlimited number of negative results by being incompetent.
  - throwup238 3y ago
    > Anyone can produce an unlimited number of negative results by being incompetent. This is well said, I'm surprised I haven't heard it expressed before. Assuming competence is rarer than incompetence, rewarding negative results will probably just drive the incentives even more to the extremes. Instead of outright fraud, everyone will get to hide behind "oops."
  - renonce 3y ago
    I can see the reason for not rewarding individual negative results, but not for not publishing them at all. Any single positive result is almost certainly the cumulation of lots of work, most of them being negative. You can’t pretend that you arrived at this positive result with zero failed attempts. Why can’t we ask for past negative results along with every positive result, just for the sake of completeness? The authors could have just released a bunch of Jupyter notebooks or whatever original format the negative experiment results are in - why everything in academia has to be written in a certain format where every word is carefully scrutinized to be considered useful?
- nextos 3y ago
  Another crucial factor is the principal investigator (PI) culture which has, at least in most sciences, turned professors into rent seekers. Nowadays, professors tend to play a middlemen role. Apply for grants, advertise results, and claim credit. Nothing else. Most of the time, they do not come up with ideas, nor care about them or do any of the hard work. Places like the Arc Institute have been born to cut PI middlemen out and get research out of this tar pit.
renonce 3y ago
It’s already very nice of the author to reflect and share his past mistakes aloud, as this would be a good start for others to raise the same question towards the science community in general. Pressing the author for detail into this specific mistake that happened years ago is nothing compared to lots of existing labs whose integrity has not been questioned at all.

daemonk 3y ago

“I had depended on the idea that scientific research is a sacred trust” And that is part of the problem. Having gone through a phd, this unspoken religiosity that underlies scientific research where you are expected to be monks for science contributes to stuff like this.

anigbrowl 3y ago
But selfless dedication to the truth is actually the standard you want. ISTM a bigger issue is the practice of grading on a curve, which incentivizes cheating; and once a student gets away with that as an undergrad, it's a pattern that's likely to persist. Some architect grads I knew once mentioned hiding their building models in the classroom ceiling in the leadup tp their final exams. When I asked why, they said that there was a non-trivial risk that if you left your model out in the classroom other students would damage it to let a leg up on the competition, and the more ambitious the final project the greater the risk. In my innocence I was quite shocked, but as so often, once you become aware of such behavior you start noticing it in many other contexts.
- cozzyd 3y ago
  My wife was briefly premed and tells me she suspects someone(s) in orgo lab were hiding reagents from others
  - hiddencost 3y ago
    My mom, in law school, said that students would rip key passages out of the law school books in the library to hurt other students.
- hiddencost 3y ago
  The incentive structure doesn't support doing science for the purpose of discovering the truth.
deleted 3y ago
[deleted]
3y ago

RcouF1uZ4gsC 3y ago

> John’s fraud left me feeling personally violated, as if a thief had come to my home and rifled through my drawers and closets. Scientific fraud is, indeed, a kind of theft. Apart from the wasted time and ill-spent money, it is a blow to one’s reputation, and it damages a lab’s credibility. I love how the author perpetrated scientific fraud and is playing the victim. Part of the responsibility of a PI is supervision. The author put their name to the paper and the presentation. It is his responsibility to verify this. Also what kind of lab culture did they create. Was there a super focus on discovery at all costs? Was the author just exploiting the post docs instead of actually properly supervising and mentoring them? It kind of reminds me of the Wells-Fargo CEO who created a culture of high demands and no supervision then acted all surprised and innocent when fraud occurred. I think the people at the top of the hierarchy need to receive more blame, rather than push blame to the overworked, underpaid lowest person on the hierarchy.

maegul 3y ago
In the abstract this is an excessive take. The point is that trust is a major component of scientific work and how it functions collectively. A effect being that when that is violated a lot breaks down with a god amount of collateral damage. With increasing complexity in research and pressure to produce and publish more, it’s a growing weakness. Bottom line is that team work and trust are now indelible parts of research in a culture predicated on individual success and contributions. Honestly not sure how science adjusts.
- RcouF1uZ4gsC 3y ago
  I see it as a symptom of our culture where benefits flow uphill and blame flows downhill. When writing papers and presenting at conferences, the PI was all about how he was an integral part of the research and if you asked why he was presenting instead of the post-doc he would have talked about his responsibilities to supervise and guide. Once the fraud came out, he is just a poor victim who can’t be expected to know the post-doc wasn’t even using the equipment and was making up data completely.
  - 3y ago

deleted 3y ago

[deleted]

alexfromapex 3y ago

So glad these types of conversations are happening. There are often difficult conversations but the fraud in research seems rampant and can cause so much damage to humanity it’s important to guard against.

donatj 3y ago

[flagged]

pseudo0 3y ago

> I had already presented John’s data at several national conferences, and we were writing a paper I thought would have a huge impact on our field. Sounds like this person was part of the problem if their idea of "supervision" was flying around taking credit for their postdoc's fraudulent research while being so out of the loop that they didn't notice the postdoc fabricating data whole cloth.

RcouF1uZ4gsC 3y ago
I bet that if the government goes after PIs who take federal money and then run a paper mill with inadequate supervision of postdocs, the situation vastly improves. If you know you can go to jail if your postdocs fabricate data, I bet you will be a lot more thorough in checking all the original data. The problem of scientific research is misaligned incentives. There is tremendous upside for publishing novel ideas and very little downside for fraud, especially for the PI who can pawn it off on the overworked, underpaid post-doc.
- dmckeon 3y ago
  Rather than an post-hoc investigation model, how about promoting reproducibility in publication, with an expectation that others can run the same study with identical equipment, models, codes, playbooks, etc. For PIs getting government funding, true reproducibility should be a requirement, and other researchers should be funded to report pro or con on the reproducibility of studies, and then ethical lapses could be much more visible. Science based solely on faith in the ethics of other researchers is just religion with more complicated rituals.
  - c0pium 3y ago
    Reproduction is not valued by the community in any way that matters. There’s no prize for reproducing, or failing to reproduce, others work. Getting a grant to do reproduction is much harder than getting the initial grant. Nobody is citing reproduction papers, so their authors can’t get that sweet sweet citation count. We as a society don’t actually value the discipline of science or the pursuit of knowledge. We value being first.

knotadev 3y ago

I must admit I was surprised to see the specific study and falsified research wasn’t discussed. I can appreciate this may have been more of a reflection than reprisal (with remediations no doubt made at the time) but if that was the case, would have been good to read it ‘unredacted’. Had the falsified data not made it out of the lab I would understand this article’s approach - the author clearly uninterested in a fruitless smear campaign - but given “lies” are out there, would have been interesting to see exactly what it was and to what extent the findings proliferated. Maybe the author didn’t want to give any more weight to incorrect information. I can only make assumptions. A challenging and nuanced situation all round no doubt.

c0pium 3y ago
The explanation is pretty simple; she’s a coward and seeking closure by unburdening herself without doing the hard thing and being fully transparent about what happened. If you helped build someone’s reputation by not bothering to verify their work and just taking credit for your involvement with them, you have an uncomfortable but undeniable responsibility to be just as strident in getting out the truth.
- jmcgough 3y ago
  Easy to say that behind a computer screen, but we're in this mess because from a game theory perspective there is no gain to be had and a huge risk to your career and/or finances by publicly naming someone. This also happened several years ago, when there was even less honesty and discussion around this topic.
  - globalnode 3y ago
    sounds to me like the author was naive and idealistic, nothing wrong with that, it hurts everyone when people do the wrong thing by others. maybe what they are doing by publishing the story is trying to warn others not to just take people at face value and be aware of your own biases. its a good lesson. no further naming or shaming required.

nothercastle 3y ago

The PI was asleep at the wheel. When my father worked in a lab he knew who was fabricating data amongst his peers in the other adjacent labs not just his colleagues in the same labs. There was no was no way stop them but he knew and it made for frequent dinner conversation.

EnigmaFlare 3y ago
Why no way to stop them? Couldn't he have reported it to the university? Sounds like he was complicit if he knew and did nothing.
- sgt101 3y ago
  That's all very well, but whistleblowers can find themselves out of a job without getting any result if they aren't in the right place at the right time. What if the OP's dad was of a minority ethnicity in a tenuous position in a racist environment? What if the family were poor with a sick child? What if he was an only parent? What if not everyone has the courage to do the right thing all the time because they are just ordinary people and not the star of a movie? What if they are scared and we don't judge them for that?
  - EnigmaFlare 3y ago
    "Because I like money" is how corrupt practices generally sustain themselves. That doesn't mean the participants aren't complicit.
    - sgt101 3y ago
      There's a big difference between a corporate executive with (say) $2m in the bank that keeps their head down and lets corruption ride, and a college professor with a family that keeps his beak out of things. A really big difference.
      - EnigmaFlare 3y ago

bunnie 3y ago

Really minor nit, but the comments in this thread refer to the PI as a 'he' or 'him'. The PI is Rosalind A. Coleman. I did not thoroughly investigate the preferred pronouns of this individual but canonically Rosalind would be assumed a 'she' and the top Google images returned for the name present as female.

pvaldes 3y ago

> I was devastated. What should I do? Require the postdoc to repeat the experiments in her presence could be one of the several solutions available

perihelions 3y ago

I strongly disagree with the other comments. This PI seemed fully involved in, did the appropriate amount of oversight over, their lab's work. They studied the data their postdoc gave them, discussed its implications, and understood it. That's adequate. It's unreasonable to expect any team to be robust against a malicious insider attack—to be able to audit things with sufficient aggressiveness, that they could catch sophisticated liars who manipulate raw instrument data. I don't have a *clue* what the solution is; but, asking ordinary people to turn into FBI detectives is not a viable one. It won't work in the real world, with real humans. It's a fantasy. I sympathize with the outrage in this thread. But, go past your initial hot-takes: would you apply the same standards to yourself? Suppose you're in software, and one of your coworkers is sneaking subtle, obfuscated security bugs into your production code. Is it your fault if you fail to catch that? Would you consider yourself culpable if you did a code review and failed to notice some "Underhanded C Contest"-tier sophisticated backdoor? Because IMHO that is utterly impossible; and I think it's similarly impossible to catch data manipulation, reliably, if it's sophisticated enough. Solutions need to be grounded in what's achievable.

flobosg 3y ago
> that they could catch sophisticated liars who manipulate raw instrument data There was no raw data manipulation, but fabrication of derived data. These kind of too-good-to-be-true results are easily auditable, if anything by replicating the experiment. The PI’s blind enthusiasm prevented her to do her due diligence. > would you apply the same standards to yourself? While I was in academia I checked basically every kind of raw biological data I was getting. Routine QC allowed me, for instance, to find contaminants in some of the reagents the lab was using.
- achileas 3y ago
  For many labs, there are no resources, time, or ability to rerun every bit of data collection that happens. When I was in academia, our lab had a hard enough time recruiting participants (we were a cog neuro lab) and our experiments were several hours long per participant with enough individual variation to be useless without several months of data collection and analysis to find any sort of signal in the noisy data. And that’s a single lab, with PIs overseeing several and collaborating with even more, an element of trust is going to be needed if anything is to ever get done.

2devnull 3y ago

“I had depended on the idea that scientific research is a sacred trust, not only to the other individuals in one’s group, but also to the research enterprise as a whole.” The concern of the author is ultimately one of status, preeminence. Of course people motivated by prestige will not always be trustworthy. Her sin isn’t being too trusting, it’s a lack of self awareness.