5 ms·

There is a scientific fraud epidemic – and we are ignoring the cure

17 points by johntfella 3y ago

bell-cot 3y ago
Of the players at the table on this one - who really wants a cure?
- geysersam 3y ago
  The public and private interests financing the bulk of the research?
  - bell-cot 3y ago
    "The public" is at best a distant observer. And they seem quite content with "big, shiny, and sounds cool" wastes of $billions, such as the Senate Launch System. And which private interests? Much of what companies give to universities is status marker stuff, where research outcomes don't much matter. And in cases where some WidgetCo supports specific research, which is of practical interest to them - WidgetCo wants honest results for their research, and doesn't much care about fraud at other universities, or in other fields of study.
- mock-possum 3y ago
  People with integrity
  - bell-cot 3y ago
    It would be lovely if more people with absolute integrity (vs. "integrity, but...") were at the decision-making table on this. Sadly, that seems not to be the case. And those actually at the table are very well-entrenched, with strong interests in maintaining the status quo.
eviks 3y ago
> And that, really, is the problem: the lack of due diligence means the rewards for bending or breaking the scientific rules tend to outweigh the incentives to observe them. Well, change the reward: > “publish or perish” culture that rates academics according to how often they are published and cited Who is forcing the legions of supposedly very smart people to treat garbage volume like gold coins while overworking themselves into depression?
- stefantalpalaru 3y ago
  [dead]
- rsfern 3y ago
  Attention (for quality work or not) is the coin of the realm, so why would you expect researchers not to optimize for flashy clickbait over time consuming rigorous self-reflecting work? People have been identifying the incentive structure as a problem for a long time, but it’s not so easy to change because it requires a coordinated effort from the top of the social hierarchy - the folks on hiring/tenure/grant review committees all benefit from the eyeballs game themselves, and they’ve spent their whole careers getting good at it (many of them do rigorous work of course). The journals, which in most fields still coordinate the peer review process, also have perhaps even stronger incentive to optimize for clickbait, to the extent of sometimes letting poor quality work slide. I think it’s a long road that starts with a more frank and open review/publishing system
  - eviks 3y ago
    If they optimized for flashy clickbait, they wouldn't waste so much time on peer review, nor flood the journals no one will ever pay attention to with AI-generated articles
cs702 3y ago
In practice, "peer review" is almost never actually a peer review, but more like a quick pass-through in which unpaid reviewers who have little free time have no choice but to do as little work as possible, filtering out any work that doesn't make sense to them, perhaps making the odd exception for well-known authors with sterling pedigrees. That's why and how so many fraudulent work passes "peer review," while groundbreaking work by real innovators pursuing unpopular research, like Katalin Karikó[a], have to fight like hell to get funding, to get their work published, to get noticed. I imagine the least-worst solution might be in the form of open arXiv-like repositories for papers + code + data, enabling anyone anywhere to study, debate, and replicate the merits of anyone else's work regardless of pedigree. Every finding is tentative unless it is mathematically proven or successfully replicated. --- [a] https://en.wikipedia.org/wiki/Katalin_Karik%C3%B3 https://en.wikipedia.org/wiki/Katalin_Karik%C3%B3

17 points by johntfella 3y ago

bell-cot 3y ago

Of the players at the table on this one - who really wants a cure?

geysersam 3y ago
The public and private interests financing the bulk of the research?
- bell-cot 3y ago
  "The public" is at best a distant observer. And they seem quite content with "big, shiny, and sounds cool" wastes of $billions, such as the Senate Launch System. And which private interests? Much of what companies give to universities is status marker stuff, where research outcomes don't much matter. And in cases where some WidgetCo supports specific research, which is of practical interest to them - WidgetCo wants honest results for their research, and doesn't much care about fraud at other universities, or in other fields of study.
mock-possum 3y ago
People with integrity
- bell-cot 3y ago
  It would be lovely if more people with absolute integrity (vs. "integrity, but...") were at the decision-making table on this. Sadly, that seems not to be the case. And those actually at the table are very well-entrenched, with strong interests in maintaining the status quo.

eviks 3y ago

> And that, really, is the problem: the lack of due diligence means the rewards for bending or breaking the scientific rules tend to outweigh the incentives to observe them. Well, change the reward: > “publish or perish” culture that rates academics according to how often they are published and cited Who is forcing the legions of supposedly very smart people to treat garbage volume like gold coins while overworking themselves into depression?

stefantalpalaru 3y ago
[dead]
rsfern 3y ago
Attention (for quality work or not) is the coin of the realm, so why would you expect researchers not to optimize for flashy clickbait over time consuming rigorous self-reflecting work? People have been identifying the incentive structure as a problem for a long time, but it’s not so easy to change because it requires a coordinated effort from the top of the social hierarchy - the folks on hiring/tenure/grant review committees all benefit from the eyeballs game themselves, and they’ve spent their whole careers getting good at it (many of them do rigorous work of course). The journals, which in most fields still coordinate the peer review process, also have perhaps even stronger incentive to optimize for clickbait, to the extent of sometimes letting poor quality work slide. I think it’s a long road that starts with a more frank and open review/publishing system
- eviks 3y ago
  If they optimized for flashy clickbait, they wouldn't waste so much time on peer review, nor flood the journals no one will ever pay attention to with AI-generated articles

cs702 3y ago

In practice, "peer review" is almost never actually a peer review, but more like a quick pass-through in which unpaid reviewers who have little free time have no choice but to do as little work as possible, filtering out any work that doesn't make sense to them, perhaps making the odd exception for well-known authors with sterling pedigrees. That's why and how so many fraudulent work passes "peer review," while groundbreaking work by real innovators pursuing unpopular research, like Katalin Karikó[a], have to fight like hell to get funding, to get their work published, to get noticed. I imagine the least-worst solution might be in the form of open arXiv-like repositories for papers + code + data, enabling anyone anywhere to study, debate, and replicate the merits of anyone else's work regardless of pedigree. Every finding is tentative unless it is mathematically proven or successfully replicated. --- [a] https://en.wikipedia.org/wiki/Katalin_Karik%C3%B3 https://en.wikipedia.org/wiki/Katalin_Karik%C3%B3