4 ms·

The problem is we don't know how many of those other lines of evidence are also fraudulent. What we do know is that finding academic fraud is like shooting fis

The problem is we don't know how many of those other lines of evidence are also fraudulent. What we do know is that finding academic fraud is like shooting fish in a barrel. It's way too easy even for totally unfunded volunteers. The situation in science right now is reminiscent of the situation in computer security circa 1999. Back then there were so many easily exploited RCE vulns in foundational infrastructure that "hacked by teenagers" became a TV trope. Almost anyone could sit down and find a way to hack into a network without much effort. Vendors engaged in a lot of screaming and denial, sometimes even attacking the people who found exploits for them. But, the industry changed. White hats became widespread, companies hired those ex-teenage hackers in their thousands. They put every employee through trainings. Then when that wasn't enough they started paying out millions in bug bounties, buying sandboxing companies (like Google did for Chrome) and more. Companies did this because bad security was hurting their reputations and in a competitive market that is an exploitable weakness. It was a form of self defense. Academic science is excluded from market mechanisms by legal fiat, supposedly on the basis that this will yield better research outcomes. What it's actually done is shield universities from any need to make hard changes. Their reputation is tanking but they just don't seem to care and why should they? They'll get grants from the NIH anyway because they're all as bad as each other, and nobody in politics is talking about a total defunding of the sector yet. Unfortunately even if the grants were cut off tomorrow they probably would still find it hard to change. It's becoming apparent that if a culture tolerates fraud for decades then eventually the leadership of these institutions are people who got there through fraud. How are universities or journals going to reform when the people who run them know full well they can't crack down on fraud without exposing their own career to audit risk?

dbspin 2y ago
You've diagnosed the issue then recommended leaches and bleeding. 'Market mechanisms' already exist - no company is precluded from doing research. We can discuss the deficit in research investment that short termist 'market mechanisms' have led to in contemporary corporations, but its absurd to claim that the market can or should replace academic research. It's pretty clear that in practice the 'publish or perish' academic culture fostered by the financialisation of the university, the rise of the adjunct and the death of tenure have contributed heavily to the falsification of research. In practice encouraging replication, self directed research and significantly more 'pure' 'blue sky' research than governments and universities are inclined to do, would significantly alleviate this problem. Academia needs far less exposure to 'market mechanisms'. They're perverse incentives for good science.
- mike_hearn 2y ago
  To say there's a deficit in research spending pre-supposes there's a correct absolute level of resource allocation that is easily discoverable. But the reason we need markets in the first place is that governments aren't able to discover the right levels of resource allocation on their own. That's why the USSR ended up with lots of steel foundries but no software companies, nor even any competitor to the internet. If anything research is clearly over-funded right now, especially in some fields like medical research. That's why so many scientists turn to fraud in desperation. There just aren't enough genuine discoveries to go round. > encouraging replication, self directed research and significantly more 'pure' 'blue sky' research Replication is the opposite of self directed blue sky research, so how do you suggest it be encouraged whilst simultaneously reducing the power of the money givers even further? BTW, many bogus papers will replicate whilst still being wrong. Replicability is not a synonym for good science. Really, this is a very common claim that needs to go away. The solution to fraud and incompetence is not to give incompetent fraudsters even more freedom and money.
  - nyssos 2y ago
    > But the reason we need markets in the first place is that governments aren't able to discover the right levels of resource allocation on their own. Markets don't discover "the right" (i.e. social welfare maximizing) resource allocation, they discover, under ideal conditions, a pareto-optimal allocation. This is a very weak condition: "Jeff Bezos owns everything" is pareto-optimal. You need redistribution if you want to turn that into a socially optimal one. In realistic conditions, they don't even do that: market mechanisms will reliably underinvest in things with positive externalities, and the larger the externality the greater the degree of misallocation. The positive externalities of research are enormous compared to the value captured by the researchers.
    - mike_hearn 2y ago
      No market system anywhere ends up with one person owning everything. That is the standard under non-market systems, though. > The positive externalities of research ... are only relevant if it's actually research, and not something that looks a lot like research without actually being so.
mistermann 2y ago
> Academic science is excluded from market mechanisms by legal fiat... And a world class propaganda game. Go onto mainstream social media, pick a science fan at random out of the crowd and pick an argument with them about science. The average fan would have you believe that science is just short of perfect. In a sense it's hard to blame these people, considering what two years of the media machine's full court press of pro-science culture war propaganda during a global pandemic does to a mind of average intelligence. I mean, I suppose "winning" is important folks, but is losing your soul in the process worth it?
pocketsand 2y ago
> Academic science is excluded from market mechanisms by legal fiat. This to me just seems untrue. What is your basis for this claim? There is plenty of research privately funded by corporations, some of which is very influential. Often this work is published by university researchers. Ask any university researcher about the numerous compliance courses we all have to take about funding and conflict of interest. It is true that the biggest funders (NSF, NIH) are not market-focused, but for good reason. The market does not prioritize the public good. I know first hand -- my son has a rare disease (1 out of 20,000 people). There are many drug companies putting vast resources into drug development in the hopes of a huge payoff. In reality this benefits a small number of people (I remain grateful for how improvements have helped us). I'm grateful our major scientific funding bodies are not swayed entirely by market influences because it would lead to us focusing on a narrow set of scientific problems which would ultimately limit the way it helps the public good. Im any event, I work in biomedical research. I think your diagnosis (incentives, process) is correct, but the way you discuss the attitudes and motives of researchers is wrong-headed. You say: > Their reputation is tanking but they just don't seem to care and why should they? They'll get grants from the NIH anyway because they're all as bad as each other, and nobody in politics is talking about a total defunding of the sector yet. You're talking about hundreds of thousands of researchers as if they're all psychotic citation fanatics with no care for truth. That is not reality. I think the kind of psychotic, data-manipulating researcher who would put people's health and lives at risk for citations -- or fabricate data sets out of thin air -- are vanishingly small. We can point to a handful of them -- the author of this paper, and the Daniel Ariely's and Francesca Gino's of the world -- but there are tens of thousands of people in every field working on research in good faith, with utmost care. The vast, vast majority never have any scandal, never get caught up in data manipulation, and so on. No field I know of out-right tolerates fraud (and I follow all the retraction stories fairly closely). I think the closest we get to "toleration" is researchers dealing with scientific problems who more or less say "we're not going to publicly flay you but behind your back we're all going to know what you did and your future is limited when it comes to big grants, prestigious invitations, and so on." People who are credibly accused of fraud become pariahs and often targets of scorn not only within the research community but in the press and wider community. The most serious issue IMO is not outright but poor norms around scientific practice, leading to p-hacking, harking, and other "forking paths" problems. Calling that type of behavior "fraudulent" is perhaps justifiable under some ways of thinking, but I think the word fraud mischaracterizes what is going on. There are, in fact, many serious efforts to root out this type of behavior and put in transparency rules to open up research to scrutiny, including among funders like the NIH.
Log_out_ 2y ago
Maybe it should become best practice to publish honeypots. Anyone who claims to reproduce these papers or cites thsm gets autolisted for fraud.

dbspin 2y ago

You've diagnosed the issue then recommended leaches and bleeding. 'Market mechanisms' already exist - no company is precluded from doing research. We can discuss the deficit in research investment that short termist 'market mechanisms' have led to in contemporary corporations, but its absurd to claim that the market can or should replace academic research. It's pretty clear that in practice the 'publish or perish' academic culture fostered by the financialisation of the university, the rise of the adjunct and the death of tenure have contributed heavily to the falsification of research. In practice encouraging replication, self directed research and significantly more 'pure' 'blue sky' research than governments and universities are inclined to do, would significantly alleviate this problem. Academia needs far less exposure to 'market mechanisms'. They're perverse incentives for good science.

mike_hearn 2y ago
To say there's a deficit in research spending pre-supposes there's a correct absolute level of resource allocation that is easily discoverable. But the reason we need markets in the first place is that governments aren't able to discover the right levels of resource allocation on their own. That's why the USSR ended up with lots of steel foundries but no software companies, nor even any competitor to the internet. If anything research is clearly over-funded right now, especially in some fields like medical research. That's why so many scientists turn to fraud in desperation. There just aren't enough genuine discoveries to go round. > encouraging replication, self directed research and significantly more 'pure' 'blue sky' research Replication is the opposite of self directed blue sky research, so how do you suggest it be encouraged whilst simultaneously reducing the power of the money givers even further? BTW, many bogus papers will replicate whilst still being wrong. Replicability is not a synonym for good science. Really, this is a very common claim that needs to go away. The solution to fraud and incompetence is not to give incompetent fraudsters even more freedom and money.
- nyssos 2y ago
  > But the reason we need markets in the first place is that governments aren't able to discover the right levels of resource allocation on their own. Markets don't discover "the right" (i.e. social welfare maximizing) resource allocation, they discover, under ideal conditions, a pareto-optimal allocation. This is a very weak condition: "Jeff Bezos owns everything" is pareto-optimal. You need redistribution if you want to turn that into a socially optimal one. In realistic conditions, they don't even do that: market mechanisms will reliably underinvest in things with positive externalities, and the larger the externality the greater the degree of misallocation. The positive externalities of research are enormous compared to the value captured by the researchers.
  - mike_hearn 2y ago
    No market system anywhere ends up with one person owning everything. That is the standard under non-market systems, though. > The positive externalities of research ... are only relevant if it's actually research, and not something that looks a lot like research without actually being so.

mistermann 2y ago

> Academic science is excluded from market mechanisms by legal fiat... And a world class propaganda game. Go onto mainstream social media, pick a science fan at random out of the crowd and pick an argument with them about science. The average fan would have you believe that science is just short of perfect. In a sense it's hard to blame these people, considering what two years of the media machine's full court press of pro-science culture war propaganda during a global pandemic does to a mind of average intelligence. I mean, I suppose "winning" is important folks, but is losing your soul in the process worth it?

pocketsand 2y ago

> Academic science is excluded from market mechanisms by legal fiat. This to me just seems untrue. What is your basis for this claim? There is plenty of research privately funded by corporations, some of which is very influential. Often this work is published by university researchers. Ask any university researcher about the numerous compliance courses we all have to take about funding and conflict of interest. It is true that the biggest funders (NSF, NIH) are not market-focused, but for good reason. The market does not prioritize the public good. I know first hand -- my son has a rare disease (1 out of 20,000 people). There are many drug companies putting vast resources into drug development in the hopes of a huge payoff. In reality this benefits a small number of people (I remain grateful for how improvements have helped us). I'm grateful our major scientific funding bodies are not swayed entirely by market influences because it would lead to us focusing on a narrow set of scientific problems which would ultimately limit the way it helps the public good. Im any event, I work in biomedical research. I think your diagnosis (incentives, process) is correct, but the way you discuss the attitudes and motives of researchers is wrong-headed. You say: > Their reputation is tanking but they just don't seem to care and why should they? They'll get grants from the NIH anyway because they're all as bad as each other, and nobody in politics is talking about a total defunding of the sector yet. You're talking about hundreds of thousands of researchers as if they're all psychotic citation fanatics with no care for truth. That is not reality. I think the kind of psychotic, data-manipulating researcher who would put people's health and lives at risk for citations -- or fabricate data sets out of thin air -- are vanishingly small. We can point to a handful of them -- the author of this paper, and the Daniel Ariely's and Francesca Gino's of the world -- but there are tens of thousands of people in every field working on research in good faith, with utmost care. The vast, vast majority never have any scandal, never get caught up in data manipulation, and so on. No field I know of out-right tolerates fraud (and I follow all the retraction stories fairly closely). I think the closest we get to "toleration" is researchers dealing with scientific problems who more or less say "we're not going to publicly flay you but behind your back we're all going to know what you did and your future is limited when it comes to big grants, prestigious invitations, and so on." People who are credibly accused of fraud become pariahs and often targets of scorn not only within the research community but in the press and wider community. The most serious issue IMO is not outright but poor norms around scientific practice, leading to p-hacking, harking, and other "forking paths" problems. Calling that type of behavior "fraudulent" is perhaps justifiable under some ways of thinking, but I think the word fraud mischaracterizes what is going on. There are, in fact, many serious efforts to root out this type of behavior and put in transparency rules to open up research to scrutiny, including among funders like the NIH.

Log_out_ 2y ago

Maybe it should become best practice to publish honeypots. Anyone who claims to reproduce these papers or cites thsm gets autolisted for fraud.