9 ms·

This sort of behavior is only going to worsen in the coming decades as academics become more desperate. It's a prisoner's dilemma: if everyone is exaggerating t

by eig 2y ago

This sort of behavior is only going to worsen in the coming decades as academics become more desperate. It's a prisoner's dilemma: if everyone is exaggerating their results you have to as well or you will be fired. It's even more dire for the thousands of visa students. The situation is similar to the "Market for lemons" in cars: if the market is polluted with lemons (fake papers), you are disincentivized to publish a plum (real results), since no one can tell it's not faked. You are instead incentivized to take a plum straight to industry and not disseminate it at all. Pharma companies are already known to closely guard their most promising data/results. Similar to the lemon market in cars, I think the only solution is government regulation. In fact, it would be a lot easier than passing lemon laws since most labs already get their funding from the government! Prior retractions should have significant negative impact on grant scores. This would not only incentivize labs, but would also incentivize institutions to hire clean scientists since they have higher grant earning potential.

jimbokun 2y ago
My recommendation is for journals to place at least equal importance to publishing replications as for the original studies. Studies that have not been replicated should be published clearly marked as preliminary results. And then other scientists can pick those up and try to replicate them. And institutions need to give near equal weight to replications as to original research when deciding on promotions. Should be considered every researchers responsibility to contribute to the overall field.
- eig 2y ago
  The problem with putting the onus on the journals is there is no incentive for them to reward replications. Journals don't make money on replicated results. Customers don't buy the replication paper they just read the abstract to see if it worked or not. I do like the idea of institutions giving tenure to people with results that have stood the test of time, but again, there is no incentive to do so. Institutions want superstar faculty, they care less about whether the results are true. The only real incentive that I think can be targeted is still grant money, but I would love to be proved wrong.
  - jimbokun 2y ago
    If all that's true, we should just shut down all the science institutions across the board. They're worth nothing if they are not vigorously pursuing the truth about the world.
- scarmig 2y ago
  You should be able to build an entire career out of replications: hired at the best universities, published in the top journals, social prestige and respect. To the point where every novel study is replicated and published at least once. Until we get to that point, there will be far fewer replications than needed for a healthy scientific system.
  - bonoboTP 2y ago
    > social prestige and respect This one is the showstopper. No matter what you do with rules and regulations, if people aren't impressed by it at a watercooler conversation, or when chatting at a cocktail party at a conference, or when showing a politician around in your lab then nothing else matters. How prestigious something is is not a lever you control.
    - BobaFloutist 2y ago
      There absolutely exist skilled scientists that would happily make a living unglamorously replicating studies, if the money was there. Prestige is a nice motivator, but making a living at all is always the baseline, and is often sufficient.
    - jimbokun 2y ago
      There needs to be a new norm where prestige and respect is conveyed by having a replicated result. Not just the initial paper.
  - m_fayer 2y ago
    Similarly, ambitious and difficult experiments that don’t pan out should also be richly rewarded. You just did all of science the service of clearly marking that tempting path with a big “don’t bother” sign, thus saving resources and pointing the ship a little closer to the direction of truth.
    - creer 2y ago
      Yeah, this is something I don't fully understand. It's work to format and package everything for publication - and it's work which by then may have lost funding since it's failed. And by which time you might be discouraged. BUT like you say all the science has been done, and getting one more serious publication out of it should be rewarding. It's also a chance for the scientist to show that they were serious, competent, diligent in doing the work. It should count as well as a standard publication. It's a chance to collect consulting contracts later. Etc. Management and support should encourage these publications.
- mlsu 2y ago
  We can solve this at the grant level. Stipulate that for every new paper a group publishes from a grant, that group must also publish a replication of an existing finding. Publication would happen in pairs, so that every novel thing would be matched with a replication. Replications could be matched with grants: if you receive $100,000 grant, you'd get the $100,000 you need, plus another $100,000 which you could use to publish a replication of a previous $100,000 grant. Researchers can choose which findings they replicate, but with restrictions, e.g. you can't just choose your group's previous thing. I think if we did this, researchers would naturally be incentivized to publish experiments that are easier to replicate and of course fraud like this would be caught eventually. I bet we could throw away half of publications tomorrow and see no effect on the actual pace of progress in science.
  - scarmig 2y ago
    > I bet we could throw away half of publications tomorrow and see no effect on the actual pace of progress in science. It might actually improve the pace of science, if the half eliminated were not replicable and the remaining half were written by researchers knowing that they would likely face a replication attempt.
  - breuleux 2y ago
    There would still be incentives for collusion (I "reproduce" your research, you "reproduce" mine), and researchers pretending to reproduce papers but actually not bothering (especially if they believe that the original research was done properly). Ultimately, I'm not sure how to incentivize reproduction of research: it's very easy to fake a successful reproduction (you already know the results, and the original researcher will not challenge you), so you don't want to reward that too much. Whereas incentivizing failed reproductions might lead some scientists to sabotage their own reproduction efforts in ways that are subtle enough to have plausible deniability. Proceeding by pairs is probably not enough. You probably need 5-6 replications per paper to make sure that at least one attempt is honest and competent, and make the others afraid to do the wrong thing and stand out.
    - jimbokun 2y ago
- waveBidder 2y ago
  journals have zero incentives to care about any of this.
- calebh 2y ago
  This stuff happens in Computer Science too. Back around 2018 or so I was working on a problem that required graph matching (a relaxed/fuzzy version of the graph isomorphism problem) and was trying algorithms from many different papers. Many of the algorithms I tried to implement didn't work at all, despite considerable effort to get them to behave. In one particularly egregious (and highly cited) example, the algorithm in the paper differed from the provided code on GitHub. I emailed the authors trying to figure out what was going wrong, and they tried to get funding from me for support. My manager wanted me to right a literature review paper which skewered all of these bad papers, but I refused since I thought it would hurt my career. Ironically the algorithm that ended up working the best was from one of the more unknown papers, with few citations.
  - ammbauer 2y ago
    Name and shame :)
    - robotelvis 2y ago
      Calling out bad work is career suicide. You are defecting on your tribe. That’s half of the problem.
      - bernawil 2y ago
        can't you do it anonymously?
  - creer 2y ago
    Beautiful. And thanks for the testimony. Ironically, this may have helped your product or research: Yes you spent more time on the BS, but in the end you found and used an algorithm both better and more obscure. While your competitors struggled with worse ones. Messed up incentives again.
- dyauspitr 2y ago
  I think it’s a great idea. It would also give the army of phds an endless stream of real tangible work and a way to quickly make a name for themselves by disproving results.
- snowwrestler 2y ago
  Replications are not very scientifically useful. If there were flaws in the design of the original experiment, replicating the experiment will also replicate the flaws. What we should aim for is confirmation: a different experiment that tests the underlying phenomenon that was the subject of the first paper.
  - projektfu 2y ago
    Replications don't frequently get published but they do get attempted, because any decent researcher is going to replicate a result they rely on to build the next step. Unfortunately, you can get stuck in the mud as I did and be unable to replicate the prior findings. Is it technique or were the original results in error? We'll never know. Building more results without replications is what caused the psychology crisis. Apparently every lab accepted the p<0.05 results or stated correlations of prior studies and just ran more studies until they got their own that was publishable. Since everyone "knew" that the prior result was true, like priming or whatever, they could conclude anything they wanted, because ex absurdum quodlibet.
    - Zamiel_Snawley 2y ago
      Reproducibility should be a fundamental quality of published experiments. If the published work under specifies the experiment such that it is unreproducible, that means the results can’t be reliably extrapolated because there are unstated conditions.
      - projektfu 2y ago
        Well, yeah, but there are established techniques that are still very finicky. For example, staining frozen sections with fluorescent antibodies can go wrong in many ways and favors the experienced. Electron microscopy can take a lot of training to get right, and also requires careful staining techniques to get meaningful results. RNA work (e.g. FISH) is very sensitive to the presence of RNase which is ubiquitous and difficult to exclude from preparations. So a procedure can be specified that is reproducible but getting the same conditions is more difficult than, say, using Nix.
- breuleux 2y ago
  I'd be careful about that. Faking replications is even easier than faking research, so if you place a lot of importance on them, expect the rate of fraud in replication studies to explode. This is a very difficult problem to solve.
  - jimbokun 2y ago
    To elaborate, you need to give equal value to well implemented studies failing to replicate a result, as you do to studies replicating a result.
    - breuleux 2y ago
      Well of course, but I don't think that would necessarily help much. The point is that you don't really need to do anything: you know what the results should be, and you know you are unlikely to get pushback, so there's only an incentive to do the strict minimum to create plausibility that you ran the experiments. Basically, I think there is a sizable risk that a large number of replications would be fraudulent or half-assed, which dilutes their value. Paradoxically, the more this policy suppresses fraud or mistakes in original research, the less people will perform replication in good faith. I could be wrong, but people are endlessly creative at subverting systems when the stakes are high, so I'm wary of simple solutions. To be fair, it's probably better than the current system, just not as much as we'd like.
- insane_dreamer 2y ago
  > And then other scientists can pick those up and try to replicate them. unless there are grants specifically for that purpose, then it's not going to happen; and it's hard to apply for a grant just to replicate someone else's results verbatim. (usually you're testing the theory but with a different experiment and set of data which is much more interesting than simply repeating what they did with their data; in fact replicating it with a different set of data is important in order to see if the results weren't cherry-picked to fit the original dataset).
- creer 2y ago
  It seems surprisingly hard to counter scientific fraud via a system change. The incentives are messed up all the way around. If the older author is your advisor and you feel one of their juniors is cutting corners or the elder is cutting corners, you better think twice about what move will help your career. If confirming a recent result counts toward tenure, then presto you have an incentive for fraudulent replication (what's the chance it's incorrect anyway? The original author is a big shot.) Going against the previous acclaimed result takes guts especially in a small field where it might kill your career if YOU got it wrong somehow - So you need to have much stronger results than the original research, and good luck with that. We might say "this is perfect work for aspiring student researchers, and done all the time" - to reimplement some legendary science experiment - but no, not when it's a leading edge poorly understood experiment, and not when that same grad student is already running to try and produce original research themselves. The big funders might dedicate money to replicated research that everybody is enthusiastic about (before everyone relies on it). But some research takes years to run. Other research is at the edge of what's possible. Other research is led by a big shot nobody dares to take on. Etc etc. So where is the incentive then? The incentive might be to take the money, fully intending to return an inconclusive result. Some research is taken on now. But only AFTER it's relied on by lots of people. Or much later when better ideas had the time to emerge on how to test the idea more cleverly i.e. cheaper and faster. And that's not great because costly in all the wasted effort by others, based on a fraudulent result. And all the mindshare the bad result now has. This is messed up.
hilux 2y ago
> ... as academics become more desperate. Yes and ... we're already there.
Lionga 2y ago
BS governmental desperation to show any "result" (even if it is fake) is what brought us here. As scientist have to show more fake results to get more grants. Removing the government from science could help, not the other way around.
- SV_BubbleTime 2y ago
  Good luck with that sentiment here. People just went through the last five years and will go to their graves defending what they saw first hand. To admit that maybe those moves and omissions weren’t helpful would be to admit their ideology was wrong. And that can not be.
  - Me000 2y ago
    You know you wrote this on the Internet? The thing the government created to do science?
    - Lionga 2y ago
      a) anything that the government done for the internet was before science was corrupted by government b) what the government did for the internet was 1% of at the very, very best. And there is a chance so close to 100% that a similar thing would have been done without government even the 1% does not matter.
abigail95 2y ago
you don't need regulation for a stable durable goods market. income and credit shocks cause turnover of good quality stock in the secondary market.
fluidcruft 2y ago
I wonder if there are any studies on whether fraud increased after the Bayh-Dole Act. There's certainly fraud for prestige, that's pretty expected. But mixing in financial benefits increases the reward and brings administrators into play.
dyauspitr 2y ago
It could also have a chilling effect on a lot of breakthrough research. If people are willing to put out what they mostly think is right, it might set back progress decades as well.
robwwilliams 2y ago
The incentive structures in science has been relatively stable since I entered the field in 1980 (neuroscience, developmental biology, genetics). Quality and quantity of science is extraordinary, but peer review is worse than bad. There are almost no incentives to review the work of your colleagues properly. It does not pay bills and you can make enemies easily. But there was no golden era of science to look back on. It has always been a wonderful productive mess—much like the rest of life. As least it moves forward—and now exceedingly rapidly. Almost unbelievably, there are far worse crimes than fraud that we completely ignore. There are crimes associated with social convention in science of the type discussed by Karl Herrup with respect to 20 years of misguided focus on APP and abeta fragments in Alzheimer’s disease: https://mitpress.mit.edu/9780262546010/how-not-to-study-a-disease/ https://mitpress.mit.edu/9780262546010/how-not-to-study-a-di... This could be called the “misdemeanors of scientific social inertia”. Or the “old boys network”. There is also an invisible but insidious crime of data evaporation. Almost no funders will fund data preservation. Even genomics struggles but is way ahead in biomedical research. Neuroscience is pathetic in this regard (and I chaired the Society for Neuroscience’s Neuroinformatics Committee). I have a talk on this socio-political crime of data evaporation. https://www.youtube.com/watch?v=4ZhnXU8gV44&embeds_referring_euri=https%3A%2F%2Fgenenetwork.org%2F https://www.youtube.com/watch?v=4ZhnXU8gV44&embeds_referring...
houston_Euler 2y ago
While Akerlof's Market for Lemons did consider cases where government intervention is necessary to preserve a market, like with health insurance markets (Medicare), he describes the "market for lemons" in the used car market as having been solved by warranties. If someone brings a plum to a market for lemons, they can distinguish the quality of their product by offering a warranty on its purchase, something that sellers of lemons would be unwilling to do, because they want to pass the cost burden of the lemon onto the purchaser. The full paper is fairly accessible, and worth a read. Not sure how this could be applied to academia, one of the problems is that there can be significant gaps between perpetrating fraud and having it discovered, so the violators might still have an incentive to cheat.
pc86 2y ago
> if everyone is exaggerating their results you have to as well or you will be fired. Is this really the case the though? Isn't the whole point of tenure (or a big selling point at least) insulating academics from capricious firings? The big question I have is that there are names on these fraudulent papers, so why are these people still employed? If you generate fictitious data to get published, you should lose any research or teaching job you have, and have to work at McDonald's or a warehouse for the rest of your life. There are plenty of people who want to be professors that we can eliminate the ones who will lie while doing it without losing much (perhaps anything). If your job was funded by taxpayer funds there should be criminal charges associated with willfully and knowingly fabricating data, results, or methods. At that point you're literally lying in order to steal taxpayer funds, it's no different than a city manager embezzling or grabbing a stack of $20 bills out of the cash register.
- robotelvis 2y ago
  Well you aren’t going to get tenure unless you distort your results and it’s hard to change established habits. That, and you select for the kind of people who are willing to fake results to further their own careers.
  - pc86 2y ago
    Yeah I agree with you, I guess I just keep coming back to "make the punishment so eye-wateringly harsh very few people are stupid enough to try it." Most types of fraud carry a $100-250k monetary penalty and up to 20, 25, 30 years in prison. The number of people willing to fabricate research data decreases dramatically if you're going to have to pay the grant back from your $20/hr warehouse job after you spend the better part of a decade in a minimum security prison.
    - robotelvis 2y ago
      The flip side is that if punishments are eye watering harsh then people will be even less willing to inflict them. Bear in mind also that the vast majority of academic fraud isn’t cut and dried easily proved. It’s p hacking or “accidental” flaws in an analysis, or forgetting to mention some important detail.

by eig 2y ago

jimbokun 2y ago

My recommendation is for journals to place at least equal importance to publishing replications as for the original studies. Studies that have not been replicated should be published clearly marked as preliminary results. And then other scientists can pick those up and try to replicate them. And institutions need to give near equal weight to replications as to original research when deciding on promotions. Should be considered every researchers responsibility to contribute to the overall field.

eig 2y ago
The problem with putting the onus on the journals is there is no incentive for them to reward replications. Journals don't make money on replicated results. Customers don't buy the replication paper they just read the abstract to see if it worked or not. I do like the idea of institutions giving tenure to people with results that have stood the test of time, but again, there is no incentive to do so. Institutions want superstar faculty, they care less about whether the results are true. The only real incentive that I think can be targeted is still grant money, but I would love to be proved wrong.
- jimbokun 2y ago
  If all that's true, we should just shut down all the science institutions across the board. They're worth nothing if they are not vigorously pursuing the truth about the world.
scarmig 2y ago
You should be able to build an entire career out of replications: hired at the best universities, published in the top journals, social prestige and respect. To the point where every novel study is replicated and published at least once. Until we get to that point, there will be far fewer replications than needed for a healthy scientific system.
- bonoboTP 2y ago
  > social prestige and respect This one is the showstopper. No matter what you do with rules and regulations, if people aren't impressed by it at a watercooler conversation, or when chatting at a cocktail party at a conference, or when showing a politician around in your lab then nothing else matters. How prestigious something is is not a lever you control.
  - BobaFloutist 2y ago
    There absolutely exist skilled scientists that would happily make a living unglamorously replicating studies, if the money was there. Prestige is a nice motivator, but making a living at all is always the baseline, and is often sufficient.
  - jimbokun 2y ago
    There needs to be a new norm where prestige and respect is conveyed by having a replicated result. Not just the initial paper.
- m_fayer 2y ago
  Similarly, ambitious and difficult experiments that don’t pan out should also be richly rewarded. You just did all of science the service of clearly marking that tempting path with a big “don’t bother” sign, thus saving resources and pointing the ship a little closer to the direction of truth.
  - creer 2y ago
    Yeah, this is something I don't fully understand. It's work to format and package everything for publication - and it's work which by then may have lost funding since it's failed. And by which time you might be discouraged. BUT like you say all the science has been done, and getting one more serious publication out of it should be rewarding. It's also a chance for the scientist to show that they were serious, competent, diligent in doing the work. It should count as well as a standard publication. It's a chance to collect consulting contracts later. Etc. Management and support should encourage these publications.
mlsu 2y ago
We can solve this at the grant level. Stipulate that for every new paper a group publishes from a grant, that group must also publish a replication of an existing finding. Publication would happen in pairs, so that every novel thing would be matched with a replication. Replications could be matched with grants: if you receive $100,000 grant, you'd get the $100,000 you need, plus another $100,000 which you could use to publish a replication of a previous $100,000 grant. Researchers can choose which findings they replicate, but with restrictions, e.g. you can't just choose your group's previous thing. I think if we did this, researchers would naturally be incentivized to publish experiments that are easier to replicate and of course fraud like this would be caught eventually. I bet we could throw away half of publications tomorrow and see no effect on the actual pace of progress in science.
- scarmig 2y ago
  > I bet we could throw away half of publications tomorrow and see no effect on the actual pace of progress in science. It might actually improve the pace of science, if the half eliminated were not replicable and the remaining half were written by researchers knowing that they would likely face a replication attempt.
- breuleux 2y ago
  There would still be incentives for collusion (I "reproduce" your research, you "reproduce" mine), and researchers pretending to reproduce papers but actually not bothering (especially if they believe that the original research was done properly). Ultimately, I'm not sure how to incentivize reproduction of research: it's very easy to fake a successful reproduction (you already know the results, and the original researcher will not challenge you), so you don't want to reward that too much. Whereas incentivizing failed reproductions might lead some scientists to sabotage their own reproduction efforts in ways that are subtle enough to have plausible deniability. Proceeding by pairs is probably not enough. You probably need 5-6 replications per paper to make sure that at least one attempt is honest and competent, and make the others afraid to do the wrong thing and stand out.
  - jimbokun 2y ago
waveBidder 2y ago
journals have zero incentives to care about any of this.
calebh 2y ago
This stuff happens in Computer Science too. Back around 2018 or so I was working on a problem that required graph matching (a relaxed/fuzzy version of the graph isomorphism problem) and was trying algorithms from many different papers. Many of the algorithms I tried to implement didn't work at all, despite considerable effort to get them to behave. In one particularly egregious (and highly cited) example, the algorithm in the paper differed from the provided code on GitHub. I emailed the authors trying to figure out what was going wrong, and they tried to get funding from me for support. My manager wanted me to right a literature review paper which skewered all of these bad papers, but I refused since I thought it would hurt my career. Ironically the algorithm that ended up working the best was from one of the more unknown papers, with few citations.
- ammbauer 2y ago
  Name and shame :)
  - robotelvis 2y ago
    Calling out bad work is career suicide. You are defecting on your tribe. That’s half of the problem.
    - bernawil 2y ago
      can't you do it anonymously?
- creer 2y ago
  Beautiful. And thanks for the testimony. Ironically, this may have helped your product or research: Yes you spent more time on the BS, but in the end you found and used an algorithm both better and more obscure. While your competitors struggled with worse ones. Messed up incentives again.
dyauspitr 2y ago
I think it’s a great idea. It would also give the army of phds an endless stream of real tangible work and a way to quickly make a name for themselves by disproving results.
snowwrestler 2y ago
Replications are not very scientifically useful. If there were flaws in the design of the original experiment, replicating the experiment will also replicate the flaws. What we should aim for is confirmation: a different experiment that tests the underlying phenomenon that was the subject of the first paper.
- projektfu 2y ago
  Replications don't frequently get published but they do get attempted, because any decent researcher is going to replicate a result they rely on to build the next step. Unfortunately, you can get stuck in the mud as I did and be unable to replicate the prior findings. Is it technique or were the original results in error? We'll never know. Building more results without replications is what caused the psychology crisis. Apparently every lab accepted the p<0.05 results or stated correlations of prior studies and just ran more studies until they got their own that was publishable. Since everyone "knew" that the prior result was true, like priming or whatever, they could conclude anything they wanted, because ex absurdum quodlibet.
  - Zamiel_Snawley 2y ago
    Reproducibility should be a fundamental quality of published experiments. If the published work under specifies the experiment such that it is unreproducible, that means the results can’t be reliably extrapolated because there are unstated conditions.
    - projektfu 2y ago
      Well, yeah, but there are established techniques that are still very finicky. For example, staining frozen sections with fluorescent antibodies can go wrong in many ways and favors the experienced. Electron microscopy can take a lot of training to get right, and also requires careful staining techniques to get meaningful results. RNA work (e.g. FISH) is very sensitive to the presence of RNase which is ubiquitous and difficult to exclude from preparations. So a procedure can be specified that is reproducible but getting the same conditions is more difficult than, say, using Nix.
breuleux 2y ago
I'd be careful about that. Faking replications is even easier than faking research, so if you place a lot of importance on them, expect the rate of fraud in replication studies to explode. This is a very difficult problem to solve.
- jimbokun 2y ago
  To elaborate, you need to give equal value to well implemented studies failing to replicate a result, as you do to studies replicating a result.
  - breuleux 2y ago
    Well of course, but I don't think that would necessarily help much. The point is that you don't really need to do anything: you know what the results should be, and you know you are unlikely to get pushback, so there's only an incentive to do the strict minimum to create plausibility that you ran the experiments. Basically, I think there is a sizable risk that a large number of replications would be fraudulent or half-assed, which dilutes their value. Paradoxically, the more this policy suppresses fraud or mistakes in original research, the less people will perform replication in good faith. I could be wrong, but people are endlessly creative at subverting systems when the stakes are high, so I'm wary of simple solutions. To be fair, it's probably better than the current system, just not as much as we'd like.
insane_dreamer 2y ago
> And then other scientists can pick those up and try to replicate them. unless there are grants specifically for that purpose, then it's not going to happen; and it's hard to apply for a grant just to replicate someone else's results verbatim. (usually you're testing the theory but with a different experiment and set of data which is much more interesting than simply repeating what they did with their data; in fact replicating it with a different set of data is important in order to see if the results weren't cherry-picked to fit the original dataset).
creer 2y ago
It seems surprisingly hard to counter scientific fraud via a system change. The incentives are messed up all the way around. If the older author is your advisor and you feel one of their juniors is cutting corners or the elder is cutting corners, you better think twice about what move will help your career. If confirming a recent result counts toward tenure, then presto you have an incentive for fraudulent replication (what's the chance it's incorrect anyway? The original author is a big shot.) Going against the previous acclaimed result takes guts especially in a small field where it might kill your career if YOU got it wrong somehow - So you need to have much stronger results than the original research, and good luck with that. We might say "this is perfect work for aspiring student researchers, and done all the time" - to reimplement some legendary science experiment - but no, not when it's a leading edge poorly understood experiment, and not when that same grad student is already running to try and produce original research themselves. The big funders might dedicate money to replicated research that everybody is enthusiastic about (before everyone relies on it). But some research takes years to run. Other research is at the edge of what's possible. Other research is led by a big shot nobody dares to take on. Etc etc. So where is the incentive then? The incentive might be to take the money, fully intending to return an inconclusive result. Some research is taken on now. But only AFTER it's relied on by lots of people. Or much later when better ideas had the time to emerge on how to test the idea more cleverly i.e. cheaper and faster. And that's not great because costly in all the wasted effort by others, based on a fraudulent result. And all the mindshare the bad result now has. This is messed up.

hilux 2y ago

> ... as academics become more desperate. Yes and ... we're already there.

Lionga 2y ago

BS governmental desperation to show any "result" (even if it is fake) is what brought us here. As scientist have to show more fake results to get more grants. Removing the government from science could help, not the other way around.

SV_BubbleTime 2y ago
Good luck with that sentiment here. People just went through the last five years and will go to their graves defending what they saw first hand. To admit that maybe those moves and omissions weren’t helpful would be to admit their ideology was wrong. And that can not be.
- Me000 2y ago
  You know you wrote this on the Internet? The thing the government created to do science?
  - Lionga 2y ago
    a) anything that the government done for the internet was before science was corrupted by government b) what the government did for the internet was 1% of at the very, very best. And there is a chance so close to 100% that a similar thing would have been done without government even the 1% does not matter.

abigail95 2y ago

you don't need regulation for a stable durable goods market. income and credit shocks cause turnover of good quality stock in the secondary market.

fluidcruft 2y ago

I wonder if there are any studies on whether fraud increased after the Bayh-Dole Act. There's certainly fraud for prestige, that's pretty expected. But mixing in financial benefits increases the reward and brings administrators into play.

dyauspitr 2y ago

It could also have a chilling effect on a lot of breakthrough research. If people are willing to put out what they mostly think is right, it might set back progress decades as well.

robwwilliams 2y ago

The incentive structures in science has been relatively stable since I entered the field in 1980 (neuroscience, developmental biology, genetics). Quality and quantity of science is extraordinary, but peer review is worse than bad. There are almost no incentives to review the work of your colleagues properly. It does not pay bills and you can make enemies easily. But there was no golden era of science to look back on. It has always been a wonderful productive mess—much like the rest of life. As least it moves forward—and now exceedingly rapidly. Almost unbelievably, there are far worse crimes than fraud that we completely ignore. There are crimes associated with social convention in science of the type discussed by Karl Herrup with respect to 20 years of misguided focus on APP and abeta fragments in Alzheimer’s disease: https://mitpress.mit.edu/9780262546010/how-not-to-study-a-disease/ https://mitpress.mit.edu/9780262546010/how-not-to-study-a-di... This could be called the “misdemeanors of scientific social inertia”. Or the “old boys network”. There is also an invisible but insidious crime of data evaporation. Almost no funders will fund data preservation. Even genomics struggles but is way ahead in biomedical research. Neuroscience is pathetic in this regard (and I chaired the Society for Neuroscience’s Neuroinformatics Committee). I have a talk on this socio-political crime of data evaporation. https://www.youtube.com/watch?v=4ZhnXU8gV44&embeds_referring_euri=https%3A%2F%2Fgenenetwork.org%2F https://www.youtube.com/watch?v=4ZhnXU8gV44&embeds_referring...

houston_Euler 2y ago

While Akerlof's Market for Lemons did consider cases where government intervention is necessary to preserve a market, like with health insurance markets (Medicare), he describes the "market for lemons" in the used car market as having been solved by warranties. If someone brings a plum to a market for lemons, they can distinguish the quality of their product by offering a warranty on its purchase, something that sellers of lemons would be unwilling to do, because they want to pass the cost burden of the lemon onto the purchaser. The full paper is fairly accessible, and worth a read. Not sure how this could be applied to academia, one of the problems is that there can be significant gaps between perpetrating fraud and having it discovered, so the violators might still have an incentive to cheat.

pc86 2y ago

> if everyone is exaggerating their results you have to as well or you will be fired. Is this really the case the though? Isn't the whole point of tenure (or a big selling point at least) insulating academics from capricious firings? The big question I have is that there are names on these fraudulent papers, so why are these people still employed? If you generate fictitious data to get published, you should lose any research or teaching job you have, and have to work at McDonald's or a warehouse for the rest of your life. There are plenty of people who want to be professors that we can eliminate the ones who will lie while doing it without losing much (perhaps anything). If your job was funded by taxpayer funds there should be criminal charges associated with willfully and knowingly fabricating data, results, or methods. At that point you're literally lying in order to steal taxpayer funds, it's no different than a city manager embezzling or grabbing a stack of $20 bills out of the cash register.

robotelvis 2y ago
Well you aren’t going to get tenure unless you distort your results and it’s hard to change established habits. That, and you select for the kind of people who are willing to fake results to further their own careers.
- pc86 2y ago
  Yeah I agree with you, I guess I just keep coming back to "make the punishment so eye-wateringly harsh very few people are stupid enough to try it." Most types of fraud carry a $100-250k monetary penalty and up to 20, 25, 30 years in prison. The number of people willing to fabricate research data decreases dramatically if you're going to have to pay the grant back from your $20/hr warehouse job after you spend the better part of a decade in a minimum security prison.
  - robotelvis 2y ago
    The flip side is that if punishments are eye watering harsh then people will be even less willing to inflict them. Bear in mind also that the vast majority of academic fraud isn’t cut and dried easily proved. It’s p hacking or “accidental” flaws in an analysis, or forgetting to mention some important detail.