4 ms·

There are at least two problems with this argument. First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper wh

There are at least two problems with this argument. First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others. That's why we hear a lot about fraud in top tier venues. Lower ones probably have more problematic publications. Second, having fake results won't guarantee you a high quality publication, leave alone position in the future. I can imagine some kind of first order phase transition in which the amount of fraudsters becomes so large that self correction breaks and instead they start to endorse each other. But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.

jvans 3y ago
It doesn't have to be outright fraud. I imagine the vast majority of manipulations are extremely subtle. Intentional selection bias, multiple hypothesis testing without correction or reporting the failed hypotheses(p hacking). Most of these would improve your chances of finding important results significantly and be almost impossible to detect
- EvgeniyZh 3y ago
  For the first argument I can't really speak beyond my field (physics, or specifically condensed matter physics), where I think it's fairly hard to be subtle. Often it's partial data presentation which allows for inverse Occam razor [1] (proposing fancy explaination instead of a simple one), which is why full data publication requirement are important (and seem to happen more). In smaller sub (sub)fields sometimes fraud becomes large enough to self-sustain, like it happened with Majoranas [2,3] but even that kind of self-corrected. [1] https://arxiv.org/abs/2204.08284 https://arxiv.org/abs/2204.08284 [2] https://twitter.com/spinespresso/status/1503352928656138241 https://twitter.com/spinespresso/status/1503352928656138241 [3] https://espressospin.org/2022/11/17/the-fallen-angel-particle/ https://espressospin.org/2022/11/17/the-fallen-angel-particl...
- mcpackieh 3y ago
  > without reporting the failed hypotheses(p hacking). This one is so subtle, it's probable that many of the people doing it don't even realize they're doing something wrong.
  - deleted 3y ago
    [deleted]
ramblenode 3y ago
> First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others. Fake is a spectrum. Soft faking is easy and widespread--"p-hacking" is the polite term. The stakes for faking it are low because most research is low impact, most research is seldom reproduced, and even bad research is rarely retracted. For that which is reproduced there is plausible deniability--"researcher degrees of freedom". Getting a high impact study published is more important than having one retracted, so researchers follow that incentive structure. > But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet. Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades.
- EvgeniyZh 3y ago
  > The stakes for faking it are low because most research is low impact ... Getting a high impact study published That's contradictory. For low impact studies you can fake whatever you want, because nobody really cares. That probably wouldn't promote your career either tho. Publishing bunch of organic semiconductor papers in Nature and Science will promote your career, but won't hold. That's the point. > Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades. Not deeply, but I surely can come up with bio/med research with real life applications that appeared in last 2-3 decades. Say mRNA vaccines?
  - kortilla 3y ago
    The entire fields of sociology, social psychology, economics, literature, theological studies, etc are all low impact. Even if you get in the best journals, it’s high impact for your career but low impact to society so it will get very little scrutiny apart from the tiny slice of academia you compete with.
    - EvgeniyZh 3y ago
      Again, I know that academia is very different among fields, but usually high impact for career ~ high impact for (sub)field, meaning people will want to look at your data and possibly try to replicate it. Outside scrutiny for science is extremely rare anyway
NovaDudely 3y ago
It doesn't even need to be deliberately faked data but more data published without much due process to ensure it validity. I have had about three people working in the research fields that have used the term, off the record of course, "Publish or perish". Get it out the door, try to get the funding and hopefully if it works out, solve the issues before it becomes a problem.

jvans 3y ago

It doesn't have to be outright fraud. I imagine the vast majority of manipulations are extremely subtle. Intentional selection bias, multiple hypothesis testing without correction or reporting the failed hypotheses(p hacking). Most of these would improve your chances of finding important results significantly and be almost impossible to detect

EvgeniyZh 3y ago
For the first argument I can't really speak beyond my field (physics, or specifically condensed matter physics), where I think it's fairly hard to be subtle. Often it's partial data presentation which allows for inverse Occam razor [1] (proposing fancy explaination instead of a simple one), which is why full data publication requirement are important (and seem to happen more). In smaller sub (sub)fields sometimes fraud becomes large enough to self-sustain, like it happened with Majoranas [2,3] but even that kind of self-corrected. [1] https://arxiv.org/abs/2204.08284 https://arxiv.org/abs/2204.08284 [2] https://twitter.com/spinespresso/status/1503352928656138241 https://twitter.com/spinespresso/status/1503352928656138241 [3] https://espressospin.org/2022/11/17/the-fallen-angel-particle/ https://espressospin.org/2022/11/17/the-fallen-angel-particl...
mcpackieh 3y ago
> without reporting the failed hypotheses(p hacking). This one is so subtle, it's probable that many of the people doing it don't even realize they're doing something wrong.
- deleted 3y ago
  [deleted]

ramblenode 3y ago

> First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others. Fake is a spectrum. Soft faking is easy and widespread--"p-hacking" is the polite term. The stakes for faking it are low because most research is low impact, most research is seldom reproduced, and even bad research is rarely retracted. For that which is reproduced there is plausible deniability--"researcher degrees of freedom". Getting a high impact study published is more important than having one retracted, so researchers follow that incentive structure. > But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet. Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades.

EvgeniyZh 3y ago
> The stakes for faking it are low because most research is low impact ... Getting a high impact study published That's contradictory. For low impact studies you can fake whatever you want, because nobody really cares. That probably wouldn't promote your career either tho. Publishing bunch of organic semiconductor papers in Nature and Science will promote your career, but won't hold. That's the point. > Have you taken a look at the biosciences, social sciences, and medicine? Basic research hasn't been reliable for decades. Not deeply, but I surely can come up with bio/med research with real life applications that appeared in last 2-3 decades. Say mRNA vaccines?
- kortilla 3y ago
  The entire fields of sociology, social psychology, economics, literature, theological studies, etc are all low impact. Even if you get in the best journals, it’s high impact for your career but low impact to society so it will get very little scrutiny apart from the tiny slice of academia you compete with.
  - EvgeniyZh 3y ago
    Again, I know that academia is very different among fields, but usually high impact for career ~ high impact for (sub)field, meaning people will want to look at your data and possibly try to replicate it. Outside scrutiny for science is extremely rare anyway

NovaDudely 3y ago

It doesn't even need to be deliberately faked data but more data published without much due process to ensure it validity. I have had about three people working in the research fields that have used the term, off the record of course, "Publish or perish". Get it out the door, try to get the funding and hopefully if it works out, solve the issues before it becomes a problem.