6 ms·

Ok, I know this is a bit of a stretch, but bear with me: I think we are seeing something similar to the doping scandal that took down Lance Armstrong and many o

Ok, I know this is a bit of a stretch, but bear with me: I think we are seeing something similar to the doping scandal that took down Lance Armstrong and many others a while back. There was at least one year's Tour de France in which all but one participant were later found to have been doping. In other words, you didn't get to that level of that competitive activity, if you didn't cheat, because it wasn't possible to outperform people with such a significant advantage. It actually became a contest, not just of bicycling, but of doping, because getting away with it was quite difficult and it took years for the world of cycling to get good enough countermeasures to shift the advantage to non-doping. Academic research has, for decades now, been a very competitive, high stakes endeavor. Many fields have more people trying to work in it, than there are spots (i.e. grants and endowed chairs) available. If you have twice as many aspirants as slots available, and "only" 25% of them cheat (in this case, fudge their data to get more interesting results), then you get something like 50% of your field filled with fraud. Moreover, the closer they are to the top, the higher the likelihood that they are "doping". I live in Austin, TX, and I remember when nearly all of Lance Armstrong's competitors had been busted, but he had not yet. I said to other people, "well, that's it then, he must have been doping. You can't win the Tour de France 7 years in a row against doping opponents, if you're clean. Either doping doesn't work, or he was doping." I recall several people disagreeing, convinced that he was clean. He wasn't clean.

nemo44x 3y ago
Universities are in the business of creating/discovering truth and shipping it. A Positivist review quickly shows things like math are easy to prove true. As you work your ways down to the humanities it becomes increasingly difficult. And competitive. Shipping truth that isn’t powerful isn’t as attractive to your customers as truth that is. So you’re incentivized to develop truth that is. And you do this by hacking the accepted standards in knowledge pursuit by starting with a conclusion and working backwards. You tell a story based on the evidence that’s useful. Now it’s “the science” or at the very least “there’s studies” and this is useful to both the customers (NGOs, journals, activists, lobbyists, media, anyone that wants to influence policy) and the university (attracts money, reputation and status) and the people shipping it (tenure, book deals, speaking fees). It’s not a conspiracy. It’s just simple incentives. The poor guy who spends his time figuring out what the truth is not or that ships truth that isn’t immediately useful to the customers is looking for a new job after his grant dries up.
- derbOac 3y ago
  The other level of problem is that truth in reality means consensus. You can argue about what truth is in reality, but functionally it means consensus. So it's not even about what is or is not true, or what is recognized as useful in the short term, it's that what defines truth at any given time is what is popularly believed to be true. This sounds very flaky but it has real consequences because the incentives aren't just to find positive results, it's to find positive results that are likely to be accepted by the majority of the academic field. This kind of positive feedback loop is bound to lead to pressure because science almost by definition has to get some things incorrect — it's how progress is made. So people are actively punished for going against the grain, or exploring areas where our knowledge is limited and therefore prone to lots of negative results because we just don't know.
  - nemo44x 3y ago
    > The other level of problem is that truth in reality means consensus That’s the job of the prestigious media. They manufacture consent to the masses. They propagate right-think. Again, not a conspiracy but simply incentives. Anyways, there is objective truth. Not everything can be settled that way though and you’re right in that “truth” can mean consensus in many instances. The problem with that is that in many things the dissent is tasked with proving claims untrue, because of the consensus, rather than those making claim to knowledge having to prove it is true. There’s a lot of this today.
  - mcpackieh 3y ago
    Perceived truth is consensus. But greater truth, reality itself, remains true even if not a single person is able to perceive it. You can't actually modify reality with wishful thinking.
  - cb504 3y ago
    For these scientific papers, at a minimum truth==reproducibility. For humanities and social sciences, it could mean consensus, but in hard science it is different. You are doing experiments that produce data. If someone else repeats the experiment, they should get the same data. In this case, no one could reproduce the results of the papers in question. I wonder how widespread this has become. How many scientists, driven to publish, know that it is too expensive to reproduce their experiment?
Graziano_M 3y ago
> It actually became a contest, not just of bicycling, but of doping, because getting away with it was quite difficult and it took years for the world of cycling to get good enough countermeasures to shift the advantage to non-doping. Not just a contest of hiding doping, but of winning against others who were doping. If everyone is doping, then you could argue it's still a fair playing field, just with a higher skill ceiling.
- noah_buddy 3y ago
  It’s not a fair playing field because there were still people who did not cheat.
  - Graziano_M 3y ago
    That's why I said, "If everyone is doping" :)
    - optimalsolver 3y ago
      But that's not what you were actually implying.
  - DrBenCarson 3y ago
    As long as they knew others were doping, I would argue it was still a fair playing field. The real advantage is in others not knowing you are illegally enhancing performance.
  - jnsaff2 3y ago
    https://en.m.wikipedia.org/wiki/Christophe_Bassons https://en.m.wikipedia.org/wiki/Christophe_Bassons
- friendzis 3y ago
  > If everyone is doping, then you could argue it's still a fair playing field, just with a higher skill ceiling. No, because now you have at least two contests in parallel: the race event for the cyclist and through-the-year "doping and not getting caught" for medical/management team. For example one prominent olympic level swimmer was disqualified because they failed to appear for testing multiple times. Why would one do that just to come back after disqualification?
johntiger1 3y ago
You've hit the nail head on. Also applies to China's anti-corruption programs. Everyone who achieves any level of political power is corrupt, but framing it in an anti-corruption campaign allows you to target your political enemies.
- FrustratedMonky 3y ago
  No need to single out China. All politics, and guess for Academia, can extend to all human organizations, Church, School, Government.
  - merman 3y ago
    You’re making the preposterous assertion that all countries’ politicians are equally corrupt. That needs some extraordinary data to support.
    - FrustratedMonky 3y ago
      Didn't say equal. Just that all 'large' human organizations tend towards corruption. Churches, Governments, Academia. Hierarchies form, incentives drift over time and become mismatched with original intent. The proof is in history, look at Church abuse, Inquisition, look at grade fixing, admissions scandals, look at Volkswagen tweaking emissions, look at CEO's lying to promote products. Our entire society is based on the ebb and flow of some kind of corruption, or ebb-flow of Law. Even today, in America, half the population think that the laws themselves are corrupt and should be broken. Judges aren't trusted. So who is corrupt if you don't have any foundation to judge anything against? I'm totally against China taking Taiwan, but from a 'certain point of view', they have point. It was a part of China before the war, and the mainland 'won'. So they think of it as unification. Again. I am totally against China taking Taiwan. Just think it is bit more nuanced. Actually, think we are agreeing.
      - merman 3y ago
        If there’s no need to ‘single out’ China then there’s no need to ‘single out’ doping in cycling, and no need to ‘single out’ this one particular Stanford President researcher, etc. The objective of your comment, that I am pointing out, is that you are hand-waving corruption in Chinese politics away by saying “but what about…”
azuriten 3y ago
> There was at least one year's Tour de France in which all but one participant were later found to have been doping. In other words, you didn't get to that level of that competitive activity, if you didn't cheat, because it wasn't possible to outperform people with such a significant advantage. This is not true. Even during the 1904 Tour de France where 9 people were disqualifed because of, among other actions, illegal use of cars or trains [1] - 27 riders finished the race. Tour de France in the modern era has up to 180+ competitors lining up, and there hasn't been a case of 100+ riders being disqualified for doping. [1] https://en.wikipedia.org/wiki/1904_Tour_de_France#Disqualification https://en.wikipedia.org/wiki/1904_Tour_de_France#Disqualifi...
- rukuu001 3y ago
  Yeah not the whole peleton, but most of Armstrong’s main rivals right? Like Contador etc. (I know his test was contentious).
- areyousure 3y ago
  Indeed, the claim needs to be adjusted to be correct. One of the worst recent years was 2005: https://en.wikipedia.org/wiki/Doping_at_the_Tour_de_France#2005_Tour_de_France https://en.wikipedia.org/wiki/Doping_at_the_Tour_de_France#2... There, the claim is that of the top 10 finishers, all but one participant were either stripped of result in that race or failed tests/were sanctioned for doping in another race. (The remaining participant, originally #8 Cadel Evans, was a documented client of Michele Ferrari.)
  - tfyoung 3y ago
    A one-off consultation prior to his road racing career to see if he the right kind of fitness for road. I don't think the association should be used to tarnish Cadel without further evidence.
- rossdavidh 3y ago
  Thank you for the correction, I either heard wrong back then, or misremembered. I think the analogy/logic still applies, though. Actually, this suggests even more strongly that the higher their position in a competitive field, the more likely they are doping...
alijrtaliw 3y ago
I (and others before and after me) filed complaints that my advisor at UT Austin was falsifying data. University ignored me and gave him tenure. It is commonplace and admin doesn't care as long as it doesn't affect the endowment.
- canjobear 3y ago
  Are you willing to say who it is?
  - bigfoot675 3y ago
    Yeah I'm also curious
- deleted 3y ago
  [deleted]
EvgeniyZh 3y ago
There are at least two problems with this argument. First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others. That's why we hear a lot about fraud in top tier venues. Lower ones probably have more problematic publications. Second, having fake results won't guarantee you a high quality publication, leave alone position in the future. I can imagine some kind of first order phase transition in which the amount of fraudsters becomes so large that self correction breaks and instead they start to endorse each other. But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.
- jvans 3y ago
  It doesn't have to be outright fraud. I imagine the vast majority of manipulations are extremely subtle. Intentional selection bias, multiple hypothesis testing without correction or reporting the failed hypotheses(p hacking). Most of these would improve your chances of finding important results significantly and be almost impossible to detect
  - EvgeniyZh 3y ago
    For the first argument I can't really speak beyond my field (physics, or specifically condensed matter physics), where I think it's fairly hard to be subtle. Often it's partial data presentation which allows for inverse Occam razor [1] (proposing fancy explaination instead of a simple one), which is why full data publication requirement are important (and seem to happen more). In smaller sub (sub)fields sometimes fraud becomes large enough to self-sustain, like it happened with Majoranas [2,3] but even that kind of self-corrected. [1] https://arxiv.org/abs/2204.08284 https://arxiv.org/abs/2204.08284 [2] https://twitter.com/spinespresso/status/1503352928656138241 https://twitter.com/spinespresso/status/1503352928656138241 [3] https://espressospin.org/2022/11/17/the-fallen-angel-particle/ https://espressospin.org/2022/11/17/the-fallen-angel-particl...
  - mcpackieh 3y ago
    > without reporting the failed hypotheses(p hacking). This one is so subtle, it's probable that many of the people doing it don't even realize they're doing something wrong.
AmericanChopper 3y ago
Doping in sports is so much more complicated than most people think it is. For starters there’s so many performance enhancing substances, and entire underground industries for developing and producing them. All of these drugs would be banned by the blanket prohibition on unapproved substances, but in practice many of them don’t have any screening procedures, so won’t turn up in any tests. There’s also the fact that many performance enhancing substances are biologically identical to substances your body is naturally producing (like bio-identical testosterone for example). The only way to test for these is to test the ratios of certain metabolites (with a margin for error to avoid false positives). In practice this basically means any athlete can do steroid cycles, as long as they are only boosting their test levels up to that of a genetically elite outlier. You also have the therapeutic use exemption, which allows for legitimate therapeutic use of banned substances. Testosterone replacement therapy is a legitimate therapeutic use of steroids, and some sports are known for abusing TUEs for this usecase. Testing for PEDs is way less precise than a lot of people think it is, and any motivated athlete can get away with PED use to some extent, no matter how much testing you subject them to. There’s good reason to be suspicious of a lot of athletes. One massively suspect category of athlete is all of the ones in/near their 40s who somehow manage to maintain physiques as good as/better than they had in their early 20s.
- notknifescience 3y ago
  I would be very much interested in reading more about this, do you have any good articles you could point me to?
  - AmericanChopper 3y ago
    This paper is a bit out of date, but gives a reasonable overview of the challenges of testing for testosterone supplementation https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2657495/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2657495/ Paper on challenges of detecting growth hormone https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830297/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830297/ Testing for both of those substances has improved, but still leave plenty of room for abuse. Use of "unknown" or the constant stream newly developed substances is naturally not rigorously studied, but here's a couple of articles referencing the challenges they pose. https://thetruthaboutforensicscience.com/undetectable-performance-enhancing-drugs-in-sports/ https://thetruthaboutforensicscience.com/undetectable-perfor... https://bleacherreport.com/articles/1955231-undetectable-the-new-ped-that-could-be-in-the-olympics-nfl-and-mlb-now https://bleacherreport.com/articles/1955231-undetectable-the... UFC is notorious for testosterone TUEs, but that's mostly because they regulate things differently to most other sports https://www.espn.com/espn/otl/story/_/id/10500652/therapeutic-use-exemptions-testosterone-mma-outpace-other-sports-lines-finds https://www.espn.com/espn/otl/story/_/id/10500652/therapeuti... Top flight soccer is another sport that's constantly garnering attention for TUEs, but most of the coverage of that is speculative, because other than high level statistics the TUE process is naturally not very transparent (and most of the recent coverage on the topic has been replaced by controversy relating to trans athletes).
Georgelemental 3y ago
I know several Tour de France aficionados, who would claim that doping remains rampant, they are just better at hiding it now.
- deleted 3y ago
  [deleted]
Al-Khwarizmi 3y ago
I don't think this kind of fraud is even remotely close in scale to the cycling doping you mention. My primary reason for believing this is that as a scientist myself, I've never done it, and none of my coauthors has ever proposed to do something like that. I have seen other kinds of fraud (e.g. ghost authorship) but not result manipulation. But of course, If I were a fraudster myself I would also say that I've never done it :) So a more general argument: scientific "doping" is just not as straightforward as cycling doping. In cycling, it's clear that if you don't get caught (and have no scruples) you benefit from doping, it makes you stronger. In science, you first need an idea that looks original/promising, and then you do the experiments and see if the idea works or not. This kind of "doping" will only help if you (1) are good at generating ideas, but (2) these ideas often don't work. At least in my field, this is possible but far from the most common situation to be. And in other situations, there is not much benefit to faking results.
- awesomeMilou 3y ago
  > if you (1) are good at generating ideas I have been joking with my former boss (who is a professor) about how easy it is to come up with good ideas. We each have five to six One Billion Dollar ideas before breakfast, and another 1-2 while in the shower! /s Coming up with ideas is _ridicolously_ easy, executing on them is the hard part. If you're a seasoned researcher, it shouldn't be hard to guess at the "meta" of your field, you're probably aware of promising ideas discussed among your colleagues. What if you faked results, proving one of these ideas? And the bar for cheating is _suprisingly_ low. I'm not a researcher myself, but I've experienced how my work is torn apart in peer review. But if you're dealing with data, apparently no one makes sure to actually check it before approval.[0][1] [0] https://youtu.be/d2Tm3Yx4HWI?t=846 https://youtu.be/d2Tm3Yx4HWI?t=846 (analysis on the original data from the Francesca Gino scandal, which already looks suspicious without statistical analysis) [1] https://youtu.be/KsSuhP60qnI https://youtu.be/KsSuhP60qnI (faked data from the Hendrik Schoen scandal at Bell Labs, as long as you make outrageous claims no one can verify, you can continue to publish in top journals)
  - Al-Khwarizmi 3y ago
    I suppose this is heavily discipline-specific. In my specific subfield of CS, the execution is often a Python program or bunch of scripts that can be (and often are) coded by an undergrad if given enough guidance on what exactly is wanted. And then of course the way of presenting it in the paper, choosing which data to show, etc. but cheating doesn't help much there. The hard part in my view is coming up with what Python program should be written (i.e. coming up with a relevant algorithm, experiment, evaluation methodology, etc.). I can see how in other fields, e.g. biomedical, execution could be more challenging and ideas more plentiful. Anyway, I don't think ideas (read: good ones, because obviously anyone can have nonsense ideas) can be a commodity in any field. It is quite widely accepted that, e.g., when applying for an ERC grant (the highest-regarded individual research grants in Europe), having a good idea is as important as the CV if not more. This wouldn't be the case if coming up with good ideas were so trivial.
LatteLazy 3y ago
Any competition without an effective referee (from the Tour to Academics to presidential races) will devolve to the lowest level. People always seem surprised by this...
brailsafe 3y ago
I suppose you could probably also say the same of certain housing markets. If the cost to rent or buy a place is sufficiently high and the system actively prevents the prices ever coming down to a point where people who didn't inherit huge sums or came into that money some other way have a chance, you might as well do whatever you can because the constraints have been laid before you. At a certain point, you're just evening the playing field and surviving.

nemo44x 3y ago

Universities are in the business of creating/discovering truth and shipping it. A Positivist review quickly shows things like math are easy to prove true. As you work your ways down to the humanities it becomes increasingly difficult. And competitive. Shipping truth that isn’t powerful isn’t as attractive to your customers as truth that is. So you’re incentivized to develop truth that is. And you do this by hacking the accepted standards in knowledge pursuit by starting with a conclusion and working backwards. You tell a story based on the evidence that’s useful. Now it’s “the science” or at the very least “there’s studies” and this is useful to both the customers (NGOs, journals, activists, lobbyists, media, anyone that wants to influence policy) and the university (attracts money, reputation and status) and the people shipping it (tenure, book deals, speaking fees). It’s not a conspiracy. It’s just simple incentives. The poor guy who spends his time figuring out what the truth is not or that ships truth that isn’t immediately useful to the customers is looking for a new job after his grant dries up.

derbOac 3y ago
The other level of problem is that truth in reality means consensus. You can argue about what truth is in reality, but functionally it means consensus. So it's not even about what is or is not true, or what is recognized as useful in the short term, it's that what defines truth at any given time is what is popularly believed to be true. This sounds very flaky but it has real consequences because the incentives aren't just to find positive results, it's to find positive results that are likely to be accepted by the majority of the academic field. This kind of positive feedback loop is bound to lead to pressure because science almost by definition has to get some things incorrect — it's how progress is made. So people are actively punished for going against the grain, or exploring areas where our knowledge is limited and therefore prone to lots of negative results because we just don't know.
- nemo44x 3y ago
  > The other level of problem is that truth in reality means consensus That’s the job of the prestigious media. They manufacture consent to the masses. They propagate right-think. Again, not a conspiracy but simply incentives. Anyways, there is objective truth. Not everything can be settled that way though and you’re right in that “truth” can mean consensus in many instances. The problem with that is that in many things the dissent is tasked with proving claims untrue, because of the consensus, rather than those making claim to knowledge having to prove it is true. There’s a lot of this today.
- mcpackieh 3y ago
  Perceived truth is consensus. But greater truth, reality itself, remains true even if not a single person is able to perceive it. You can't actually modify reality with wishful thinking.
- cb504 3y ago
  For these scientific papers, at a minimum truth==reproducibility. For humanities and social sciences, it could mean consensus, but in hard science it is different. You are doing experiments that produce data. If someone else repeats the experiment, they should get the same data. In this case, no one could reproduce the results of the papers in question. I wonder how widespread this has become. How many scientists, driven to publish, know that it is too expensive to reproduce their experiment?

Graziano_M 3y ago

> It actually became a contest, not just of bicycling, but of doping, because getting away with it was quite difficult and it took years for the world of cycling to get good enough countermeasures to shift the advantage to non-doping. Not just a contest of hiding doping, but of winning against others who were doping. If everyone is doping, then you could argue it's still a fair playing field, just with a higher skill ceiling.

noah_buddy 3y ago
It’s not a fair playing field because there were still people who did not cheat.
- Graziano_M 3y ago
  That's why I said, "If everyone is doping" :)
  - optimalsolver 3y ago
    But that's not what you were actually implying.
- DrBenCarson 3y ago
  As long as they knew others were doping, I would argue it was still a fair playing field. The real advantage is in others not knowing you are illegally enhancing performance.
- jnsaff2 3y ago
  https://en.m.wikipedia.org/wiki/Christophe_Bassons https://en.m.wikipedia.org/wiki/Christophe_Bassons
friendzis 3y ago
> If everyone is doping, then you could argue it's still a fair playing field, just with a higher skill ceiling. No, because now you have at least two contests in parallel: the race event for the cyclist and through-the-year "doping and not getting caught" for medical/management team. For example one prominent olympic level swimmer was disqualified because they failed to appear for testing multiple times. Why would one do that just to come back after disqualification?

johntiger1 3y ago

You've hit the nail head on. Also applies to China's anti-corruption programs. Everyone who achieves any level of political power is corrupt, but framing it in an anti-corruption campaign allows you to target your political enemies.

FrustratedMonky 3y ago
No need to single out China. All politics, and guess for Academia, can extend to all human organizations, Church, School, Government.
- merman 3y ago
  You’re making the preposterous assertion that all countries’ politicians are equally corrupt. That needs some extraordinary data to support.
  - FrustratedMonky 3y ago
    Didn't say equal. Just that all 'large' human organizations tend towards corruption. Churches, Governments, Academia. Hierarchies form, incentives drift over time and become mismatched with original intent. The proof is in history, look at Church abuse, Inquisition, look at grade fixing, admissions scandals, look at Volkswagen tweaking emissions, look at CEO's lying to promote products. Our entire society is based on the ebb and flow of some kind of corruption, or ebb-flow of Law. Even today, in America, half the population think that the laws themselves are corrupt and should be broken. Judges aren't trusted. So who is corrupt if you don't have any foundation to judge anything against? I'm totally against China taking Taiwan, but from a 'certain point of view', they have point. It was a part of China before the war, and the mainland 'won'. So they think of it as unification. Again. I am totally against China taking Taiwan. Just think it is bit more nuanced. Actually, think we are agreeing.
    - merman 3y ago
      If there’s no need to ‘single out’ China then there’s no need to ‘single out’ doping in cycling, and no need to ‘single out’ this one particular Stanford President researcher, etc. The objective of your comment, that I am pointing out, is that you are hand-waving corruption in Chinese politics away by saying “but what about…”

azuriten 3y ago

> There was at least one year's Tour de France in which all but one participant were later found to have been doping. In other words, you didn't get to that level of that competitive activity, if you didn't cheat, because it wasn't possible to outperform people with such a significant advantage. This is not true. Even during the 1904 Tour de France where 9 people were disqualifed because of, among other actions, illegal use of cars or trains [1] - 27 riders finished the race. Tour de France in the modern era has up to 180+ competitors lining up, and there hasn't been a case of 100+ riders being disqualified for doping. [1] https://en.wikipedia.org/wiki/1904_Tour_de_France#Disqualification https://en.wikipedia.org/wiki/1904_Tour_de_France#Disqualifi...

rukuu001 3y ago
Yeah not the whole peleton, but most of Armstrong’s main rivals right? Like Contador etc. (I know his test was contentious).
areyousure 3y ago
Indeed, the claim needs to be adjusted to be correct. One of the worst recent years was 2005: https://en.wikipedia.org/wiki/Doping_at_the_Tour_de_France#2005_Tour_de_France https://en.wikipedia.org/wiki/Doping_at_the_Tour_de_France#2... There, the claim is that of the top 10 finishers, all but one participant were either stripped of result in that race or failed tests/were sanctioned for doping in another race. (The remaining participant, originally #8 Cadel Evans, was a documented client of Michele Ferrari.)
- tfyoung 3y ago
  A one-off consultation prior to his road racing career to see if he the right kind of fitness for road. I don't think the association should be used to tarnish Cadel without further evidence.
rossdavidh 3y ago
Thank you for the correction, I either heard wrong back then, or misremembered. I think the analogy/logic still applies, though. Actually, this suggests even more strongly that the higher their position in a competitive field, the more likely they are doping...

alijrtaliw 3y ago

I (and others before and after me) filed complaints that my advisor at UT Austin was falsifying data. University ignored me and gave him tenure. It is commonplace and admin doesn't care as long as it doesn't affect the endowment.

canjobear 3y ago
Are you willing to say who it is?
- bigfoot675 3y ago
  Yeah I'm also curious
deleted 3y ago
[deleted]

EvgeniyZh 3y ago

There are at least two problems with this argument. First, faking a research in a way that won't be detected is hard, especially if aim for high impact paper which probably will be studied by others. That's why we hear a lot about fraud in top tier venues. Lower ones probably have more problematic publications. Second, having fake results won't guarantee you a high quality publication, leave alone position in the future. I can imagine some kind of first order phase transition in which the amount of fraudsters becomes so large that self correction breaks and instead they start to endorse each other. But that'd be evident from the outside, since science will stop producing real life results. We're definitely not there yet.

jvans 3y ago
It doesn't have to be outright fraud. I imagine the vast majority of manipulations are extremely subtle. Intentional selection bias, multiple hypothesis testing without correction or reporting the failed hypotheses(p hacking). Most of these would improve your chances of finding important results significantly and be almost impossible to detect
- EvgeniyZh 3y ago
  For the first argument I can't really speak beyond my field (physics, or specifically condensed matter physics), where I think it's fairly hard to be subtle. Often it's partial data presentation which allows for inverse Occam razor [1] (proposing fancy explaination instead of a simple one), which is why full data publication requirement are important (and seem to happen more). In smaller sub (sub)fields sometimes fraud becomes large enough to self-sustain, like it happened with Majoranas [2,3] but even that kind of self-corrected. [1] https://arxiv.org/abs/2204.08284 https://arxiv.org/abs/2204.08284 [2] https://twitter.com/spinespresso/status/1503352928656138241 https://twitter.com/spinespresso/status/1503352928656138241 [3] https://espressospin.org/2022/11/17/the-fallen-angel-particle/ https://espressospin.org/2022/11/17/the-fallen-angel-particl...
- mcpackieh 3y ago
  > without reporting the failed hypotheses(p hacking). This one is so subtle, it's probable that many of the people doing it don't even realize they're doing something wrong.

AmericanChopper 3y ago

Doping in sports is so much more complicated than most people think it is. For starters there’s so many performance enhancing substances, and entire underground industries for developing and producing them. All of these drugs would be banned by the blanket prohibition on unapproved substances, but in practice many of them don’t have any screening procedures, so won’t turn up in any tests. There’s also the fact that many performance enhancing substances are biologically identical to substances your body is naturally producing (like bio-identical testosterone for example). The only way to test for these is to test the ratios of certain metabolites (with a margin for error to avoid false positives). In practice this basically means any athlete can do steroid cycles, as long as they are only boosting their test levels up to that of a genetically elite outlier. You also have the therapeutic use exemption, which allows for legitimate therapeutic use of banned substances. Testosterone replacement therapy is a legitimate therapeutic use of steroids, and some sports are known for abusing TUEs for this usecase. Testing for PEDs is way less precise than a lot of people think it is, and any motivated athlete can get away with PED use to some extent, no matter how much testing you subject them to. There’s good reason to be suspicious of a lot of athletes. One massively suspect category of athlete is all of the ones in/near their 40s who somehow manage to maintain physiques as good as/better than they had in their early 20s.

notknifescience 3y ago
I would be very much interested in reading more about this, do you have any good articles you could point me to?
- AmericanChopper 3y ago
  This paper is a bit out of date, but gives a reasonable overview of the challenges of testing for testosterone supplementation https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2657495/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2657495/ Paper on challenges of detecting growth hormone https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830297/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830297/ Testing for both of those substances has improved, but still leave plenty of room for abuse. Use of "unknown" or the constant stream newly developed substances is naturally not rigorously studied, but here's a couple of articles referencing the challenges they pose. https://thetruthaboutforensicscience.com/undetectable-performance-enhancing-drugs-in-sports/ https://thetruthaboutforensicscience.com/undetectable-perfor... https://bleacherreport.com/articles/1955231-undetectable-the-new-ped-that-could-be-in-the-olympics-nfl-and-mlb-now https://bleacherreport.com/articles/1955231-undetectable-the... UFC is notorious for testosterone TUEs, but that's mostly because they regulate things differently to most other sports https://www.espn.com/espn/otl/story/_/id/10500652/therapeutic-use-exemptions-testosterone-mma-outpace-other-sports-lines-finds https://www.espn.com/espn/otl/story/_/id/10500652/therapeuti... Top flight soccer is another sport that's constantly garnering attention for TUEs, but most of the coverage of that is speculative, because other than high level statistics the TUE process is naturally not very transparent (and most of the recent coverage on the topic has been replaced by controversy relating to trans athletes).

Georgelemental 3y ago

I know several Tour de France aficionados, who would claim that doping remains rampant, they are just better at hiding it now.

deleted 3y ago
[deleted]

Al-Khwarizmi 3y ago

I don't think this kind of fraud is even remotely close in scale to the cycling doping you mention. My primary reason for believing this is that as a scientist myself, I've never done it, and none of my coauthors has ever proposed to do something like that. I have seen other kinds of fraud (e.g. ghost authorship) but not result manipulation. But of course, If I were a fraudster myself I would also say that I've never done it :) So a more general argument: scientific "doping" is just not as straightforward as cycling doping. In cycling, it's clear that if you don't get caught (and have no scruples) you benefit from doping, it makes you stronger. In science, you first need an idea that looks original/promising, and then you do the experiments and see if the idea works or not. This kind of "doping" will only help if you (1) are good at generating ideas, but (2) these ideas often don't work. At least in my field, this is possible but far from the most common situation to be. And in other situations, there is not much benefit to faking results.

awesomeMilou 3y ago
> if you (1) are good at generating ideas I have been joking with my former boss (who is a professor) about how easy it is to come up with good ideas. We each have five to six One Billion Dollar ideas before breakfast, and another 1-2 while in the shower! /s Coming up with ideas is _ridicolously_ easy, executing on them is the hard part. If you're a seasoned researcher, it shouldn't be hard to guess at the "meta" of your field, you're probably aware of promising ideas discussed among your colleagues. What if you faked results, proving one of these ideas? And the bar for cheating is _suprisingly_ low. I'm not a researcher myself, but I've experienced how my work is torn apart in peer review. But if you're dealing with data, apparently no one makes sure to actually check it before approval.[0][1] [0] https://youtu.be/d2Tm3Yx4HWI?t=846 https://youtu.be/d2Tm3Yx4HWI?t=846 (analysis on the original data from the Francesca Gino scandal, which already looks suspicious without statistical analysis) [1] https://youtu.be/KsSuhP60qnI https://youtu.be/KsSuhP60qnI (faked data from the Hendrik Schoen scandal at Bell Labs, as long as you make outrageous claims no one can verify, you can continue to publish in top journals)
- Al-Khwarizmi 3y ago
  I suppose this is heavily discipline-specific. In my specific subfield of CS, the execution is often a Python program or bunch of scripts that can be (and often are) coded by an undergrad if given enough guidance on what exactly is wanted. And then of course the way of presenting it in the paper, choosing which data to show, etc. but cheating doesn't help much there. The hard part in my view is coming up with what Python program should be written (i.e. coming up with a relevant algorithm, experiment, evaluation methodology, etc.). I can see how in other fields, e.g. biomedical, execution could be more challenging and ideas more plentiful. Anyway, I don't think ideas (read: good ones, because obviously anyone can have nonsense ideas) can be a commodity in any field. It is quite widely accepted that, e.g., when applying for an ERC grant (the highest-regarded individual research grants in Europe), having a good idea is as important as the CV if not more. This wouldn't be the case if coming up with good ideas were so trivial.

LatteLazy 3y ago

Any competition without an effective referee (from the Tour to Academics to presidential races) will devolve to the lowest level. People always seem surprised by this...

brailsafe 3y ago

I suppose you could probably also say the same of certain housing markets. If the cost to rent or buy a place is sufficiently high and the system actively prevents the prices ever coming down to a point where people who didn't inherit huge sums or came into that money some other way have a chance, you might as well do whatever you can because the constraints have been laid before you. At a certain point, you're just evening the playing field and surviving.