7 ms·

Students and researchers send their own paper to plagiarism checker to look for "real" and unintended flags before actually submitting the papers, and make revi

by rs186 2y ago

Students and researchers send their own paper to plagiarism checker to look for "real" and unintended flags before actually submitting the papers, and make revisions accordingly. This is a known, standard practice that is widely accepted. And let's say someone modifies their faked lab results so that no AI can detect any evidence of photoshopping images. Their results get published. Well, nobody will be able to reproduce their work (unless other people also publish fraudulent work from there), and fellow researchers will raise questions, like, a lot of them. Also, guess what, even today, badly photoshopped results often don't get caught for a few years, and in hindsight that's just some low effort image manipulation -- copying a part of image and paste it elsewhere. I doubt any of this changes anything. There is a lot of competition in academia, and depending on the field, things may move very fast. Getting away with AI detection of fraudulent work likely doesn't give anyone enough advantage to survive in a competitive field.

dccsillag 2y ago
I've never seen this done in a research setting. Not sure about how much of a standard practice it is.
- StableAlkyne 2y ago
  It may be field specific, but I've also never heard of anyone running a manuscript through a plagiarism checker in chemistry.
  - deleted 2y ago
    [deleted]
abirch 2y ago
You're right that this won't change the incentives for the dishonest researchers. Unfortunately there's not an equivalent of "short sellers" in research, people who are incentivized for finding fraud. AI is definitely a good thing (TM) for those honest researchers.
owl_vision 2y ago
Unless documented and reproducible, it does not exist. This was the minimum guide when I worked with researchers. I plus 1 your doubt in the last paragraph.
BurningFrog 2y ago
Not in academia, but what I hear is that very few results are ever attempted to be reproduced. So if you publish an unreproducible paper, you can probably have a full career without anyone noticing.
- jfengel 2y ago
  Papers that can't be reproduced sound like they're not very useful, either. I know it's not as simple as that, and "useful" can simply mean "cited" (a sadly overrated metric). But surely it's easier to get hired if your work actually results in something somebody uses.
  - dgfitz 2y ago
    > Papers that can't be reproduced sound like they're not very useful, either. They’re not useful at all. Reproduction of results isn’t sexy, nobody does it. Almost feels like science is built on a web on funding trying to buy the desired results.
    - qpiox 2y ago
      Reproduction is rarely done because it is not "new science". Everyone is funding only "new science".
    - jfengel 2y ago
      Reproduction is boring, but it would often happen incidentally to building off someone else's results. You tell me that this reaction creates X, and I need X to make Y. If I can't make my Y, sooner or later it's going to occur to me that X is the cause. Like I said, I know it's never that easy. Bench work is hard and there are a million reasons why your idea failed, and you may not take the time to figure out why. You won't report such failures. And complicated results, like in sociology, are rarely attributable to anything.
      - mike_hearn 2y ago
        That's true for some kinds of research but a lot of academic output isn't as firm as "X creates Y". Replicability is overrated anyway. Loads of bad papers will replicate just fine if you try. They're still making false claims. https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science...
azan_ 2y ago
>Their results get published. Well, nobody will be able to reproduce their work (unless other people also publish fraudulent work from there), and fellow researchers will raise questions, like, a lot of them. Sadly you seem to underestimate how widespread fraud is in academia and overestimate how big the punishment is. In the worst case when someone finds you are guilty of fraud, you will get slap on the wrist. In the usual case absolutely nothing will happen and you will be free to keep publishing fraud.
- matthewdgreen 2y ago
  I don’t actually believe that this is true if “academia” is defined as the set of reputable researchers from R1 schools and similar. If you define Academia as “anyone anywhere in the world who submits research papers” then yes, it has vast amounts of fraud in the same way that most email is spam. Within the reputable set, as someone convinced that fraud is out of control, have you ever tried to calculate the fraud rate as a percentage with numerator and denominator (either number of papers published or number of reputable researchers. I would be very interested and stunned if it was over .1% or even .01%.
  - azan_ 2y ago
    There is lots of evidence that p-hacking is widespread (some estimate that up to 20% are p-hacked). This problem also exists in top instutions, in fact in some fields it appears that this problem is WORSE in higher ranking unis - https://mitsloan.mit.edu/sites/default/files/inline-files/P-hacking%20in%20Experimental%20Accounting%20Studies.pdf https://mitsloan.mit.edu/sites/default/files/inline-files/P-...
    - cycomanic 2y ago
      Where is that evidence? The paper you cite suggests that p hacking is done in experimental accounting studies but not archival. Generally speaking, evidence suggests that fraud rates are low ( lower than in most other human endeavours). This study cites 2% [1]. This is similar to numbers that Elizabeth Bik reports. For comparison self reported doping rates were between 6 and 9% here [2] [1] https://pmc.ncbi.nlm.nih.gov/articles/PMC5723807/ https://pmc.ncbi.nlm.nih.gov/articles/PMC5723807/ [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC11102888/ https://pmc.ncbi.nlm.nih.gov/articles/PMC11102888/
cycomanic 2y ago
Which researchers are using plagiarism detectors? I'm not aware that this is a known and widely accepted practice. They are used by students and teachers for student papers (in courses etc), but nobody i know would use them for submitting research. I also don't see the point for why even unethical researchers would use it, it wouldn't increase your acceptance chances dramatically.
mezyt 2y ago
> Well, nobody will be able to reproduce their work (unless other people also publish fraudulent work from there) In theory, yes, in practice, the original result for amyloid beta protein as the main cause of Alzheimer were faked and it wasn't caught for 16 years. A member of my family took med based on it and died in the meantime.

by rs186 2y ago

dccsillag 2y ago

I've never seen this done in a research setting. Not sure about how much of a standard practice it is.

StableAlkyne 2y ago
It may be field specific, but I've also never heard of anyone running a manuscript through a plagiarism checker in chemistry.
- deleted 2y ago
  [deleted]

abirch 2y ago

You're right that this won't change the incentives for the dishonest researchers. Unfortunately there's not an equivalent of "short sellers" in research, people who are incentivized for finding fraud. AI is definitely a good thing (TM) for those honest researchers.

owl_vision 2y ago

Unless documented and reproducible, it does not exist. This was the minimum guide when I worked with researchers. I plus 1 your doubt in the last paragraph.

BurningFrog 2y ago

Not in academia, but what I hear is that very few results are ever attempted to be reproduced. So if you publish an unreproducible paper, you can probably have a full career without anyone noticing.

jfengel 2y ago
Papers that can't be reproduced sound like they're not very useful, either. I know it's not as simple as that, and "useful" can simply mean "cited" (a sadly overrated metric). But surely it's easier to get hired if your work actually results in something somebody uses.
- dgfitz 2y ago
  > Papers that can't be reproduced sound like they're not very useful, either. They’re not useful at all. Reproduction of results isn’t sexy, nobody does it. Almost feels like science is built on a web on funding trying to buy the desired results.
  - qpiox 2y ago
    Reproduction is rarely done because it is not "new science". Everyone is funding only "new science".
  - jfengel 2y ago
    Reproduction is boring, but it would often happen incidentally to building off someone else's results. You tell me that this reaction creates X, and I need X to make Y. If I can't make my Y, sooner or later it's going to occur to me that X is the cause. Like I said, I know it's never that easy. Bench work is hard and there are a million reasons why your idea failed, and you may not take the time to figure out why. You won't report such failures. And complicated results, like in sociology, are rarely attributable to anything.
    - mike_hearn 2y ago
      That's true for some kinds of research but a lot of academic output isn't as firm as "X creates Y". Replicability is overrated anyway. Loads of bad papers will replicate just fine if you try. They're still making false claims. https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science...

azan_ 2y ago

>Their results get published. Well, nobody will be able to reproduce their work (unless other people also publish fraudulent work from there), and fellow researchers will raise questions, like, a lot of them. Sadly you seem to underestimate how widespread fraud is in academia and overestimate how big the punishment is. In the worst case when someone finds you are guilty of fraud, you will get slap on the wrist. In the usual case absolutely nothing will happen and you will be free to keep publishing fraud.

matthewdgreen 2y ago
I don’t actually believe that this is true if “academia” is defined as the set of reputable researchers from R1 schools and similar. If you define Academia as “anyone anywhere in the world who submits research papers” then yes, it has vast amounts of fraud in the same way that most email is spam. Within the reputable set, as someone convinced that fraud is out of control, have you ever tried to calculate the fraud rate as a percentage with numerator and denominator (either number of papers published or number of reputable researchers. I would be very interested and stunned if it was over .1% or even .01%.
- azan_ 2y ago
  There is lots of evidence that p-hacking is widespread (some estimate that up to 20% are p-hacked). This problem also exists in top instutions, in fact in some fields it appears that this problem is WORSE in higher ranking unis - https://mitsloan.mit.edu/sites/default/files/inline-files/P-hacking%20in%20Experimental%20Accounting%20Studies.pdf https://mitsloan.mit.edu/sites/default/files/inline-files/P-...
  - cycomanic 2y ago
    Where is that evidence? The paper you cite suggests that p hacking is done in experimental accounting studies but not archival. Generally speaking, evidence suggests that fraud rates are low ( lower than in most other human endeavours). This study cites 2% [1]. This is similar to numbers that Elizabeth Bik reports. For comparison self reported doping rates were between 6 and 9% here [2] [1] https://pmc.ncbi.nlm.nih.gov/articles/PMC5723807/ https://pmc.ncbi.nlm.nih.gov/articles/PMC5723807/ [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC11102888/ https://pmc.ncbi.nlm.nih.gov/articles/PMC11102888/

cycomanic 2y ago

Which researchers are using plagiarism detectors? I'm not aware that this is a known and widely accepted practice. They are used by students and teachers for student papers (in courses etc), but nobody i know would use them for submitting research. I also don't see the point for why even unethical researchers would use it, it wouldn't increase your acceptance chances dramatically.

mezyt 2y ago

> Well, nobody will be able to reproduce their work (unless other people also publish fraudulent work from there) In theory, yes, in practice, the original result for amyloid beta protein as the main cause of Alzheimer were faked and it wasn't caught for 16 years. A member of my family took med based on it and died in the meantime.