4 ms·

0.04% is an extremely small number and (it needs to be said) also includes papers retracted due to errors and other good-faith corrections. Remember that we wan

0.04% is an extremely small number and (it needs to be said) also includes papers retracted due to errors and other good-faith corrections. Remember that we want people to retract flawed papers! treating it as evidence of fraud is not only a mischaracterization of the result but also a choice that is bad for a society that wants quality scientific results. The other two metrics seem pretty weak. 1.9% of papers in a vast database containing 40 journals show signs of duplication. But then dig into the details: apparently a huge fraction of those are in one journal and in two specific years. Look at Figure 1 and it just screams “something very weird is going on here, let’s look closely at this methodology before we accept the top line results.” The final result is a meta-survey based on surveys done across scientists all over the world, including surveys that are written in other languages, presumably based on scientists also publishing in smaller local journals. Presumably this covers a vast range of scientists with different reputations. As I said before, if you cast a wide net that includes everyone doing science in the entire world, I bet you’ll find tons of fraud. This study just seems to do that.

mike_hearn 2y ago
The point about 0.04% is not that it's low, it's that it should be much higher. Getting even obviously fraudulent papers retracted is difficult and the image duplications are being found by unpaid volunteers, not via some comprehensive process so the numbers are lower bounds, not upper. You can find academic fraud in bulk with a tool as simple as grep and yet papers found that way are typically not retracted. Example, select the tortured phrases section of this database. It's literally nothing fancier than a big regex: https://dbrech.irit.fr/pls/apex/f?p=9999:24 https://dbrech.irit.fr/pls/apex/f?p=9999:24:::::: Randomly chosen paper: https://link.springer.com/article/10.1007/s11042-025-20660-1 https://link.springer.com/article/10.1007/s11042-025-20660-1 "A novel approach on heart disease prediction using optimized hybrid deep learning approach", published in Multimedia Tools and Applications. This paper has been run through a thesaurus spinner yielding garbage text like "To advance the expectation exactness of the anticipated heart malady location show" (heart disease -> heart malady). It also has nothing to do with the journal it's published in. Now you might object that the paper in question comes from India and not an R1 American university, which is how you're defining reputable. The journal itself does, though. It's edited by an academic in the Dept. of Computer Science and Engineering, Florida Atlantic University, which is an R1. It also has many dozens of people with the title of editor at other presumably reputable western universities like Brunel in the UK, the University of Salerno, etc: https://link.springer.com/journal/11042/editorial-board https://link.springer.com/journal/11042/editorial-board Clearly, none of the so-called editors of the journal can be reading what's submitted to it. Zombie journals run by well known publishers like Spring Nature are common. They auto-publish blatant spam yet always have a gazillion editors at well known universities. This stuff is so basic both generation and detection predate LLMs entirely, but it doesn't get fixed. Then you get into all the papers that aren't trivially fake but fake in advanced undetectable ways, or which are merely using questionable research practices... the true rate of retraction if standards were at the level laymen imagine would be orders of magnitude higher.
- fc417fc802 2y ago
  > found by unpaid volunteers, not via some comprehensive process "Unpaid volunteers" describes the majority of the academic publication process so I'm not sure what you're point is. It's also a pretty reasonable approach - readers should report issues. This is exactly how moderation works the web over. Mind that I'm not arguing in favor of the status quo. Merely pointing out that this isn't some smoking gun. > you might object that the paper in question comes from India and not an R1 American university Yes, it does rather seem that you're trying to argue one thing (ie the mainstream scientific establishment of the western world is full of fraud) while selecting evidence from a rather different bucket (non-R1 institutions, journals that aren't mainstream, papers that aren't widely cited and were probably never read by anyone). > The journal itself does, though. It's edited by an academic in ... That isn't how anyone I've ever worked with assessed journal reputability. At a glance that journal doesn't look anywhere near high end to me. Remember that, just as with books, anyone can publish any scientific writeup that they'd like. By raw numbers, most published works of fiction aren't very high quality.[0] That doesn't say anything about the skilled fiction authors or the industry as a whole though. > but it doesn't get fixed. Is there a problem to begin with? People are publishing things. Are you seriously suggesting that we attempt to regulate what people are permitted to publish or who academics are permitted to associate with on the basis of some magical objective quality metric that doesn't currently exist? If you go searching for trash you will find trash. Things like industry and walk of life have little bearing on it. Trash is universal. You are lumping together a bunch of different things that no professional would ever consider to belong to the same category. If you want to critique mainstream scientific research then you need to present an analysis of sources that are widely accepted as being mainstream. [0] https://www.goodreads.com/book/show/18628458-taken-by-the-t-rex https://www.goodreads.com/book/show/18628458-taken-by-the-t-...
  - mike_hearn 2y ago
    The inconsistent standards seen in this type of discussion damages sympathy amongst the public, and causes people who could be allies in future to just give up. Every year there are more articles on scientific fraud appear in all kinds of places, from newspapers to HN to blogs yet the reaction is always https://prod-printler-front-as.azurewebsites.net/media/photo/101063.jpg?mode=pad&width=1200&rnd=0.0.1 https://prod-printler-front-as.azurewebsites.net/media/photo... Academics draw a salary to do their job, but when they go AWOL on tasks critical to their profession suddenly they're all unpaid volunteers. This Is Fine. Journals don't retract fraudulent articles without a fight, yet the low retraction rate is evidence that This Is Fine. The publishing process is a source of credibility so rigorous it places academic views well above those of the common man, but when it publishes spam on auto-pilot suddenly journals are just some kind of abandoned subreddit and This Is Fine "but I'm not arguing in favor of it". And the darned circular logic. Fraud is common but This Is Fine because reputable sources don't do it, where the definition of reputable is totally ad-hoc beyond not engaging in fraud. This thread is an exemplar: today reputable means American R1 universities because they don't do bad stuff like that, except when their employees sign off on it but that's totally different. The editor of The Lancet has said probably half of what his journal publishes is wrong [1] but This Is Fine until there's "an analysis of sources that are widely accepted as being mainstream". Reputability is meaningless. Many of the supposedly top universities have hosted star researchers, entire labs [2] and even presidents who were caught doing long cons of various kinds. This Is Not Fine. [1] https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736%2815%2960696-1.pdf https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736... [2] https://arstechnica.com/science/2024/01/top-harvard-cancer-researchers-accused-of-scientific-fraud-37-studies-affected/ https://arstechnica.com/science/2024/01/top-harvard-cancer-r...

mike_hearn 2y ago

The point about 0.04% is not that it's low, it's that it should be much higher. Getting even obviously fraudulent papers retracted is difficult and the image duplications are being found by unpaid volunteers, not via some comprehensive process so the numbers are lower bounds, not upper. You can find academic fraud in bulk with a tool as simple as grep and yet papers found that way are typically not retracted. Example, select the tortured phrases section of this database. It's literally nothing fancier than a big regex: https://dbrech.irit.fr/pls/apex/f?p=9999:24 https://dbrech.irit.fr/pls/apex/f?p=9999:24:::::: Randomly chosen paper: https://link.springer.com/article/10.1007/s11042-025-20660-1 https://link.springer.com/article/10.1007/s11042-025-20660-1 "A novel approach on heart disease prediction using optimized hybrid deep learning approach", published in Multimedia Tools and Applications. This paper has been run through a thesaurus spinner yielding garbage text like "To advance the expectation exactness of the anticipated heart malady location show" (heart disease -> heart malady). It also has nothing to do with the journal it's published in. Now you might object that the paper in question comes from India and not an R1 American university, which is how you're defining reputable. The journal itself does, though. It's edited by an academic in the Dept. of Computer Science and Engineering, Florida Atlantic University, which is an R1. It also has many dozens of people with the title of editor at other presumably reputable western universities like Brunel in the UK, the University of Salerno, etc: https://link.springer.com/journal/11042/editorial-board https://link.springer.com/journal/11042/editorial-board Clearly, none of the so-called editors of the journal can be reading what's submitted to it. Zombie journals run by well known publishers like Spring Nature are common. They auto-publish blatant spam yet always have a gazillion editors at well known universities. This stuff is so basic both generation and detection predate LLMs entirely, but it doesn't get fixed. Then you get into all the papers that aren't trivially fake but fake in advanced undetectable ways, or which are merely using questionable research practices... the true rate of retraction if standards were at the level laymen imagine would be orders of magnitude higher.

fc417fc802 2y ago
> found by unpaid volunteers, not via some comprehensive process "Unpaid volunteers" describes the majority of the academic publication process so I'm not sure what you're point is. It's also a pretty reasonable approach - readers should report issues. This is exactly how moderation works the web over. Mind that I'm not arguing in favor of the status quo. Merely pointing out that this isn't some smoking gun. > you might object that the paper in question comes from India and not an R1 American university Yes, it does rather seem that you're trying to argue one thing (ie the mainstream scientific establishment of the western world is full of fraud) while selecting evidence from a rather different bucket (non-R1 institutions, journals that aren't mainstream, papers that aren't widely cited and were probably never read by anyone). > The journal itself does, though. It's edited by an academic in ... That isn't how anyone I've ever worked with assessed journal reputability. At a glance that journal doesn't look anywhere near high end to me. Remember that, just as with books, anyone can publish any scientific writeup that they'd like. By raw numbers, most published works of fiction aren't very high quality.[0] That doesn't say anything about the skilled fiction authors or the industry as a whole though. > but it doesn't get fixed. Is there a problem to begin with? People are publishing things. Are you seriously suggesting that we attempt to regulate what people are permitted to publish or who academics are permitted to associate with on the basis of some magical objective quality metric that doesn't currently exist? If you go searching for trash you will find trash. Things like industry and walk of life have little bearing on it. Trash is universal. You are lumping together a bunch of different things that no professional would ever consider to belong to the same category. If you want to critique mainstream scientific research then you need to present an analysis of sources that are widely accepted as being mainstream. [0] https://www.goodreads.com/book/show/18628458-taken-by-the-t-rex https://www.goodreads.com/book/show/18628458-taken-by-the-t-...
- mike_hearn 2y ago
  The inconsistent standards seen in this type of discussion damages sympathy amongst the public, and causes people who could be allies in future to just give up. Every year there are more articles on scientific fraud appear in all kinds of places, from newspapers to HN to blogs yet the reaction is always https://prod-printler-front-as.azurewebsites.net/media/photo/101063.jpg?mode=pad&width=1200&rnd=0.0.1 https://prod-printler-front-as.azurewebsites.net/media/photo... Academics draw a salary to do their job, but when they go AWOL on tasks critical to their profession suddenly they're all unpaid volunteers. This Is Fine. Journals don't retract fraudulent articles without a fight, yet the low retraction rate is evidence that This Is Fine. The publishing process is a source of credibility so rigorous it places academic views well above those of the common man, but when it publishes spam on auto-pilot suddenly journals are just some kind of abandoned subreddit and This Is Fine "but I'm not arguing in favor of it". And the darned circular logic. Fraud is common but This Is Fine because reputable sources don't do it, where the definition of reputable is totally ad-hoc beyond not engaging in fraud. This thread is an exemplar: today reputable means American R1 universities because they don't do bad stuff like that, except when their employees sign off on it but that's totally different. The editor of The Lancet has said probably half of what his journal publishes is wrong [1] but This Is Fine until there's "an analysis of sources that are widely accepted as being mainstream". Reputability is meaningless. Many of the supposedly top universities have hosted star researchers, entire labs [2] and even presidents who were caught doing long cons of various kinds. This Is Not Fine. [1] https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736%2815%2960696-1.pdf https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736... [2] https://arstechnica.com/science/2024/01/top-harvard-cancer-researchers-accused-of-scientific-fraud-37-studies-affected/ https://arstechnica.com/science/2024/01/top-harvard-cancer-r...