9 ms·
Fake scientific papers are alarmingly common
- just_a_quack 3y agoI remember when scigen successfully published one or two papers in a journal.
- nephanth 3y agoI wonder what are the incentives/ reasons for producing all those fake papers It's not like paper authors get any kind of royalties. Some journals even make you pay to publish. So why are they doing that? Maybe that's what we need to attack
- vhcr 3y ago> STM hasn’t yet generated figures on accuracy or false-positive rates because the project is too new. But catching as many fakes as possible typically produces more false positives. Sabel’s tool correctly flagged nearly 90% of fraudulent or retracted papers in a test sample. However, it marked up to 44% of genuine papers as fake
- wongarsu 3y ago> so results still need to be confirmed by skilled reviewers So there is some human review involved. Which is presumably how they got to the headline figures of 34% of neuroscience papers and 24% of medicine papers are fake. Still, flagging 44% of genuine papers as fake doesn't sound very useful. The process only about halves your workload compared to just checking all the papers. In any large-scale rollout they would have to set a way higher threshold, and hope they still catch a useful number of fraudulent papers when using a threshold that detects 10% or 1% of genuine papers as fake.
- pvaldes 3y agoIf they thing that people can do research in medicine without having legal access to the patients (AKA some kind of Hospital affiliation), they are clueless about medical research. They don't seem to understand how much hospitals, medical companies and academy are interwined now. Lots of relevant physicians are also teachers, direct a research team or are testers from new products. They are also tagging all independent non affiliated researchers as fake. Do they know how many young people are doing science in the universities as temporary collabo-slaves without right to a nice personal mail?. Their detector would tag Einstein and Erdos as fake scientists by Pete's sake!. They just have a narrow vision tunnel about how the real research works
- mnd999 3y agoJust wait until ChatGPT starts writing them.
- bookofjoe 3y ago>A Doctor Published Several Research Papers With Breakneck Speed. ChatGPT Wrote Them All. https://www.thedailybeast.com/how-this-doctor-wrote-dozens-of-science-papers-with-chatgpt https://www.thedailybeast.com/how-this-doctor-wrote-dozens-o... https://archive.ph/u9wyq https://archive.ph/u9wyq
- mnd999 3y agoYeah, if the incentives are to publish convincing papers with less emphasis on quality or, you know, good research then this is going to happen and it’s going to happen a lot. The article you link doesn’t say if the papers are any good. It does suggest that they were in smaller niche journals so I suspect not. On the upside, I can see the potential though for literature review type papers.
- alsodumb 3y agoUnfortunately, it's an open secret that fake or low-effort almost useless papers are very common in every area of scientific research. Typically, it doesn't affect people working in that specific area - they develop/have a sixth sense to detect bullshit papers - it comes with experience but depends on several factors including the authors reputation, their institution (for the first screening), what journal/conference the paper was published in, authors other work, and sometimes things as simple as how much effort was put into the figures, polishing the text, etc. Some of these things are LLM proof, some of them are not - e.g. a senior professor I was talking to, who's been getting like 50-100 emails a week from non-english speaking countries (primarily India, China, Pakistan, and Bangladesh) mentioned that the quality of the text in the emails went up significantly almost overnight after ChatGPT was made open to public. It'll be interesting to see how things change in the next few months/years.
- 2devnull 3y agoRight. If one can develop a sixth sense about bullshit papers then so too can an LLM. If you have a bullshit paper you will need to pipe it through an LLM to debullshittify it so that reviewers cannot tell. The reviewers themselves may need LLM to fight the rising tide of passable bullshit papers. None of that seems productive to me, just throwing gasoline on the dumpster fire of phacking, credential inflation, publish or perish, etc..
- version_five 3y agoRight- academic papers are written for academicians who don't have any issues separating good papers and journals from bad. The fact that many journals have set themselves up as or allowed themselves to become part of the tenure and promotion metrics game, is more of an issue with tenure and promotion. If the requirement for simple metrics dissapeared, the fake papers would go away on their own. In any event, it's not really a problem for researchers.
- alsodumb 3y agoYup, that’s sums up the incentive for publishing so many papers and get citations. Some professor put it in a nice way - the current system motivates us to think of research in terms of LPUs - least publishing units. No matter how established your lab is, you’d try to publish as soon as possible, leading to a lot of papers with not a lot of contribution. If tenure committees and all other systems that gauge academicians require people to say present their only top 3 or 5 seminal papers, then people would try to put their best work out there without the constant pressure of always publishing - win win for everyone. Unfortunately, the ones with the power to make these changes are the ones gaining the most in the current system so it’s unlikely to happen.
- tyjen 3y agoSadly, this doesn't even include the studies with authors who produce poor experiments and theories, or go out of their way to prove their results; effectively, generating additional scientific publication waste we have to sift through to find genuine material, or worse off, that people then use to create policies impacting large populations that are doomed to fail in the long-run. The image this creates for me, is building a house on quicksand.
- pessimizer 3y agoIn essence, there's more than enough to deal with regarding bad science that was done in good faith. There have got to be better ways to filter out bad science offered in bad faith.
- mycologos 3y agoScientific misconduct like deliberately falsifying results should be more of a career-ender.
- burnished 3y agoI think the incentive structure has to be fixed - either making paper mills unnatractive, or removing the demand for their services, whichever. I think filtering out bad faith efforts is too challenging because the pool of people capable of doing so is so limited, hell it might take longer to review and reject such a thing than to make it.
- tyjen 3y agoAn author and journal rating services evaluating merits across time. While poor scientists become evident to competing authors in their respective fields, a policymaker or journalist may not take the time to figure out if the results are meritable or a background of the authors and utilize the results to support their position. In a perfect world, these authors should be filtered during the "peer review" process, but the process seems... corrupted?
- noufalibrahim 3y agoSo's fake science.
- aurizon 3y agoThe cash cows, AKA Elsevier et al, need to do more to stem the flow of BS. The problem is the proliferation of well crafted, but fake, papers has grown enormously over the past 25 years as the cows rely on free paper editors - who are swamped by this duty = time for paid scientists to winnow the chaff. Sadly the cows are a greedy lot. Only way out is fully open. Back in the day when Nobel was born, the journals and authors circulated as near free resources, with authors mailing free copies on request, and now emailing them (often this is interdicted by the cows) and journal fees being modest - covering production costs. Nobel would be (IMHO) royally pissed at the present state. So I suggest the Nobel Committee introduce a policy that only openly published papers would be read and considered by the committee - This would put a tiger among the pigeons(Cows) and change things - say, after Jan 1 2024?
- bookofjoe 3y agoAs one who published primarily in the 1970s and 1980s [https://scholar.google.com/citations?user=5DdrMc8AAAAJ&hl=en https://scholar.google.com/citations?user=5DdrMc8AAAAJ&hl=en] I can confirm that I mailed reprints of my requested papers to whomever requested them, for free. Note: I paid for the reprints and the postage, often expensive foreign rates.
- smcin 3y agoOk, but how many citations/yr and reprints/yr did you get? The volume of literature has scaled exponentially since.
- ratg13 3y ago>Sabel’s tool relies on just two indicators—authors who use private, noninstitutional email addresses, and those who list an affiliation with a hospital. Can someone explain why the affiliation with a hospital is used as a key indicator?
- revelio 3y agoA huge part of the problem is driven by Chinese hospitals. The PRC decided that they wanted China to catch up to the west in science, so made a role that to get promoted as a doctor you have to get some papers published in international journals. That applies to all doctors. But they're, you know, busy doctoring and don't have time out energy to do that. At the same time they'd like to get promoted. So they buy papers.
- rst 3y agoThe metrics used in this paper are... deeply flawed, to the point that the authors admit that they label nearly half of known good papers in a curated sample as "fake" -- and particularly likely to generate false positives for researchers whose institutions don't, say, run their own email systems (as is common in large chunks of the world). Here's a rundown of the flaws from an epidemiologist with a sideline in scientific communication: https://fediscience.org/@ct_bergstrom/110357259338364341 https://fediscience.org/@ct_bergstrom/110357259338364341
- Eddy_Viscosity2 3y agoIronically, this would mean that this paper is "fake".
- pessimizer 3y agoLooks like the "misinformation" industry is branching out.
- largepeepee 3y agoYou know what's funny? Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Kinda similar to those researchers years back who proved how easy it was to go into certain social science journals as long as you copied their ideology.
- boomboomsubban 3y ago>Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Not by the definition of "fake" used in the article, as the data wouldn't be plagiarized or fabricated. It'd just be shitty data.
- newswasboring 3y agoIt's a medRxive preprint. It didn't get published anywhere. Science (the magazine) has lowered it's standards.
- 3y ago
- dmbche 3y agoI'd love to read it, but it's blocked. Anyone can summarize?
- tracer4201 3y ago[dead]
- __MatrixMan__ 3y ago> Such manuscripts threaten to corrupt the scientific literature, misleading readers and potentially distorting systematic reviews. Is treating "the scientific literature" as a single thing perhaps a habit worth giving up? As convenient as it would be to be able to just blindly trust something because of where it is published, that model hasn't shown itself to be especially robust in other cases (e.g. the news media). Elsewhere, this is a red flag: > I trust it because of which aggregator aggregated it Should we really make an exception for science? I think that academia is a bit biased towards optimism about publisher-based root-of-trust models because scientific publishing is a relatively unweaponized space. Sure, shenanigans happen, but not at the same scale as elsewhere. The fakers are just trying to get another published paper, they're for the most part not trying to mislead. It's only fake news with a lowercase-f. Sure, let's try to create a medium we can trust, but let's not get our hopes too high about it. That's energy better spent augmenting the ability of a reader or researcher to decide whether to trust a paper based on it's content or based on it having been endorsed or authored by somebody that they explicitly (or transitively) trust.
- burnished 3y agoI disagreed with you until the last paragraph. Lots of things authentically just rely on a high degree of trust and I suspect trying to engineer human systems to be zero trust will make them deeply pathological. But tempering our expectations while working to meaningfully improve on conditions? Aces, all for it.
- __MatrixMan__ 3y agoI agree that zero trust is in most cases a problematic goal. It's really root-of-trust vs web-of-trust that I'm on about here. If peer review is the product then the trust should be peer to peer. It feels like we're treating the publishers themselves as an authority, which I dislike.
- burnished 3y agoThank you for clarifying. The publishers ostensibly occupy a role of stewardship, I suspect the model must have made sense at one point. I admit its hard to see them as much more than rent extractors these days. The nature of trust relationships seems to trend towards aggregation and centralization. Do you have any thoughts on how a web of trust can sustain itself, or is that perhaps not a concern if a centralization appears to reflect a network concensus?
- Strilanc 3y ago> Sabel’s tool relies on just two indicators — authors who use private, noninstitutional email addresses, and [...] Uh huh. I didn't realize until today that all my papers are fake because I give contact information that won't go stale in 3 years, instead of my work email.
- FabHK 3y agoThat’s not what the paper says, I think (even though the badly written article can easily be understood that way).
- Strilanc 3y agoReading the paper it seems like a pretty accurate description. The paper just calls it a "private email" instead of a "non-institutional email". For example (@@@ emphasis is mine): > To identify indicators able to red-flagged fake publications (RFPs), we sent questionnaires to authors. Based on author responses, three indicators were identified: @@@“author's private email”@@@, “international co-author” and “hospital affiliation”. > For Studies 1 to 6 we identified two easy-to-detect indicators, where a publication was labelled as RFP: @@@if an author used a private email@@@ and had no international partner. > Then we combined the two best indicators (@@@“author's private email”@@@ and “hospital affiliation”) to form a classification (tallying) rule: “If both indicators are present, classify as a potential fake, otherwise not” (the “AND” rule) (Katsikopoulos et al., 2020). Fun bonus there with the 2020 book citation for the concept of an AND gate in a classifier.
- juujian 3y agoLove that! I never understood why so many of us would use their affiliation's email address in print if they know that they would only be there for another 2--3 years.
- olddustytrail 3y agoI suspect all your papers are fake, simply because you don't understand the number "two". I would allow just one valid paper with that inability.
- Turukawa 3y agoThe researchers in this paper use an astonishingly biased "fake paper detector", requiring only two conditions to be met for any paper to be considered "fake": 1. Use a non-institutional email address, or have a hospital affiliation, 2. Have no international co-authors. And they acknowledge 86% sensitivity and 44% specificity. It's a coin-toss which biases massively against research from outside the US and Western Europe. This "paper" is bigoted nonsense. https://fediscience.org/@ct_bergstrom/110357278154604907 https://fediscience.org/@ct_bergstrom/110357278154604907
- FabHK 3y agoNo. They use 400 known fakes and 400 matched (presumed) non-fakes to estimate the sensitivity and specificity of their indicator, then apply that indicator to the full universe, then employ the estimated sensitivity and specificity to the obtained measurement to estimate the approximate actual rate of false papers. If you know the true prevalence of a disease in a population, and the sensitivity and specificity of your test, you can predict how many positive measurements you obtain. Vice versa, from the (flawed raw) measurement, given sensitivity and specificity, you can estimate the true prevalence. Furthermore, they’re explicitly saying that “red flagging” by their simple indicator doesn’t mean that the paper is fake, but that it merits higher scrutiny. ETA: I mean, it could still all be bullshit (by virtue of some bias or so), but you’ll need to argue a bit harder to establish that. ETA2: Actually, not sure that’s what they’ve done. They might have just reported the raw (very bad) measurement (that they call “potential red flagged fake paper”), without doing the obvious next step outlined above, and without applying any confidence intervals. So, it might actually be a pretty crap paper (though possibly technically correct) coupled with some mediocre reporting layered on top. Isn’t basic statistics taught anymore?
- newswasboring 3y ago> Furthermore, they’re explicitly saying that “red flagging” by their simple indicator doesn’t mean that the paper is fake, but that it merits higher scrutiny. Then they and science should change their sensationalist headline. It's ironic that a paper about fakeness of something uses a borderline misleading title.
- mtkhaos 3y agoIt would be nice if the Scientific community had the same rigor as a test driven development pipeline. Strange world
- throwoutway 3y agoIm afraid "new tools" aren't going to "tackle" the problem. There are source problem (bad incentives, low integrity, people-pleasing behavior), and second-order tools that amplify that (second-order problems). Adding new tools to 'detect' that don't solve the original problem, they might reduce the second-order problem, but do not touch the source problem. These are band-aids trying to stop a flood of bad science
- placesalt 3y agoI'm not sure what this says about my turn of mind - probably too devious. But I wonder if one tack that fraudsters could follow would be to publish a paper with the named author(s) being legitimate scientists, and then include some citations inside the paper to the fraudster's other papers. You'd need to use some obfuscated correspondence email to complete the loop.
- aurizon 3y agoI am amazed at how well Alexandra Elbakyan has created and promoted sci-hub to fight these journal cash cows, and appalled at the way these journals have tried to block her. They now digitally watermark every journal downloaded at colleges etc, so they can ID the provenance of the journals = she must obfuscate this as best she can. The journals try to punish universities that leak papers to sci-hub Give her a wave.... https://sci-hub.se/alexandra https://sci-hub.se/alexandra
- aurizon 3y agohttps://www.stm-assoc.org/stm-integrity-hub/ https://www.stm-assoc.org/stm-integrity-hub/
- casey2 3y ago>“It will never be a [fully] automated process,” he says. Rather, the tools are like “a spam filter … you still want to go through your spam filter every week” to check for erroneously flagged legitimate content. Even the article makes it clear that this is just a wide net for an automatic first pass. Of course, it is biased towards countries with lax standards.
- oldstrangers 3y agoI wrote one (https://solipsismwow.com/#paper https://solipsismwow.com/#paper). But it's literally fake. Thanks GPT.
- DoreenMichele 3y agoI've read other stuff related to this issue. It seems to me our current system exists in a social reality for which our metrics of authenticity were not designed and it harms both credentialization -- which is recognized as a problem -- and also serious science in ways that are not readily acknowledged as a problem. Mendel, father of genetics, failed to become an accredited teacher. His work on genetics would likely get no recognition in this environment of credentialism is king. Some guy who knows enough about genetics he created his own home pill to deliver genes into his gut to fix his lactose intolerance is being ignored by the world. Someone recently told me on HN that his video sounds like a scam video of a sort that is common (probably in a redacted comment). I have a genetic disorder, which fails to pass the credentialism test. For that and other reasons, I didn't bother to say anything like "Sorry you don't know enough about genetics to follow it." The individual wanted to know where the "studies" and "papers" were. And they likely don't exist and will never exist because there's no profit in it for someone else to try to build on his work. I don't know how we fix this, but the world has changed and it's valuing the facade of scientific work more than actual scientific work and it makes me want to scream.
- elbigbad 3y ago> Some guy who knows enough about genetics he created his own home pill to deliver genes into his gut to fix his lactose intolerance is being ignored by the world. Someone recently told me on HN that his video sounds like a scam video of a sort that is common (probably in a redacted comment). To be honest, I know nothing other than your description and it 100% sounds like either a scam or there are some variables that are not being controlled for. I’m a little shocked that you seem to have fallen for it, unless there is just a lot more to the story…
- DoreenMichele 3y agohttps://www.popularmechanics.com/science/health/a17804128/scientist-painstakingly-documents-his-own-diy-gene-therapy/ https://www.popularmechanics.com/science/health/a17804128/sc... I don't know why you would be "shocked" that I "fell for it." Most of the world thinks I'm a nutter who imagines I'm getting well from my genetic disorder and dismisses my progress as "placebo effect" -- which would give me a mind more powerful than Darth Vader -- or just deluded bullshit. So either I understand genetics and medical stuff better than average, or I'm absolutely the kind of fool who falls for bullshit scams on the internet.
- netzego 3y ago'In an example of Brandolini's law [...] "It took this guy 15 minutes to make his video and it took me three days to fact-check."' [1] [1] https://en.m.wikipedia.org/wiki/Brandolini%27s_law https://en.m.wikipedia.org/wiki/Brandolini%27s_law
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- winstonprivacy 3y agoI found one the other day in the area of finance. The Chinese researchers claimed to have discovered a small tweak to a long established indicator which they described as giving a remarkable increase in r-squared value across a cross section of markets. Sounds great, who wouldn't want to use this? So I implemented and find that their increase was due entirely to applying a log transform of the input variables. The resulting clusters were tighter, but it had zero predictive capability. Very disappointing but in my experience, this is not uncommon.
- belter 3y agoWhy Most Published Research Findings Are False - https://journals.plos.org/plosmedicine/article/file?id=10.1371/journal.pmed.0020124&type=printable https://journals.plos.org/plosmedicine/article/file?id=10.13...
- natural219 3y agoIf people think that 100%-fake papers, with completely made up data and process are bad... wait until people learn how bad 30%-fake papers are, with real cherry-picked data and absurd levels of p-hacking :p
- fuzzfactor 3y agoNot so much like this in natural science.
- Throw73849 3y ago[flagged]
- phyzome 3y agoFlagged for being complete bull pucky.
- galaxyLogic 3y agoA Chatbot could create fake scientific papers, right?
- godelski 3y agoWe must always put this in context, and I think we need to be careful about the narratives. Here's a few rules of thumbs - Realistically the only people who can determine if a work is sound or not are other researchers in that same field. - Peer review is a weak signal: reviewers are good at recognizing bad papers but not good at recognizing good papers (read this carefully). - Most papers aren't highly influential. Thus meaning that we don't rely heavily on the results of most works (we rely weakly or purely for citations). - The more influential a work is the more likely it is to be reproduced and scrutinized. - Benchmarks are benchmarks, nothing more. Benchmarks are weak signals at best and shouldn't be used to make strong conclusions. Be that a p-value, FID, or even likelihood. So we have to keep this in mind for a lot of reasons. One is how we discuss with the public. Headlines like this often make people grow wary of science. While scrutiny is good we have a good history of being successful. All processes are noisy but the cream has is more likely to come to the top and the surface is less noisy. It also tells us about who we should be listening to when taking advice and summaries of works. If you believe the news has failed us, then look to the sources. I see many who only get their science from news sources that claim scientists are corrupt. I found this odd, especially considering I've worked at national labs and I can tell you that no one there is doing it for the money. You'd have to be a fucking idiot to do science for money. It doesn't pay well, you never get real time off, there is a high barrier to entry, and you are under high amounts of pressure. We're on a forum with Silicon Valley wages: the average physicist wage is 100k, what you'd make with a BS in CS but need an advanced degree for working at a lab. Let try to compare likes and likes by looking at LLNL. As a PhD physicist you'll make between $150k and $200/yr. You'll make the same as a PhD computer scientist. Yeah, this seems good, but we need to consider that if you drove 45 minutes west then that would be your base salary and you'd be making the same in other compensations. You can easily verify this and there's plenty of people you can ask for personal experience (I've seen people jump ship often). This doesn't prove that they aren't corrupt, but it provides strong evidence that if these people were motivated by monetary compensations (or even prestige) then there are far better opportunities for them. Another important aspect, which I think is critical to forums like this, is to be careful how you as a non domain expert. Opinions are fine and no one should prevent you from having them. But the confidence in your opinion should be proportional to your qualifications. If you're an expert in one domain I'm sure you're frustrated by how many people discuss your domain as if they knew so much and they get so much wrong. How wrong answers float to the top of forums (HN and Reddit) and the gems are hidden. This usually comes down to a lack of nuanced understanding. Simple answers are almost never correct. Murry Gell-Mann amnesia doesn't just apply to reading the news. Discussions can be had without teaching. Scientific discussions aren't done through debate. Determine your goals, and ask yourself if the way you are discussing allows you to change your opinion or not. Make sure you're on the same page as others, using the same assumptions (this is a key failure point). I'll argue to go in with care. If you don't, you're just adding to the noise.
- ad48 3y agoHi, my name is Adam Day. I was interviewed for this piece in Science. If you are interested to learn more about papermills, I have a popular blog on the subject. https://medium.com/@clearskiesadam https://medium.com/@clearskiesadam Also happy to answer any questions you might have.
- naveen99 3y agoNoise will always be a larger infinity than signal. Karma is the signal.
- aj7 3y agoNow, how do you deal with the false positives?
- pvaldes 3y agoThe requirement to be introduced in the club by a international (ehum... anglo-saxon) partner is very condescending, and scientific colonialism at its best. The burden of the white scientist. People can do science on local problems without being babysat by a foreigner that most of the time will just appear and sign.