4 ms·

The metrics used in this paper are... deeply flawed, to the point that the authors admit that they label nearly half of known good papers in a curated sample as

by rst 3y ago

The metrics used in this paper are... deeply flawed, to the point that the authors admit that they label nearly half of known good papers in a curated sample as "fake" -- and particularly likely to generate false positives for researchers whose institutions don't, say, run their own email systems (as is common in large chunks of the world). Here's a rundown of the flaws from an epidemiologist with a sideline in scientific communication: https://fediscience.org/@ct_bergstrom/110357259338364341 https://fediscience.org/@ct_bergstrom/110357259338364341

Eddy_Viscosity2 3y ago
Ironically, this would mean that this paper is "fake".
pessimizer 3y ago
Looks like the "misinformation" industry is branching out.
largepeepee 3y ago
You know what's funny? Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Kinda similar to those researchers years back who proved how easy it was to go into certain social science journals as long as you copied their ideology.
- boomboomsubban 3y ago
  >Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Not by the definition of "fake" used in the article, as the data wouldn't be plagiarized or fabricated. It'd just be shitty data.
- newswasboring 3y ago
  It's a medRxive preprint. It didn't get published anywhere. Science (the magazine) has lowered it's standards.
- cauch 3y ago
  Well, there is a difference between "fake science" and "tried to do correct science but ending being wrong". If the second is "fake science", then basically all that Newton has ever produced is "fake science". For the social science journals bit, are you thinking of the "grievance studies affair": https://en.wikipedia.org/wiki/Grievance_studies_affair https://en.wikipedia.org/wiki/Grievance_studies_affair ? Ironically, this study has generated a lot of "fake news" on the field of social science. The conclusions of this study were widely spread mainly by people for ideological reason. When we look at the study in question, it's clear the conclusions are quite different than what the rumors say. For example, the same researchers tried such hoax before the ones they mention in their study, except that these hoaxes failed to be published, and they "forgot" to mention it. They did not have any control group, neither as "correct article" or "article defending the opposite ideology" (so, how can we conclude that the reason these bad articles were published were because of ideology if you don't know how many articles are published without being critically reviewed). They also count as valid a lot of journals that are pay-to-publish and not seriously used in the field. One of the author, ironically, ended up supporting platforms publishing conspiracy theories (and he was even banned from Twitter) (not that the study should be judged based on that, but it's a funny anecdote: the author who, according to some, had the courage to defend real science against bad woke ideology, who ends up demonstrating that he never cared about real science and is driven by ideology not science)
  - kevviiinn 3y ago
    A reviewer should have seen that massive red flag
  - caddemon 3y ago
    There's also a difference between outright fake science i.e. lies/fabricated data in the manuscript and bad science i.e. the conclusions drawn by the authors were always "fake" because of bad practices but if you look at the details of the work they are honest about what they did. Of course ideally you would minimize both types of bad paper, but the latter isn't too damaging to the system in isolation while the former can cause a handful of papers to mislead a subfield of science for years. Also how to screen for and how to systemically discourage these two things could be quite different.

by rst 3y ago

Eddy_Viscosity2 3y ago

Ironically, this would mean that this paper is "fake".

pessimizer 3y ago

Looks like the "misinformation" industry is branching out.

largepeepee 3y ago

You know what's funny? Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Kinda similar to those researchers years back who proved how easy it was to go into certain social science journals as long as you copied their ideology.

boomboomsubban 3y ago
>Even if the numbers are hot garbage, they proved the point about how easy it is to publish fake science papers, since it got published. Not by the definition of "fake" used in the article, as the data wouldn't be plagiarized or fabricated. It'd just be shitty data.
newswasboring 3y ago
It's a medRxive preprint. It didn't get published anywhere. Science (the magazine) has lowered it's standards.
cauch 3y ago
Well, there is a difference between "fake science" and "tried to do correct science but ending being wrong". If the second is "fake science", then basically all that Newton has ever produced is "fake science". For the social science journals bit, are you thinking of the "grievance studies affair": https://en.wikipedia.org/wiki/Grievance_studies_affair https://en.wikipedia.org/wiki/Grievance_studies_affair ? Ironically, this study has generated a lot of "fake news" on the field of social science. The conclusions of this study were widely spread mainly by people for ideological reason. When we look at the study in question, it's clear the conclusions are quite different than what the rumors say. For example, the same researchers tried such hoax before the ones they mention in their study, except that these hoaxes failed to be published, and they "forgot" to mention it. They did not have any control group, neither as "correct article" or "article defending the opposite ideology" (so, how can we conclude that the reason these bad articles were published were because of ideology if you don't know how many articles are published without being critically reviewed). They also count as valid a lot of journals that are pay-to-publish and not seriously used in the field. One of the author, ironically, ended up supporting platforms publishing conspiracy theories (and he was even banned from Twitter) (not that the study should be judged based on that, but it's a funny anecdote: the author who, according to some, had the courage to defend real science against bad woke ideology, who ends up demonstrating that he never cared about real science and is driven by ideology not science)
- kevviiinn 3y ago
  A reviewer should have seen that massive red flag
- caddemon 3y ago
  There's also a difference between outright fake science i.e. lies/fabricated data in the manuscript and bad science i.e. the conclusions drawn by the authors were always "fake" because of bad practices but if you look at the details of the work they are honest about what they did. Of course ideally you would minimize both types of bad paper, but the latter isn't too damaging to the system in isolation while the former can cause a handful of papers to mislead a subfield of science for years. Also how to screen for and how to systemically discourage these two things could be quite different.