4 ms·

The papers that most heavily cite retracted studies

57 points by XzetaU8 2y ago

evolve2k 2y ago
From dev perspective this highlights that lack of embedded version control and handling of big/security fixes for any academic paper. Dependencies that are reported to have bugs reported akin to ‘security flaws’, should lead to updates to papers with dependencies. Naively would be good to sort references into critical references and non-critical references.
- noodlesUK 2y ago
  It would be good if people would universally use any of the extant tools for citation management. It’s not a problem in CS and associated disciplines but in the humanities and social sciences a shockingly large number of researchers don’t use tools like Zotero and Mendeley. They just keep notes and copy/paste entries in a bibliography.
  - blackbrokkoli 2y ago
    Can you expand how that would help with this problem? In my understanding the tools used to write a paper are completely hidden and non-consequential once a paper is published, no?
    - btrettel 2y ago
      I don't know about the others, but Zotero automatically informs me of retractions for articles in my database: https://www.zotero.org/blog/retracted-item-notifications/ https://www.zotero.org/blog/retracted-item-notifications/ So, if a previous article is retracted when I write my own article, I'd probably know.
- blackbrokkoli 2y ago
  I blame rent-seeking academic "publishers" for the lack of these features. There are multiple high-impact features to be built in this regard, many of which would take an experienced dev around an afternoon to build, if a proper data source existed. Instead, the only way to access many papers is as an idiosyncratic PDF, often only including a post-hoc OCR layer, and of course locked behind a 49.99$ 24-hour-access paywall that can also not be automatically accessed, even if you had the required millions laying around.
blackbrokkoli 2y ago
There is a funny kind of bias here: Fields and subfields who actually retract papers are more likely to win the this negative award of "most heavily citing retracted studies" — contrast that psychology, where things are almost never retracted, and there is no way of knowing whether a paper has been replicated consistently except reading hundreds of other papers: no chance of such criticism being leveled.
- BrandoElFollito 2y ago
  This is a very good point, survivor biais in all its glory. I would be interested to see which fields have the most retracted papers, broken by impact index. I am not sure the conclusion to draw from there (most honest vs least competent vs most reproductible vs most scrutinized etc.)
- overstay8930 2y ago
  soft “science” vs hard science
  - blackbrokkoli 2y ago
    Fan favorite of the HN crowd I know, but hardly the correct line to draw. Here is one of the hundreds of papers analyzing mislabeling in genetics (notice how affected papers are usually not withdrawn): https://link.springer.com/article/10.1007/s00204-015-1632-4 https://link.springer.com/article/10.1007/s00204-015-1632-4 (Unless you consider biology soft)
mtts 2y ago
The article also alludes to this, but doesn’t make it specific: It doesn’t necessarily say much if a paper cites a paper that has since been retracted. > Unlike (retracted, 2015) we found … Or > While (retracted, 2021) focused on … the present study concerns … You’d need to do much more in depth analysis to determine to what purpose a retracted study was cited before you can establish whether there is a problem or not.
tsss 2y ago
Unfortunately, by far the worst offenders in the social studies never retract.
- jfengel 2y ago
  The social studies are well known to have a replication problem, but you don't generally retract a paper just because it can't be replicated. Even if a conclusion turns out to be wrong you generally don't retract it. Retraction usually connotes some level of fraud or other serious and deliberate problem. It's a serious black mark on your record, which we don't do merely for being wrong. Science is all about hypothesis revision, and we expect to be continuously wrong. There should perhaps be a designation for "yeah, turns out we were wrong". Instead, you're expected to just find the contradicting papers in your literature search. Saying "I have a paper which proves X" isn't a complete argument unless you've done the full search of relevant papers. That doesn't fix the replication problem in the social sciences, where the subject (groups of people) are too hard to model and control. People do too often cite a single tiny study as if it meant something, and that's just not a valid way to use science. But retraction isn't the solution, not so long as the term connotes malfeasance. Like all sciences they need the room to be honestly wrong, because that's fundamental to science.
  - tpoacher 2y ago
    I seriously challenge this mentality / interpretation of a retraction signalling fraud. This is absurd, and actively harmful. Quite the contrary, a retraction is often a signal of good scientific citizenship and due diligence, which may well come from due diligence of the authors themselves, not just external validators. NOT retracting a paper out of fear of being labelled a fraud and ostracised from the scientific community is a massive, unhealthy disincentive which we DEFINITELY do not need in the sciences. The article is VERY careful not to paint retracted papers with such ludicrous stigma; the focus is very appropriately on authors who add "inherited" citations without the necessary due diligence (i.e. actually attempting to read the paper in question, and finding out it has been retracted, for whatever reason)
  - inglor_cz 2y ago
2-3-7-43-1807 2y ago
papers citing retracted studies should be auto-flagged for peer-re-review.
- curious_cat_163 2y ago
  Great idea! The challenges one can imagine with that are: 1) human reviews are expensive 2) the citation might not have anything to do with the reason why the cited paper was retracted. Now, if it all could be automated (say with some emerging technology that can grok the context of the citation and say classify it into flag/no-flag buckets say), then we are cooking with some real fire. :-)
deleted 2y ago
[deleted]

57 points by XzetaU8 2y ago

From dev perspective this highlights that lack of embedded version control and handling of big/security fixes for any academic paper. Dependencies that are reported to have bugs reported akin to ‘security flaws’, should lead to updates to papers with dependencies. Naively would be good to sort references into critical references and non-critical references.

noodlesUK 2y ago
It would be good if people would universally use any of the extant tools for citation management. It’s not a problem in CS and associated disciplines but in the humanities and social sciences a shockingly large number of researchers don’t use tools like Zotero and Mendeley. They just keep notes and copy/paste entries in a bibliography.
- blackbrokkoli 2y ago
  Can you expand how that would help with this problem? In my understanding the tools used to write a paper are completely hidden and non-consequential once a paper is published, no?
  - btrettel 2y ago
    I don't know about the others, but Zotero automatically informs me of retractions for articles in my database: https://www.zotero.org/blog/retracted-item-notifications/ https://www.zotero.org/blog/retracted-item-notifications/ So, if a previous article is retracted when I write my own article, I'd probably know.
blackbrokkoli 2y ago
I blame rent-seeking academic "publishers" for the lack of these features. There are multiple high-impact features to be built in this regard, many of which would take an experienced dev around an afternoon to build, if a proper data source existed. Instead, the only way to access many papers is as an idiosyncratic PDF, often only including a post-hoc OCR layer, and of course locked behind a 49.99$ 24-hour-access paywall that can also not be automatically accessed, even if you had the required millions laying around.

blackbrokkoli 2y ago

There is a funny kind of bias here: Fields and subfields who actually retract papers are more likely to win the this negative award of "most heavily citing retracted studies" — contrast that psychology, where things are almost never retracted, and there is no way of knowing whether a paper has been replicated consistently except reading hundreds of other papers: no chance of such criticism being leveled.

BrandoElFollito 2y ago
This is a very good point, survivor biais in all its glory. I would be interested to see which fields have the most retracted papers, broken by impact index. I am not sure the conclusion to draw from there (most honest vs least competent vs most reproductible vs most scrutinized etc.)
overstay8930 2y ago
soft “science” vs hard science
- blackbrokkoli 2y ago
  Fan favorite of the HN crowd I know, but hardly the correct line to draw. Here is one of the hundreds of papers analyzing mislabeling in genetics (notice how affected papers are usually not withdrawn): https://link.springer.com/article/10.1007/s00204-015-1632-4 https://link.springer.com/article/10.1007/s00204-015-1632-4 (Unless you consider biology soft)

mtts 2y ago

The article also alludes to this, but doesn’t make it specific: It doesn’t necessarily say much if a paper cites a paper that has since been retracted. > Unlike (retracted, 2015) we found … Or > While (retracted, 2021) focused on … the present study concerns … You’d need to do much more in depth analysis to determine to what purpose a retracted study was cited before you can establish whether there is a problem or not.

tsss 2y ago

Unfortunately, by far the worst offenders in the social studies never retract.

jfengel 2y ago
The social studies are well known to have a replication problem, but you don't generally retract a paper just because it can't be replicated. Even if a conclusion turns out to be wrong you generally don't retract it. Retraction usually connotes some level of fraud or other serious and deliberate problem. It's a serious black mark on your record, which we don't do merely for being wrong. Science is all about hypothesis revision, and we expect to be continuously wrong. There should perhaps be a designation for "yeah, turns out we were wrong". Instead, you're expected to just find the contradicting papers in your literature search. Saying "I have a paper which proves X" isn't a complete argument unless you've done the full search of relevant papers. That doesn't fix the replication problem in the social sciences, where the subject (groups of people) are too hard to model and control. People do too often cite a single tiny study as if it meant something, and that's just not a valid way to use science. But retraction isn't the solution, not so long as the term connotes malfeasance. Like all sciences they need the room to be honestly wrong, because that's fundamental to science.
- tpoacher 2y ago
  I seriously challenge this mentality / interpretation of a retraction signalling fraud. This is absurd, and actively harmful. Quite the contrary, a retraction is often a signal of good scientific citizenship and due diligence, which may well come from due diligence of the authors themselves, not just external validators. NOT retracting a paper out of fear of being labelled a fraud and ostracised from the scientific community is a massive, unhealthy disincentive which we DEFINITELY do not need in the sciences. The article is VERY careful not to paint retracted papers with such ludicrous stigma; the focus is very appropriately on authors who add "inherited" citations without the necessary due diligence (i.e. actually attempting to read the paper in question, and finding out it has been retracted, for whatever reason)
- inglor_cz 2y ago

2-3-7-43-1807 2y ago

papers citing retracted studies should be auto-flagged for peer-re-review.

curious_cat_163 2y ago
Great idea! The challenges one can imagine with that are: 1) human reviews are expensive 2) the citation might not have anything to do with the reason why the cited paper was retracted. Now, if it all could be automated (say with some emerging technology that can grok the context of the citation and say classify it into flag/no-flag buckets say), then we are cooking with some real fire. :-)

deleted 2y ago

[deleted]