5 ms·

Refuted papers continue to be cited more than their failed replications

79 points by nabla9 3y ago

gtmitchell 3y ago
This is a systemic social problem, not a technology problem. If researchers wanted to double check every single one of their citations for subsequent retractions, they would simply have their graduate students do it. The root of the problem is that it’s impossible to build a scientific career out of publishing negative results (assuming you can even publish them at all). What actually matters is that your original publication gets into a high tier journal and gets good press. Once that’s done, no one cares if what you published was garbage or not, because by the time anyone figures that out (if they ever do), you’ll have tenure, big fat multi-year grants, and will be out on the speaking circuit.
- lupire 3y ago
  It is absolutely unconscionable that "peer review" process doesn't include "review". Replicating results should be a hard requirement for publishing a conclusion. Anything short is merely a hypothesis with some suggestive evidence, a conjecture. Imagine if mathematicians cited published conjectures but called the proven results.
  - sseagull 3y ago
    This is a common refrain, but if this was a requirement science would absolutely grind to a halt. There is just too much science being done. And in some fields, what scientists publish is generally sound, if incremental (and maybe of dubious importance). I think the tradeoff we have taken is "lots of science, mostly good" vs "little science, 100% correct". Similar tradeoff is taken with software & wikipedia, for example. Or do you apply the same rigorous standards to, say, software? Should every piece of software that is publicly distributed be thoroughly reviewed for correctness?
    - Voultapher 3y ago
      Hard disagree, set aside 30% of the budget from the get go by law and use that to let others you have no contact with try to replicate your wirk. I'm willing to bet, that if this were to be tried earnestly 99% of research would be compatible.
      - sseagull 3y ago
        You are imagining a solution to a very hard social problem. It is trivially simple on paper, but realistically extremely difficult to implement when you think it through. The fundamental issue: what is in it for the reviewers in this case? Scientists are people, and you can't really legislate interest in replicating other people's work that is unrelated to your own, unless you really spend money on it. And there's some bad news on that front: https://www.science.org/content/article/final-u-s-spending-bills-offer-gloomy-outlook-science https://www.science.org/content/article/final-u-s-spending-b...
        Voultapher 3y ago
        I'm there with you, it's a culture and social issue. That said the argument it's not fun seems quite weak to me. Scientists have to do source citation, formatting, peer review and many other things that aren't intrinsically done for selfish hedonistic reasons.
    - AndrewKemendo 3y ago
      It probably should grind to a halt because the incentives for “research” are skewed to commerce to such a degree that the structure of the investigatory mechanism is foundationally flawed
      - sseagull 3y ago
        Easy to say, but good luck with that. It's flawed for sure, but there's so much around you that comes from this system if you care to look for it.
- whiterknight 3y ago
  It also demonstrates that the popularity of a paper is in large part marketing. The original papers are simply better marketed than the corrections.
- __MatrixMan__ 3y ago
  Paper is a technology. This systemic social problem around paper has the quirks that it does because paper has limitations (you can't add highlights and margin notes after you've given it to somebody else). If we stop replicating those limitations, it's not unreasonable to expect that the social problem will change also. I don't think the answer is search engines in the traditional sense, but we probably do need something that knows how to search. If whatever we use to view research were to display warnings wherever a retracted citation was viewed (or whenever one the citations' citations were retracted...), similarly to how we display SSL certificate expiry warnings in a browser, I think that would create a dampening effect on the momentum that a retracted paper can have. "Retracted" probably isn't the only color we'd want here. "Replicated" might be another. My point is that summoning up-to-date metadata on a something published last year should just be the default view mode, not something you have to ask a grad student to go do. Knowledge graphs, not dead trees.
  - AndrewKemendo 3y ago
    >If we stop replicating those limitations, it's not unreasonable to expect that the social problem will change also. This is called technological determinism and is not an accurate heuristic for how humanity adopts tools Humans generally adopt tools broadly after the social environment allows it, not when the tool is capable. This has been shown over and over. Never has there existed a tool whose introduction was immediately and universally adopted. There’s almost always a period of introduction, then decades of middling adoption and refinement, then the social climate changes and production adjusts enough to adopt en masse. This was true for every major invention and is an artifact of human social structures
    - __MatrixMan__ 3y ago
      I don't see how these are incompatible views. Maybe it takes a decade, maybe it takes a century, maybe it doesn't happen at all... building the alternative remains the first step. Certainly no adoption is going to happen before the medium exists. Also we wouldn't be publishing articles like this one if there wasn't already some desire to change things.
AndrewKemendo 3y ago
At 40 years old, what’s clear to me is that while people have some curiosities they pursue, most have no desire to be rigorous about epistemological questions surrounding their curiosities. This extends well into people’s professional disciplines in my experience. So even for the small percentage of people who have some narrow focus of inquiry (usually at work) they only have the desire or ability to evaluate the first or second order inputs and effects. In general this isn’t a problem for day to day life. The packaged chicken on the shelf fits perfectly into recipe, and there is little to no thought of any other externalities (what systems am I “voting with my dollars” to support? What is the relative difference in supply chain between two options) However if you are going to try and align your conceptions of how systems/environments work and what you desire for future state, then you do have to do the system decomposition and that’s actually non trivial. So there’s no real “market” for epistemological rigor outside of a small group of people who come across as unreasonable in their demand for rigor, so this is the likely result of a system that is built for personal or organizational success and not an ego-free search for “truth” via intentional rigor. Luckily the small group continues to grow as a total number
- pessimizer 3y ago
  It takes too much work to verify everything in every field personally, you wouldn't be able to move at all. The problem is a complete lack of interest in any tools that would inform people about this, because entire industries and fields are built by bad papers. We seem to have a lot of hypertext systems that keep track of citations. Seems like it would be cheap at this point to put a black mark on papers that don't replicate, and to propagate that black mark to every paper that references the bad paper without also referencing the failed replication. Put a big red X over the abstracts.
  - smallmancontrov 3y ago
    > It takes too much work to verify everything in every field personally, you wouldn't be able to move at all. Does it? The rate of paper production is so high that it has become a logistical problem in and of itself and wild goose chases are common. Surely some of that energy could be profitably redirected into replication and consolidation. The black mark system could work but seems like a recipe for some nasty politics. I tend to think that baking replication into the culture might be a better approach -- science advanced beyond secret-hoarding when information sharing became institutionally valued, and I believe that it could advance beyond p-hacking if replication became institutionally valued. Fund & cite replication efforts and the rest will follow.
parpfish 3y ago
Im not sure why somebody would cite a failed replication unless they’re doing a lit review or meta analysis. Most would just stop citing the refuted paper to save word/citation counts. so rather than compare total number of citations, you want to see that: - citations per year drop once refuted - if citations continue after refutation, they co-occur with the refutation study
- Staple_Diet 3y ago
  Yes, this is a good point. Tracking citations of the original work in the absence of failed replications would be a more informative metric to measure whether a refuted study was still being used to justify arguments.
Staple_Diet 3y ago
This is an issue with using citations has the measure of everything. If I am writing a paper and mention a particular study that failed to replicate in subsequent attempts, I still cite the original paper. It is understandable that a seminal or surprising study gets cited well after it is refuted. Take for example Jaeggi's studies that claimed Working Memory training increases IQ...a claim shown to not replicate in countless follow-ups. I myself have cited a number of her papers, right before I also cite the contradictory evidence. I would expect similar patterns in other papers. A citation isn't an endorsement of the findings, but rather an acknowledgement that a piece of work is worth mentioning, whether it be in a good, or bad, light. An alternative to GoogleScholar is Research Rabbit, an AI enabled platform that maps the networks between paper allowing for an author to see who cited the paper they intend to cite, who did the original paper cite etc etc.
matanyall 3y ago
This is a huge issue. What's the point of doing replication studies (which we need) if folks just don't care about the result. It's heavy handed, but I think every time a study fails to reproduce its results, every single published paper that uses that refuted study is notified, the authors get 6 months with a warning banner on top of the article, and if a correction is not submitted, it's automatically removed. Science isn't supposed to be easy. We need a system that prioritizes high quality output, not volume.
j7ake 3y ago
Refuted paper does not mean retracted, and therefore won't stop being cited.
- snitty 3y ago
  Furthermore, it doesn't appear that they're looking at whether the reason the paper was cited was the subject of the failed replication. To analogize to the legal field, which they mention in the article, if a case I cite is good law on one point of law, but no longer good law on another I may cite it as _Plaintiff v. Defendant_, 123 F.3d 456 (1998) (reversed on other grounds, _Defendant v. Plaintiff_ 78 S.Ct. 910 (2000)) Just because one finding in a paper wasn't replicated doesn't mean that the paper can't or shouldn't be cited for other findings.
deleted 3y ago
[deleted]
layer8 3y ago
Maybe publication websites should be required to add a mark “Refuted by: <other paper>” in the PDF, similar to how RFCs have “Superseded by” and “Updated by”.
- Staple_Diet 3y ago
  There's no incentive for publishers to do that, but a good scientist will explore an area before embarking on it. This is where meta-analyses and citation mapping tools come in handy.
singularity2001 3y ago
I LOVE the new twitter functionality where others put "bad information" into context. Almost as good as the HN 'flag' button. Something like this should become standard everywhere.
- tsol 3y ago
  The community notes? It works surprisingly well over all, although it does get gamed sometimes. Notably, one criticism is that Elon himself gets a lot of community note suggestions but they don't post.

79 points by nabla9 3y ago

This is a systemic social problem, not a technology problem. If researchers wanted to double check every single one of their citations for subsequent retractions, they would simply have their graduate students do it. The root of the problem is that it’s impossible to build a scientific career out of publishing negative results (assuming you can even publish them at all). What actually matters is that your original publication gets into a high tier journal and gets good press. Once that’s done, no one cares if what you published was garbage or not, because by the time anyone figures that out (if they ever do), you’ll have tenure, big fat multi-year grants, and will be out on the speaking circuit.

lupire 3y ago
It is absolutely unconscionable that "peer review" process doesn't include "review". Replicating results should be a hard requirement for publishing a conclusion. Anything short is merely a hypothesis with some suggestive evidence, a conjecture. Imagine if mathematicians cited published conjectures but called the proven results.
- sseagull 3y ago
  This is a common refrain, but if this was a requirement science would absolutely grind to a halt. There is just too much science being done. And in some fields, what scientists publish is generally sound, if incremental (and maybe of dubious importance). I think the tradeoff we have taken is "lots of science, mostly good" vs "little science, 100% correct". Similar tradeoff is taken with software & wikipedia, for example. Or do you apply the same rigorous standards to, say, software? Should every piece of software that is publicly distributed be thoroughly reviewed for correctness?
  - Voultapher 3y ago
    Hard disagree, set aside 30% of the budget from the get go by law and use that to let others you have no contact with try to replicate your wirk. I'm willing to bet, that if this were to be tried earnestly 99% of research would be compatible.
    - sseagull 3y ago
      You are imagining a solution to a very hard social problem. It is trivially simple on paper, but realistically extremely difficult to implement when you think it through. The fundamental issue: what is in it for the reviewers in this case? Scientists are people, and you can't really legislate interest in replicating other people's work that is unrelated to your own, unless you really spend money on it. And there's some bad news on that front: https://www.science.org/content/article/final-u-s-spending-bills-offer-gloomy-outlook-science https://www.science.org/content/article/final-u-s-spending-b...
      - Voultapher 3y ago
        I'm there with you, it's a culture and social issue. That said the argument it's not fun seems quite weak to me. Scientists have to do source citation, formatting, peer review and many other things that aren't intrinsically done for selfish hedonistic reasons.
  - AndrewKemendo 3y ago
    It probably should grind to a halt because the incentives for “research” are skewed to commerce to such a degree that the structure of the investigatory mechanism is foundationally flawed
    - sseagull 3y ago
      Easy to say, but good luck with that. It's flawed for sure, but there's so much around you that comes from this system if you care to look for it.
whiterknight 3y ago
It also demonstrates that the popularity of a paper is in large part marketing. The original papers are simply better marketed than the corrections.
__MatrixMan__ 3y ago
Paper is a technology. This systemic social problem around paper has the quirks that it does because paper has limitations (you can't add highlights and margin notes after you've given it to somebody else). If we stop replicating those limitations, it's not unreasonable to expect that the social problem will change also. I don't think the answer is search engines in the traditional sense, but we probably do need something that knows how to search. If whatever we use to view research were to display warnings wherever a retracted citation was viewed (or whenever one the citations' citations were retracted...), similarly to how we display SSL certificate expiry warnings in a browser, I think that would create a dampening effect on the momentum that a retracted paper can have. "Retracted" probably isn't the only color we'd want here. "Replicated" might be another. My point is that summoning up-to-date metadata on a something published last year should just be the default view mode, not something you have to ask a grad student to go do. Knowledge graphs, not dead trees.
- AndrewKemendo 3y ago
  >If we stop replicating those limitations, it's not unreasonable to expect that the social problem will change also. This is called technological determinism and is not an accurate heuristic for how humanity adopts tools Humans generally adopt tools broadly after the social environment allows it, not when the tool is capable. This has been shown over and over. Never has there existed a tool whose introduction was immediately and universally adopted. There’s almost always a period of introduction, then decades of middling adoption and refinement, then the social climate changes and production adjusts enough to adopt en masse. This was true for every major invention and is an artifact of human social structures
  - __MatrixMan__ 3y ago
    I don't see how these are incompatible views. Maybe it takes a decade, maybe it takes a century, maybe it doesn't happen at all... building the alternative remains the first step. Certainly no adoption is going to happen before the medium exists. Also we wouldn't be publishing articles like this one if there wasn't already some desire to change things.

AndrewKemendo 3y ago

At 40 years old, what’s clear to me is that while people have some curiosities they pursue, most have no desire to be rigorous about epistemological questions surrounding their curiosities. This extends well into people’s professional disciplines in my experience. So even for the small percentage of people who have some narrow focus of inquiry (usually at work) they only have the desire or ability to evaluate the first or second order inputs and effects. In general this isn’t a problem for day to day life. The packaged chicken on the shelf fits perfectly into recipe, and there is little to no thought of any other externalities (what systems am I “voting with my dollars” to support? What is the relative difference in supply chain between two options) However if you are going to try and align your conceptions of how systems/environments work and what you desire for future state, then you do have to do the system decomposition and that’s actually non trivial. So there’s no real “market” for epistemological rigor outside of a small group of people who come across as unreasonable in their demand for rigor, so this is the likely result of a system that is built for personal or organizational success and not an ego-free search for “truth” via intentional rigor. Luckily the small group continues to grow as a total number

pessimizer 3y ago
It takes too much work to verify everything in every field personally, you wouldn't be able to move at all. The problem is a complete lack of interest in any tools that would inform people about this, because entire industries and fields are built by bad papers. We seem to have a lot of hypertext systems that keep track of citations. Seems like it would be cheap at this point to put a black mark on papers that don't replicate, and to propagate that black mark to every paper that references the bad paper without also referencing the failed replication. Put a big red X over the abstracts.
- smallmancontrov 3y ago
  > It takes too much work to verify everything in every field personally, you wouldn't be able to move at all. Does it? The rate of paper production is so high that it has become a logistical problem in and of itself and wild goose chases are common. Surely some of that energy could be profitably redirected into replication and consolidation. The black mark system could work but seems like a recipe for some nasty politics. I tend to think that baking replication into the culture might be a better approach -- science advanced beyond secret-hoarding when information sharing became institutionally valued, and I believe that it could advance beyond p-hacking if replication became institutionally valued. Fund & cite replication efforts and the rest will follow.

parpfish 3y ago

Im not sure why somebody would cite a failed replication unless they’re doing a lit review or meta analysis. Most would just stop citing the refuted paper to save word/citation counts. so rather than compare total number of citations, you want to see that: - citations per year drop once refuted - if citations continue after refutation, they co-occur with the refutation study

Staple_Diet 3y ago
Yes, this is a good point. Tracking citations of the original work in the absence of failed replications would be a more informative metric to measure whether a refuted study was still being used to justify arguments.

Staple_Diet 3y ago

This is an issue with using citations has the measure of everything. If I am writing a paper and mention a particular study that failed to replicate in subsequent attempts, I still cite the original paper. It is understandable that a seminal or surprising study gets cited well after it is refuted. Take for example Jaeggi's studies that claimed Working Memory training increases IQ...a claim shown to not replicate in countless follow-ups. I myself have cited a number of her papers, right before I also cite the contradictory evidence. I would expect similar patterns in other papers. A citation isn't an endorsement of the findings, but rather an acknowledgement that a piece of work is worth mentioning, whether it be in a good, or bad, light. An alternative to GoogleScholar is Research Rabbit, an AI enabled platform that maps the networks between paper allowing for an author to see who cited the paper they intend to cite, who did the original paper cite etc etc.

matanyall 3y ago

This is a huge issue. What's the point of doing replication studies (which we need) if folks just don't care about the result. It's heavy handed, but I think every time a study fails to reproduce its results, every single published paper that uses that refuted study is notified, the authors get 6 months with a warning banner on top of the article, and if a correction is not submitted, it's automatically removed. Science isn't supposed to be easy. We need a system that prioritizes high quality output, not volume.

j7ake 3y ago

Refuted paper does not mean retracted, and therefore won't stop being cited.

snitty 3y ago
Furthermore, it doesn't appear that they're looking at whether the reason the paper was cited was the subject of the failed replication. To analogize to the legal field, which they mention in the article, if a case I cite is good law on one point of law, but no longer good law on another I may cite it as _Plaintiff v. Defendant_, 123 F.3d 456 (1998) (reversed on other grounds, _Defendant v. Plaintiff_ 78 S.Ct. 910 (2000)) Just because one finding in a paper wasn't replicated doesn't mean that the paper can't or shouldn't be cited for other findings.

deleted 3y ago

[deleted]

layer8 3y ago

Maybe publication websites should be required to add a mark “Refuted by: <other paper>” in the PDF, similar to how RFCs have “Superseded by” and “Updated by”.

Staple_Diet 3y ago
There's no incentive for publishers to do that, but a good scientist will explore an area before embarking on it. This is where meta-analyses and citation mapping tools come in handy.

singularity2001 3y ago

I LOVE the new twitter functionality where others put "bad information" into context. Almost as good as the HN 'flag' button. Something like this should become standard everywhere.

tsol 3y ago
The community notes? It works surprisingly well over all, although it does get gamed sometimes. Notably, one criticism is that Elon himself gets a lot of community note suggestions but they don't post.