4 ms·

Those correction notices, in full

128 points by Tomte 2y ago

scubbo 2y ago
> This entry was posted in [...] Zombies I feel like I'm missing a joke or reference somewhere - can anyone help me out? I'm familiar with the concept of p-zombies, but that doesn't seem to be relevant here - and a "Zombie Statistic"[0] is sorta-related in that it refers to the psychological power of incorrect assertions, but I'm still struggling to see the link. [0] https://absolutelymaybe.plos.org/2019/11/30/the-power-of-zombie-statistics-systematic-review-edition/ https://absolutelymaybe.plos.org/2019/11/30/the-power-of-zom...
more_corn 2y ago
This is fantastic. Scientific advancement requires graceful, honest and straightforward acknowledgement of mistakes.
baggy_trough 2y ago
The cover up is usually worse than the crime.
- gcr 2y ago
  Do you think these errata being published are a good thing?
  - baggy_trough 2y ago
    Yes of course. That's the point of my comment, which is being downvoted for some reason that eludes me.
    - allannienhuis 2y ago
      Perhaps its because your comment seemed to equate errors with crimes, or at least malicious intent. The language seems a bit provocative for many, detracting from whatever message was intended.
      - gcr 2y ago
        Yeah. That's how I interpreted the comment, and why I responded apprehensively.
    - brudgers 2y ago
      Sometimes I write with "downvotes be damned" in mind and connected with the audience exactly how I intended. But not connecting with the audience is the usual reason. When it happens to me, I take it as feedback on my writing. Maybe I was unclear. Maybe I was wrong. Maybe it was written for a different audience. In those cases, I just try to improve my writing. Anyway, where can responses to your original comment go? They could dispute your maxim and the internet gets another argument where nobody changes their mind. Or they could agree with it and the internet gets another dog pile of cynicisms. Generally, those are not why people come to HN...at least when the form is one-liners. Finally, complaining about downvotes is contrary to the HN guidelines. Good luck.
      - dredmorbius 2y ago
        Excellent, helpful, and appropriate to the context BTW.
aorloff 2y ago
[flagged]
- airstrike 2y ago
  FWIW, I don't think this is really a Trump era exclusive. People have always believed whatever they wanted to believe, facts and accuracy be damned. IMHO understanding this as an inexorable human condition makes it much easier to understand the world. What actually matters is people can be persuaded to _want_ to believe different things, so the only real leverage is in shaping those wants—not in being right.
  - wrs 2y ago
    Also, if you can get people to connect intensely with just one or two of your statements, you can then make other false statements and they won’t care, because that might invalidate the one they really want to believe. The threshold is so low that the shotgun approach of just telling lies continually actually works quite well. Your statements don’t even have to be consistent, so you can A/B test the lies. The normal backpressure to this is that you lose face with your peers because you become known as a liar. But if your peers don’t influence your success, or you just have no peers, it works.
    - toomuchtodo 2y ago
      https://news.ycombinator.com/item?id=42075533 https://news.ycombinator.com/item?id=42075533 ("HN: Misinformation Does Spread Like a Virus, Epidemiology Shows") https://theconversation.com/misinformation-really-does-spread-like-a-virus-suggest-mathematical-models-drawn-from-epidemiology-242679 https://theconversation.com/misinformation-really-does-sprea... ("The Conversation: Misinformation really does spread like a virus, suggest mathematical models drawn from epidemiology")
- aprilthird2021 2y ago
  This was a prominent mark of the Bush era too (and I'm sure many other Democratic/Republican administrations, it's just my example). I was rewatching Fahrenheit 9/11 because of our government's current foreign engagements, and it's striking how even then people were duped, but back then it was by lack of information, now it is by overinformation
gcr 2y ago
[flagged]
- Throw39387 2y ago
  [flagged]
  - pessimizer 2y ago
    [flagged]
- rachofsunshine 2y ago
  The paper [1] is linked in the OP. Section 3, the section invalidated by the errata, looks at data on voter preferences. So the answer to your question is "a lot", assuming that you think that "using data on the beliefs of voters" is in some way important to a paper on electoral strategy. [1] http://www.stat.columbia.edu/~gelman/research/published/AOAS150.pdf http://www.stat.columbia.edu/~gelman/research/published/AOAS...
vasco 2y ago
I like that they are very blunt and to the point. It'd be really cool if we can add more social dynamic features to papers / research, and have people correct papers in the open, like twitter community notes, to the point that if you can really disprove it, the paper is now marked "wrong" or something like that on arxiv. Sort of inbetween collaborative science or hunting for patent prior art or bug bounty hunting. Getting paid for it would be even better, making money by sniping issues in research. One can dream. I wonder how much money in bounties it'd take to proof-invalidate whole branches of psychology (insert your own pet peeve here) research in a methodical way, probably not that much.
- srockets 2y ago
  I too, would like to have what you’re having. Academic publishing doesn’t make the authors money, it costs them money. To publish is a requirement for academic employment, but there’s no incentive to retract, either than revenge, or one’s academic honesty.
  - vasco 2y ago
    > Academic publishing doesn’t make the authors money > there’s no incentive to retract (... other than) or one’s academic honesty We agree on the problems, maybe you have other wacky potential solutions other than just saying others wouldn't work!
- amoshebb 2y ago
  I wish academic papers were more like Wikipedia articles. Currently what I'm working on is really "Building on" one pretty pivotal paper from the 90s, and there's a whole constellation of work that has spawned. So much ink is spilled re-defining the problem, and reading any paper requires going through the system model every time because tons of arbitrary decisions may have been done different. It makes it hard to compare results, and makes almost every statement that reads "Over in this area we're not innovating on, we used the SOTA" wrong, because some other group is innovating in that corner. If instead there was one canonical version of it with an edit history, and I could go try to just re-write one little para and argue in the talk section about it with the one-or-two other groups picking away at that, I feel like things could move faster and be done higher quality. It'd also be a lot easier to peek at other areas. Currently if I have a question like "What's the latest in NeRFs underwater? I remember seeing a paper about that a while ago" I've basically got no idea.
Waterluvian 2y ago
> 1999: It is not clear to us in general how to avoid this sort of false proof, the problem being that the false statement seemed so natural to us that we did not think to look at it carefully. Assuming I understand correctly, this is basically the common issue of being unable to be objective when you’ve lived and breathed a subject matter for long enough. The answer is rigorous peer review, I think.
- srockets 2y ago
  Peer reviews are (often) also done by people who live and breathe the subject matter.
timkam 2y ago
Without knowing the specific context: I think this really is a good example of how errors should be disclosed. We need to acknowledge that scientists/academics are human; even very competent mathematicians make mistakes and some of these mistakes appear in published papers. What we lack in many fields is a culture and process that allows (and ideally, encourages) one to disclose: "this was wrong, here is how I fixed it, or how it's actually correct". E.g., in the communities I know in Computer Science & AI, I rarely even see errata lists on personal webpages, not to speak of journals that provide a straightforward process for updates. I would even go so far to claim that the current culture, in which honest errors cannot be straightforwardly corrected, plays into the hands of the clearly dishonest "bad apples". Science is, obviously, not a "monotonic" process in which every single paper adds to the truth; this is practically not even the case for mathematics, which is at least monotonic on object-level (but mistakes happen all the time). As a prominent example, consider this impressive list of Feynman errata: https://www.feynmanlectures.caltech.edu/info/flp_errata.html https://www.feynmanlectures.caltech.edu/info/flp_errata.html.
einpoklum 2y ago
> "Should the Democrats move to the left? > > Because of a data coding error, all of our analysis of social issues is incorrect. Yeah, sure, that's why the analysis was incorrect, it was all about that typo...
cipheredStones 2y ago
I really enjoyed this phrasing, from the comments on the post: > I suspect the careerization of Big Science and Big Academia has a lot to do with [unwillingness to plainly admit error]. Although I have no proof to that effect. And certainly man has been subject to failings since Adam and Eve chomped.
fn-mote 2y ago
The corrections are wonderful. Concise and to the point. The author is a powerhouse in his field, known by researchers and casual readers. He has nothing to fear from publishing corrections. Nobody is trying to step on him, keep him from promotion, or argue that his research is meaningless. Other researchers have less privilege, and (imo) therefore will be less forthcoming. If you only have two papers, admitting one has a fatal flaw instead of giving more talks about it will be career suicide.

128 points by Tomte 2y ago

> This entry was posted in [...] Zombies I feel like I'm missing a joke or reference somewhere - can anyone help me out? I'm familiar with the concept of p-zombies, but that doesn't seem to be relevant here - and a "Zombie Statistic"[0] is sorta-related in that it refers to the psychological power of incorrect assertions, but I'm still struggling to see the link. [0] https://absolutelymaybe.plos.org/2019/11/30/the-power-of-zombie-statistics-systematic-review-edition/ https://absolutelymaybe.plos.org/2019/11/30/the-power-of-zom...

more_corn 2y ago

This is fantastic. Scientific advancement requires graceful, honest and straightforward acknowledgement of mistakes.

baggy_trough 2y ago

The cover up is usually worse than the crime.

gcr 2y ago
Do you think these errata being published are a good thing?
- baggy_trough 2y ago
  Yes of course. That's the point of my comment, which is being downvoted for some reason that eludes me.
  - allannienhuis 2y ago
    Perhaps its because your comment seemed to equate errors with crimes, or at least malicious intent. The language seems a bit provocative for many, detracting from whatever message was intended.
    - gcr 2y ago
      Yeah. That's how I interpreted the comment, and why I responded apprehensively.
  - brudgers 2y ago
    Sometimes I write with "downvotes be damned" in mind and connected with the audience exactly how I intended. But not connecting with the audience is the usual reason. When it happens to me, I take it as feedback on my writing. Maybe I was unclear. Maybe I was wrong. Maybe it was written for a different audience. In those cases, I just try to improve my writing. Anyway, where can responses to your original comment go? They could dispute your maxim and the internet gets another argument where nobody changes their mind. Or they could agree with it and the internet gets another dog pile of cynicisms. Generally, those are not why people come to HN...at least when the form is one-liners. Finally, complaining about downvotes is contrary to the HN guidelines. Good luck.
    - dredmorbius 2y ago
      Excellent, helpful, and appropriate to the context BTW.

aorloff 2y ago

[flagged]

airstrike 2y ago
FWIW, I don't think this is really a Trump era exclusive. People have always believed whatever they wanted to believe, facts and accuracy be damned. IMHO understanding this as an inexorable human condition makes it much easier to understand the world. What actually matters is people can be persuaded to _want_ to believe different things, so the only real leverage is in shaping those wants—not in being right.
- wrs 2y ago
  Also, if you can get people to connect intensely with just one or two of your statements, you can then make other false statements and they won’t care, because that might invalidate the one they really want to believe. The threshold is so low that the shotgun approach of just telling lies continually actually works quite well. Your statements don’t even have to be consistent, so you can A/B test the lies. The normal backpressure to this is that you lose face with your peers because you become known as a liar. But if your peers don’t influence your success, or you just have no peers, it works.
  - toomuchtodo 2y ago
    https://news.ycombinator.com/item?id=42075533 https://news.ycombinator.com/item?id=42075533 ("HN: Misinformation Does Spread Like a Virus, Epidemiology Shows") https://theconversation.com/misinformation-really-does-spread-like-a-virus-suggest-mathematical-models-drawn-from-epidemiology-242679 https://theconversation.com/misinformation-really-does-sprea... ("The Conversation: Misinformation really does spread like a virus, suggest mathematical models drawn from epidemiology")
aprilthird2021 2y ago
This was a prominent mark of the Bush era too (and I'm sure many other Democratic/Republican administrations, it's just my example). I was rewatching Fahrenheit 9/11 because of our government's current foreign engagements, and it's striking how even then people were duped, but back then it was by lack of information, now it is by overinformation

gcr 2y ago

[flagged]

Throw39387 2y ago
[flagged]
- pessimizer 2y ago
  [flagged]
rachofsunshine 2y ago
The paper [1] is linked in the OP. Section 3, the section invalidated by the errata, looks at data on voter preferences. So the answer to your question is "a lot", assuming that you think that "using data on the beliefs of voters" is in some way important to a paper on electoral strategy. [1] http://www.stat.columbia.edu/~gelman/research/published/AOAS150.pdf http://www.stat.columbia.edu/~gelman/research/published/AOAS...

vasco 2y ago

I like that they are very blunt and to the point. It'd be really cool if we can add more social dynamic features to papers / research, and have people correct papers in the open, like twitter community notes, to the point that if you can really disprove it, the paper is now marked "wrong" or something like that on arxiv. Sort of inbetween collaborative science or hunting for patent prior art or bug bounty hunting. Getting paid for it would be even better, making money by sniping issues in research. One can dream. I wonder how much money in bounties it'd take to proof-invalidate whole branches of psychology (insert your own pet peeve here) research in a methodical way, probably not that much.

srockets 2y ago
I too, would like to have what you’re having. Academic publishing doesn’t make the authors money, it costs them money. To publish is a requirement for academic employment, but there’s no incentive to retract, either than revenge, or one’s academic honesty.
- vasco 2y ago
  > Academic publishing doesn’t make the authors money > there’s no incentive to retract (... other than) or one’s academic honesty We agree on the problems, maybe you have other wacky potential solutions other than just saying others wouldn't work!
amoshebb 2y ago
I wish academic papers were more like Wikipedia articles. Currently what I'm working on is really "Building on" one pretty pivotal paper from the 90s, and there's a whole constellation of work that has spawned. So much ink is spilled re-defining the problem, and reading any paper requires going through the system model every time because tons of arbitrary decisions may have been done different. It makes it hard to compare results, and makes almost every statement that reads "Over in this area we're not innovating on, we used the SOTA" wrong, because some other group is innovating in that corner. If instead there was one canonical version of it with an edit history, and I could go try to just re-write one little para and argue in the talk section about it with the one-or-two other groups picking away at that, I feel like things could move faster and be done higher quality. It'd also be a lot easier to peek at other areas. Currently if I have a question like "What's the latest in NeRFs underwater? I remember seeing a paper about that a while ago" I've basically got no idea.

Waterluvian 2y ago

> 1999: It is not clear to us in general how to avoid this sort of false proof, the problem being that the false statement seemed so natural to us that we did not think to look at it carefully. Assuming I understand correctly, this is basically the common issue of being unable to be objective when you’ve lived and breathed a subject matter for long enough. The answer is rigorous peer review, I think.

srockets 2y ago
Peer reviews are (often) also done by people who live and breathe the subject matter.

timkam 2y ago

Without knowing the specific context: I think this really is a good example of how errors should be disclosed. We need to acknowledge that scientists/academics are human; even very competent mathematicians make mistakes and some of these mistakes appear in published papers. What we lack in many fields is a culture and process that allows (and ideally, encourages) one to disclose: "this was wrong, here is how I fixed it, or how it's actually correct". E.g., in the communities I know in Computer Science & AI, I rarely even see errata lists on personal webpages, not to speak of journals that provide a straightforward process for updates. I would even go so far to claim that the current culture, in which honest errors cannot be straightforwardly corrected, plays into the hands of the clearly dishonest "bad apples". Science is, obviously, not a "monotonic" process in which every single paper adds to the truth; this is practically not even the case for mathematics, which is at least monotonic on object-level (but mistakes happen all the time). As a prominent example, consider this impressive list of Feynman errata: https://www.feynmanlectures.caltech.edu/info/flp_errata.html https://www.feynmanlectures.caltech.edu/info/flp_errata.html.

einpoklum 2y ago

> "Should the Democrats move to the left? > > Because of a data coding error, all of our analysis of social issues is incorrect. Yeah, sure, that's why the analysis was incorrect, it was all about that typo...

cipheredStones 2y ago

I really enjoyed this phrasing, from the comments on the post: > I suspect the careerization of Big Science and Big Academia has a lot to do with [unwillingness to plainly admit error]. Although I have no proof to that effect. And certainly man has been subject to failings since Adam and Eve chomped.

fn-mote 2y ago

The corrections are wonderful. Concise and to the point. The author is a powerhouse in his field, known by researchers and casual readers. He has nothing to fear from publishing corrections. Nobody is trying to step on him, keep him from promotion, or argue that his research is meaningless. Other researchers have less privilege, and (imo) therefore will be less forthcoming. If you only have two papers, admitting one has a fatal flaw instead of giving more talks about it will be career suicide.