9 ms·

Too good to be true: when overwhelming evidence fails to convince (2016)

89 points by beefman 4y ago

Centigonal 4y ago
The way this is written, the syncretic allusions to different subject matter, the obsession with Bayes... I feel like I'm a teenager reading LessWrong articles about cognitive biases again. :')
- nomel 4y ago
  Could you expand on that a bit?
  - whatshisface 4y ago
    A huge fraction of LessWrong articles are actually about current events politics, but since you can't talk about current events on that website they mainly just allude to it while describing how they think their ideology should apply to interpreting it. (I know they don't consider it an ideology but that's just the word that describes it best.)
    - bombcar 4y ago
      If you go back in time and read "current events politics" is kind of amazingly sad how little it all matters.
O__________O 4y ago
(2016) was year of publication. Press coverage: - https://phys.org/news/2016-01-evidence-bad.amp https://phys.org/news/2016-01-evidence-bad.amp Presentation by authors: - https://m.youtube.com/watch?v=Uz6xUjJHTII https://m.youtube.com/watch?v=Uz6xUjJHTII
3fe9a03ccd14ca5 4y ago
I feel this way about the vaccine. “95% effective at preventing infection” etc etc. I really believed it would stop transmission which we also know now is not true. Should have been more skeptical. And before you say “that was before variants” remember that wasn’t the messaging. The message was “it’s safe and effective and protects others” and also “employer: based on what the CDC says take the vaccine or lose your job” A lot of damaged trust in institutions these last few years.
- galaxyLogic 4y ago
  I would say what NIH put out may have been "propaganda". But propaganda for a good reason: Saving the civilization. Think Will Smith/I am a Legend. So far we know that vaccines did very little harm and very much saved lives. And they saved lives because many people believed what NIH was saying. In other words they made a good effort in good faith. Of course if they go too inaccurate they will lose their credibility which is not good. So why would they do so, I think they didn't.
  - omginternets 4y ago
    >Think Will Smith/I am a Legend. Certainly there was great cause for concern and need for prudence in the early days of the pandemic, but COVID-19 was never an end-of-the-world scenario. Your appeal to I am Legend reinforces the parent's argument. >So far we know that vaccines did very little harm and very much saved lives. And they saved lives because many people believed what NIH was saying. In other words they made a good effort in good faith. You are begging the question: did the benefits outweigh the cost? To answer this, we need good data, unencumbered scientific debate, and time. There is reason to suspect at least some of the data and processes used to authorize vaccines were of poor quality, or perhaps even subverted; it is hopefully clear that scientific debate is more restricted than usual; and, we have not had time to observe any long-term effects of vaccines, especially on populations for which the risk of COVID-19 is extremely small (e.g. children). There is a vast middle-ground between anti-vaxxer and vax-maximalist that a reasonable and prudent person can occupy. >Of course if they go too inaccurate they will lose their credibility which is not good. So why would they do so, I think they didn't. You are presupposing that our institutions are rational actors, and that they are acting deliberately. Institutions can fail to perform their essential functions without malice. A conspiracy is not required for a more-dangerous-than-COVID vaccine policy to have taken place. Whether or not this actually happened is a matter of nuanced debate.
    - galaxyLogic 4y ago
      > You are begging the question: did the benefits outweigh the cost? Certainly a retrospective is needed, I assume somebody is doing it. But even if it turned out that benefits did not outweigh the cost, that was unknowable at the beginning of the pandemic. Therefore it was prudent and wise to err on the side of caution. Hindsight is 20-20. We know and knew that vaccines work. Vaccines save lives. That fact has not changed because of Covid has it? In the US > 1 million people died because of Covid. Most of them unvaccinated. Without vaccines it could have been millions more deaths. And with less anti-vaccine propaganda, and better pro-vaccine propaganda, it would probably been many fewer deaths. So rather than simply pondering (and suggesting) the question of whether "benefits outweighed the cost" we need to also consider the alternative-cost. How many more would have, or could have, died without the vaccines?
gotoeleven 4y ago
Since we know that the NIH will block research that doesn't fit whatever is fashionable at the moment how can we trust anything they put out? If contrary results are forbidden then its not science. See for example this story from yesterday.. https://news.ycombinator.com/item?id=33269567 https://news.ycombinator.com/item?id=33269567
- themitigating 4y ago
  Every group, business, person, and country has lied at some point in their existence. Trustworthiness is not a binary value.
  - water8 4y ago
    It should always be the standard for public institutions. Just because we are not perfect does not mean we should not strive for perfection.
    - astrange 4y ago
      This is the cause of why NIH/Fauci said untrue things about Covid; he can't say true things, because new true information is too surprising and you'd lose trust in him due to it changing too fast. Instead, he can only say things that "sound like what a trustworthy person would say", which is why they have to stick with medical dogma even when it's clearly expired.
      - water8 4y ago
        No people dont trust fauci because of his absurd sense of entitlement and authority on something that is not knowable literally. You cannot unequivocally state that there will be no 10 year side effects for something that has not existed for 10 years. It is logically and physically impossible to make that statement and say it is a fact. Fauci confuses his opinion with the word fact. his hubris is why people don’t trust him.
        4y ago
giantg2 4y ago
Natural systems are massively complex. We know a lot, but it is still a small fraction. It's right to question things that seem too good to be true. Sometimes that may lead to further discovery about the underlying mechanisms. Although there are plenty of drugs out there that we don't know why they work, we just know that they do. And on that note, most doctors (that you know personally anyways) will be skeptical of new treatments as they've seen the unintended consequences of previous miracle treatments. Things that have stood the test of time at least give you a better understanding of the potential outcomes. Edit: why disagree?
- themitigating 4y ago
  You're assuming that the default is harmless. If you are seeing a doctor to seek treatment for a condition and you do nothing you still have the condition and whatever it brings. If you are deciding whether to get a vaccine for a deadly disease but 10% of the people who get it die you need to weigh that against probability you could get the disease and die. It's not 10% death vs nothing Doing nothing is still an action.
  - giantg2 4y ago
    I'm not saying to do nothing. I'm saying new isn't necessarily better. If there's a long standing treatment with good outcomes and low side effects, then maybe it's best to use the old treatment. Take traditional joint replacement verses the cobalt replacements as an example.
- breck 4y ago
  I agree. I would frame though as even if you are 99.9% certain of something, you can never be so certain as to how important that thing is relative to other things. Often you are certain about something but it turns out to be a minor detail in a bigger game.
robocat 4y ago
More viewable link to paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/?report=classic https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/?report... Or PDF: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/pdf/rspa20150748.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/pdf/rsp...
csours 4y ago
Why isn't this in a psychology journal? This looks like it was presented at an engineering conference?
- decisionmaker 4y ago
  Looks like somebody did follow up on this in psychology: https://gershmanlab.com/pubs/BhuiGershman20a.pdf https://gershmanlab.com/pubs/BhuiGershman20a.pdf
jesuscript 4y ago
The headline is making ominous a phenomenon that English already has a word for. Mistrust. It’s not about the evidence, it’s about loss of trust. Many people have lost trust in institutions, not evidence.
- BiteCode_dev 4y ago
  Not to mention verifying evidence is very hard, manufacturing evidence can be cheap, and instutitions have a bad track record about evidences. After 20 years of internet saving the lies of people forever, showing thousands of submarine articles, photoshopped pics and now deep faked materials, can we blame people?
  - jesuscript 4y ago
    Of course not, plus the last two decades included two bullshit wars (straight lies), an economic collapse where institutions lied, and hysteria over the severity of Covid on key demographics (masks, politicization, and lockdowns on non-critical demographics, e.g non-seniors). And that’s just America (or the West). How anyone trusts foreign governments of the totalitarian type is something else entirely.
    - colpabar 4y ago
      This is what drives me absolutely nuts whenever I read about current events. Pfizer had to pay the largest fine in US history for a fraud case, yet questioning their data (which we weren't allowed to see until they were sued) is forbidden. We spent decades destroying the middle east with the justification being based on lies, yet to question the motives of the US in the current war is to be a traitor. Fauci was the one in charge of the response to AIDS, which everyone seems to agree was horrendous, yet he's now a national hero. I really struggle to understand how everyone can just go along with it all and forget about everything when the next thing starts.
      - bb88 4y ago
        > Fauci was the one in charge of the response to AIDS, which everyone seems to agree was horrendous, yet he's now a national hero. That's a very self-serving argument cherry picking from history. Fauci was initially hated by the AIDS activists in the 80s but worked to gain their trust. You also don't factor in the societal hate americans had towards the gay population preventing research towards a cure. Here's an article that gives a detailed account of what it was like back in the 80's. https://www.theatlantic.com/politics/archive/2011/06/the-heroic-story-of-how-congress-first-confronted-aids/240131/ https://www.theatlantic.com/politics/archive/2011/06/the-her...
RcouF1uZ4gsC 4y ago
It seems like advertisers know this: Witness the number of advertisements that feature: “Preferred by 4 out of 5 <x>”.
andrewmutz 4y ago
If my test suite never ever goes red, then I don't feel as confident in my code as when I have a small number of red tests.
- sixstringtheory 4y ago
  This is what mutation testing is for.
  - heavenlyblue 4y ago
    Or just plain old coverage reporting
jimjimjim 4y ago
People trust what they FEEL good about. If two equally professional looking sources say different things. One says "you are doing something wrong, you will have to change how you think" and the other says "someone else is doing something wrong, you are thinking correctly" then they will trust the second source. Even when the amount of evidence grows for the first source they will trust the second. This is especially true if they will lose face for having trusted the second source.

89 points by beefman 4y ago

The way this is written, the syncretic allusions to different subject matter, the obsession with Bayes... I feel like I'm a teenager reading LessWrong articles about cognitive biases again. :')

nomel 4y ago
Could you expand on that a bit?
- whatshisface 4y ago
  A huge fraction of LessWrong articles are actually about current events politics, but since you can't talk about current events on that website they mainly just allude to it while describing how they think their ideology should apply to interpreting it. (I know they don't consider it an ideology but that's just the word that describes it best.)
  - bombcar 4y ago
    If you go back in time and read "current events politics" is kind of amazingly sad how little it all matters.

O__________O 4y ago

(2016) was year of publication. Press coverage: - https://phys.org/news/2016-01-evidence-bad.amp https://phys.org/news/2016-01-evidence-bad.amp Presentation by authors: - https://m.youtube.com/watch?v=Uz6xUjJHTII https://m.youtube.com/watch?v=Uz6xUjJHTII

3fe9a03ccd14ca5 4y ago

I feel this way about the vaccine. “95% effective at preventing infection” etc etc. I really believed it would stop transmission which we also know now is not true. Should have been more skeptical. And before you say “that was before variants” remember that wasn’t the messaging. The message was “it’s safe and effective and protects others” and also “employer: based on what the CDC says take the vaccine or lose your job” A lot of damaged trust in institutions these last few years.

galaxyLogic 4y ago
I would say what NIH put out may have been "propaganda". But propaganda for a good reason: Saving the civilization. Think Will Smith/I am a Legend. So far we know that vaccines did very little harm and very much saved lives. And they saved lives because many people believed what NIH was saying. In other words they made a good effort in good faith. Of course if they go too inaccurate they will lose their credibility which is not good. So why would they do so, I think they didn't.
- omginternets 4y ago
  >Think Will Smith/I am a Legend. Certainly there was great cause for concern and need for prudence in the early days of the pandemic, but COVID-19 was never an end-of-the-world scenario. Your appeal to I am Legend reinforces the parent's argument. >So far we know that vaccines did very little harm and very much saved lives. And they saved lives because many people believed what NIH was saying. In other words they made a good effort in good faith. You are begging the question: did the benefits outweigh the cost? To answer this, we need good data, unencumbered scientific debate, and time. There is reason to suspect at least some of the data and processes used to authorize vaccines were of poor quality, or perhaps even subverted; it is hopefully clear that scientific debate is more restricted than usual; and, we have not had time to observe any long-term effects of vaccines, especially on populations for which the risk of COVID-19 is extremely small (e.g. children). There is a vast middle-ground between anti-vaxxer and vax-maximalist that a reasonable and prudent person can occupy. >Of course if they go too inaccurate they will lose their credibility which is not good. So why would they do so, I think they didn't. You are presupposing that our institutions are rational actors, and that they are acting deliberately. Institutions can fail to perform their essential functions without malice. A conspiracy is not required for a more-dangerous-than-COVID vaccine policy to have taken place. Whether or not this actually happened is a matter of nuanced debate.
  - galaxyLogic 4y ago
    > You are begging the question: did the benefits outweigh the cost? Certainly a retrospective is needed, I assume somebody is doing it. But even if it turned out that benefits did not outweigh the cost, that was unknowable at the beginning of the pandemic. Therefore it was prudent and wise to err on the side of caution. Hindsight is 20-20. We know and knew that vaccines work. Vaccines save lives. That fact has not changed because of Covid has it? In the US > 1 million people died because of Covid. Most of them unvaccinated. Without vaccines it could have been millions more deaths. And with less anti-vaccine propaganda, and better pro-vaccine propaganda, it would probably been many fewer deaths. So rather than simply pondering (and suggesting) the question of whether "benefits outweighed the cost" we need to also consider the alternative-cost. How many more would have, or could have, died without the vaccines?

gotoeleven 4y ago

Since we know that the NIH will block research that doesn't fit whatever is fashionable at the moment how can we trust anything they put out? If contrary results are forbidden then its not science. See for example this story from yesterday.. https://news.ycombinator.com/item?id=33269567 https://news.ycombinator.com/item?id=33269567

themitigating 4y ago
Every group, business, person, and country has lied at some point in their existence. Trustworthiness is not a binary value.
- water8 4y ago
  It should always be the standard for public institutions. Just because we are not perfect does not mean we should not strive for perfection.
  - astrange 4y ago
    This is the cause of why NIH/Fauci said untrue things about Covid; he can't say true things, because new true information is too surprising and you'd lose trust in him due to it changing too fast. Instead, he can only say things that "sound like what a trustworthy person would say", which is why they have to stick with medical dogma even when it's clearly expired.
    - water8 4y ago
      No people dont trust fauci because of his absurd sense of entitlement and authority on something that is not knowable literally. You cannot unequivocally state that there will be no 10 year side effects for something that has not existed for 10 years. It is logically and physically impossible to make that statement and say it is a fact. Fauci confuses his opinion with the word fact. his hubris is why people don’t trust him.
      - 4y ago

giantg2 4y ago

Natural systems are massively complex. We know a lot, but it is still a small fraction. It's right to question things that seem too good to be true. Sometimes that may lead to further discovery about the underlying mechanisms. Although there are plenty of drugs out there that we don't know why they work, we just know that they do. And on that note, most doctors (that you know personally anyways) will be skeptical of new treatments as they've seen the unintended consequences of previous miracle treatments. Things that have stood the test of time at least give you a better understanding of the potential outcomes. Edit: why disagree?

themitigating 4y ago
You're assuming that the default is harmless. If you are seeing a doctor to seek treatment for a condition and you do nothing you still have the condition and whatever it brings. If you are deciding whether to get a vaccine for a deadly disease but 10% of the people who get it die you need to weigh that against probability you could get the disease and die. It's not 10% death vs nothing Doing nothing is still an action.
- giantg2 4y ago
  I'm not saying to do nothing. I'm saying new isn't necessarily better. If there's a long standing treatment with good outcomes and low side effects, then maybe it's best to use the old treatment. Take traditional joint replacement verses the cobalt replacements as an example.
breck 4y ago
I agree. I would frame though as even if you are 99.9% certain of something, you can never be so certain as to how important that thing is relative to other things. Often you are certain about something but it turns out to be a minor detail in a bigger game.

robocat 4y ago

More viewable link to paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/?report=classic https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/?report... Or PDF: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/pdf/rspa20150748.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4841483/pdf/rsp...

csours 4y ago

Why isn't this in a psychology journal? This looks like it was presented at an engineering conference?

decisionmaker 4y ago
Looks like somebody did follow up on this in psychology: https://gershmanlab.com/pubs/BhuiGershman20a.pdf https://gershmanlab.com/pubs/BhuiGershman20a.pdf

jesuscript 4y ago

The headline is making ominous a phenomenon that English already has a word for. Mistrust. It’s not about the evidence, it’s about loss of trust. Many people have lost trust in institutions, not evidence.

BiteCode_dev 4y ago
Not to mention verifying evidence is very hard, manufacturing evidence can be cheap, and instutitions have a bad track record about evidences. After 20 years of internet saving the lies of people forever, showing thousands of submarine articles, photoshopped pics and now deep faked materials, can we blame people?
- jesuscript 4y ago
  Of course not, plus the last two decades included two bullshit wars (straight lies), an economic collapse where institutions lied, and hysteria over the severity of Covid on key demographics (masks, politicization, and lockdowns on non-critical demographics, e.g non-seniors). And that’s just America (or the West). How anyone trusts foreign governments of the totalitarian type is something else entirely.
  - colpabar 4y ago
    This is what drives me absolutely nuts whenever I read about current events. Pfizer had to pay the largest fine in US history for a fraud case, yet questioning their data (which we weren't allowed to see until they were sued) is forbidden. We spent decades destroying the middle east with the justification being based on lies, yet to question the motives of the US in the current war is to be a traitor. Fauci was the one in charge of the response to AIDS, which everyone seems to agree was horrendous, yet he's now a national hero. I really struggle to understand how everyone can just go along with it all and forget about everything when the next thing starts.
    - bb88 4y ago
      > Fauci was the one in charge of the response to AIDS, which everyone seems to agree was horrendous, yet he's now a national hero. That's a very self-serving argument cherry picking from history. Fauci was initially hated by the AIDS activists in the 80s but worked to gain their trust. You also don't factor in the societal hate americans had towards the gay population preventing research towards a cure. Here's an article that gives a detailed account of what it was like back in the 80's. https://www.theatlantic.com/politics/archive/2011/06/the-heroic-story-of-how-congress-first-confronted-aids/240131/ https://www.theatlantic.com/politics/archive/2011/06/the-her...

RcouF1uZ4gsC 4y ago

It seems like advertisers know this: Witness the number of advertisements that feature: “Preferred by 4 out of 5 <x>”.

andrewmutz 4y ago

If my test suite never ever goes red, then I don't feel as confident in my code as when I have a small number of red tests.

sixstringtheory 4y ago
This is what mutation testing is for.
- heavenlyblue 4y ago
  Or just plain old coverage reporting

jimjimjim 4y ago

People trust what they FEEL good about. If two equally professional looking sources say different things. One says "you are doing something wrong, you will have to change how you think" and the other says "someone else is doing something wrong, you are thinking correctly" then they will trust the second source. Even when the amount of evidence grows for the first source they will trust the second. This is especially true if they will lose face for having trusted the second source.