6 ms·

The Phrase “No Evidence” Is a Red Flag for Bad Science Communication

77 points by feross 5y ago

deleted 5y ago
[deleted]
adolph 5y ago
Trump has a history of making claims without evidence https://www.usatoday.com/story/news/politics/2017/03/05/trump-has-history-making-claims-without-evidence/98779966/ https://www.usatoday.com/story/news/politics/2017/03/05/trum...
- anonnyj 5y ago
  He's been out for about a year, let it go man.
deleted 5y ago
[deleted]
ncmncm 5y ago
This is where we get, "Absence of evidence is not evidence of absence". That can also be abused, but it takes some doing. Sometimes absence of evidence really is evidence of absence, if you would expect evidence to have surfaced by now. Nobody knows whether 5G will cause cancer. Nobody knew whether 3G and 4G would, either. We ran the experiment on the general public, and with very large N, it seems like not: no rash of cases seems to have appeared. Absent evidence is considered evidence. But that tells us nothing about 5G. It looks like they are going to run that experiment, too. (Does anybody need 5G? Doesn't seem to matter. One thing we know is you and I will be paying for it, regardless.)
- PikachuEXE 5y ago
  Logically absence of evidence would never equal to evidence of absence It at most make the theory more and more convinced over time and/or other things However the possibility of presence would not be eliminated by "absence of evidence". Unless there is evidence that also eliminates the possibility of presence.
- carlmr 5y ago
  >Does anybody need 5G? Doesn't seem to matter. One thing we know is you and I will be paying for it, regardless. Just bought a new phone. I wanted the cheaper non-5G version, since I believe the phone will be broken years or decades before Germany has useful 5G coverage. But the 4G version was sold out.
- pjc50 5y ago
  Nobody's suggested plausible mechanisms for low-power radio in those bands causing cancer, either. The only EM-induced cancer risk you should be worrying about is from the big EM emitter in the sky.
  - ncmncm 5y ago
    There is, in fact, plenty of evidence for nanoampere current causing profound biological effects. I personally attended a talk in 1987 by a biologist who induced insect larvae to grow to 4x normal size by E-M stimulation of such a current. And, there is George Becker's "The Body Electric", which anybody may read, summarizing a whole career, funded by the US Veterans Administration medical program, investigating those effects in detail. Certain modulations of radio-frequency E-M in those bands can induce current of such magnitude, even at power levels such as are produced by cell phones. But the regulatory agencies do not admit effects of modulations; regulations admit only the effect of heating by baseband energy absorption, microwave-oven style. On the plus side, the modulations used for digital transmission are not generally expected to produce coherent currents. But nobody has actually verified any of that. Generally, if you are planning to bathe an entire population in a form of radiation it has never been exposed to at any time in its entire evolutionary history, it should be your responsibility to demonstrate it won't cause any trouble. But, under present FCC rules, it appears not to be. So, instead, the experiment will again be run against the general population, and what happens will happen. Fingers crossed? We have numerous examples from the 20th century of blithely assuming that new physical phenomena are harmless or even beneficial, with subsequent harm. My mother, like many in her generation, had cancer in her nose from (ineffective!) x-radiation treatment for acne 50 years before. That was not caused by heating.
- BaronSamedi 5y ago
  Thanks to the magic of modus tollens, absence of evidence can indeed be evidence of absence. "A 100kt nuclear bomb just went off in your front yard!" Me, looking out the window, nope. It depends, of course, on the argument being made. It could be true in some cases. As cliches go this one is better than "you can't prove a negative"--not true. At least the "evidence of absence" one makes you think about causality and the nature of evidence.
  - ncmncm 5y ago
    If you look out the window, that is evidence. If you are heard to have reported looking out the window, that is itself evidence, because vaporized people don't report. So, not an example. But, actually checking should be expected before proclaiming "no evidence". And, before parroting claims where there really is no evidence. Until as recently as 2000, hospitals were routinely injecting quite large amounts of mercury-based neurotoxic disinfectant (thiomersal) into infants, when they got a dozen simultaneous vaccinations each preserved with the stuff. They stopped putting it into American vaccines about then, without admitting it had caused any harm. AFaIK it is still used in vaccines sent to poor countries. (Thiomersal is, anyway, excreted pretty quickly.)
    - eesmith 5y ago
      > without admitting it had caused any harm There's no evidence it caused harm. For example, quoting https://en.wikipedia.org/wiki/Thiomersal#Toxicology https://en.wikipedia.org/wiki/Thiomersal#Toxicology - "the World Health Organization has concluded that there is no evidence of toxicity from thiomersal in vaccines and no reason on safety grounds to change to more expensive single-dose administration" > They stopped putting it into American vaccines about then Because American worry warts were induced into a false panic by a fraud pushing a false connection between vaccinations and autism, leading to a specific belief that the mercury in thiomersal was the main factor. The US authorities believed the precautionary response of removing thiomersal would increase public confidence in the vaccination system, even without solid evidence that it caused a problem. (The evidence by comparing autism in the US with a country that didn't use thiomersal was that thiomersal did not have a contributing effect.) The US can do this because it has the money that a poorer country does not. However, this precautionary removal caused people like you to believe the authorities were hiding a connection. See https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4826a3.htm https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4826a3.htm for the stated reason. (" There is a significant safety margin incorporated into all the acceptable mercury exposure limits. Furthermore, there are no data or evidence of any harm caused by the level of exposure that some children may have encountered in following the existing immunization schedule. Infants and children who have received thimerosal-containing vaccines do not need to be tested for mercury exposure.") > when they got a dozen simultaneous vaccinations each preserved with the stuff That's exaggerating. https://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolescent.html#birth-15 https://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolesc... lists 7 simultaneous vaccinations for most infants and 11 if including all high-risk groups. However, the CDC link points out "Some but not all of the vaccines recommended routinely for children in the United States contain thimerosal". https://www.cdc.gov/vaccinesafety/concerns/thimerosal/index.html https://www.cdc.gov/vaccinesafety/concerns/thimerosal/index.... says MMR, Varicella, IPV, and pneumococcal conjugate vaccines have never contained thimerosal, which are 3 of the 11.
MichaelBurge 5y ago
I usually use a word like "guess" to describe a prior that sounds plausible but hasn't been confirmed: "Our best guess is that the moon is made of green cheese" or "Our best guess is that jumping out of a plane without a parachute would kill you, but there has not been a formal study"
- salawat 5y ago
  There actually has been a formal double blind study on whether jumping out of planes with parachutes save lives. The result found no life saving correlation, but suffered from other experimental problems. Study did not get "off the ground" so to speak due to lack of participants agreeing to participate in the at height part due to reluctance to accept the risk of being in the placebo'd control group.
comex 5y ago
There is some distinction if you know where to look. “no evidence so far that the disease can be passed through human contact” “No clear evidence Wuhan coronavirus can spread before people show symptoms” “‘No hard evidence’ new coronavirus strain is more transmissible” “At this time, there is no evidence of human-to-human transmission outside China” These are all statements that were thought at the time to be plausible, perhaps even likely, but not proven. In contrast, most of the unqualified “no evidence” claims are for statements that were thought at the time to be false: Covid is airborne; non-N95 masks are effective; Covid came from a lab (still considered doubtful), booster shots are necessary (would still be debatable if not for Omicron). On the other hand, even the least probable of those statements is still orders of magnitude more probable than “homeopathy works”, so there’s definitely a missing distinction there.
- labster 5y ago
  SARS-CoV-1 is airborne, so why would an informed person think that SARS-CoV-2 was not likely to be airborne? Masks are effective in original SARS and influenza, so why would an informed person conclude masks would not likely be effective on the new coronavirus? Influenza needs boosters annually due to variants — why would a reasonable person know either way at the beginning that boosters would be unnecessary? I think you drank too much government koolaid in your list of ostensibly false things. Worries about mask shortages, diplomatic relations with China, availability of vaccines in the global south, and inducing people to get vaccinated the first time all led to our governments lying to us and presenting it as science. I don’t think the actions were malicious, but they ended up being both wrong and bad public policy.
seanwilson 5y ago
Another phrase to avoid I think is "statically significant" or "significant". This is used even when the effect size is small, but people generally hear "significant" and think "a large amount". Small effect sizes can come from experimental bias/flaws that disappear when better experiments are done later too. You also see it in the context of "the result wasn't statistically significant but was bigger than all the rest" as if that means something.
vmception 5y ago
Seriously fuck the whole scientific and medical community for absolutely bungling the communication on this the whole time till this very day. Fumbled the ball over some pedantic irrelevant semantical bullshit that means nothing to the general public the one time the general public needs to hear from them. In addition to how “no evidence” is used around reporters, stupidly “Young” doesn't mean 55 years old anywhere, except in ICUs. And half the messaging seemd like its worrying about fat shaming, we all noticed who actually young gets affected and we all notice who is staying home whether the government is involved in that decision or not Fumbled, bungled, tumbled
eesmith 5y ago
> Is there "no evidence" that using a parachute helps prevent injuries when jumping out of planes? This was the conclusion of a cute paper in the BMJ The BMJ paper doesn't say there was no evidence! The paper explicitly says there was anecdotal evidence. Here's the relevant quote from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ : ] The perception that parachutes are a successful intervention is based largely on anecdotal evidence. Observational data have shown that their use is associated with morbidity and mortality, due to both failure of the intervention1,2 and iatrogenic complications.3 In addition, “natural history” studies of free fall indicate that failure to take or deploy a parachute does not inevitably result in an adverse outcome.4 > Their [the BMJ authors'] point was that "evidence" isn't the same thing as "peer-reviewed journal articles". How does Astral Codex Ten draw this conclusion? The phrase "peer-review" doesn't exist in the BMJ article. The point of the paper is that observational studies should not be rejected out-of-hand. Observational studies can be and are peer-reviewed. > Is there "no evidence" for alien abductions? Some of the circled headlines used "no evidence" or even "'Exactly zero' evidence." But others used "no hard evidence" and "no clear evidence", which are higher bars. So, yes, there is evidence of alien abductions, but quoting an article I found using Google Scholar, it's well characterized as: "With so little hard evidence to support anything having to do with aliens on Earth — in fact, essentially no evidence outside personal testimony" - https://cpb-us-e1.wpmucdn.com/journeys.dartmouth.edu/dist/a/1604/files/2020/12/Psych-7-Essay-3.pdf https://cpb-us-e1.wpmucdn.com/journeys.dartmouth.edu/dist/a/... (It then points out other psychological mechanisms which might explain the evidence, ending "Whether their stories are true or not, their trauma is very real, and it deserves to be treated as such.") > Is there "no evidence" homeopathy works? Sure, there's evidence. But there's also counter-evidence. Is there stronger evidence that homeopathy works than there there was evidence that COVID-19 could spread through the air-con system? Probably. So how is this example supposed to help make the author's point? > Here we should reject journal articles because they disagree with informal evidence! Q: What is the basis of the author's informal evidence? A: "a strong theoretical understanding". Q: Where did that strong theoretical understanding come from? A: Published research. So the author is rejecting a journal article because it disagrees with other journal articles, after being watered through public perceptions. That's not informal evidence! > there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil. The author earlier accepted anecdotal evidence of alien abductions as a type of evidence. How is the author so sure there is no anecdotal evidence that snake oil works? It's because of essays like these I avoid substack links.
ZeroGravitas 5y ago
My take is these two meanings: "We're actively looking for this, because if it's true it would be important, but we've not found it yet so we haven't moved to the next stage that will be required if/when we find it" vs. "People are spreading this lie and we don't want to seem super sure of ourselves because, hey, we might be wrong and we're not wanting to say that such a thing is categorically impossible but currently the experts say no" And yes, it's unfortunate that in English these two meanings coincide to a degree. Language can be hard, especially if your trying to fit it into a tweet or headline and people are actively trying to misunderstand you. But the meta point seems to be, if your political tribe all adopt very stupid opinions on something that you're an expert in, then someone outside the ingroup must be found to blame for that. Of course, if the outgroup all have a stupid opinion then it's proof of their fundamental wrongness and inferiority. But, if we had to lose one, I'd go for replacing the second meaning. I'm quite happy for it to be "Expert consensus" instead, but is that defensible against someone who politically disagrees with you? Apparently not if you follow climate change science. Who is a journalist to decide that 95% of scientists in an area agreeing is consensus. So here we are.
bennyelv 5y ago
This article is a mess. Zooming in on: Is there "no evidence" homeopathy works? No, here's a peer-reviewed study showing that it does. https://pubmed.ncbi.nlm.nih.gov/30202036/ https://pubmed.ncbi.nlm.nih.gov/30202036/ Except the study has been retracted. I quote: Following publication, the journal received criticisms regarding the rationale of this study and the plausibility of its central conclusions. Expert advice was obtained, and the following issues were determined to undermine confidence in the reliability of the study. The in vitro model does not support the main conclusion of the paper that Rhus Tox reduces pain. The qualitative and quantitative composition of the Rhus Tox extract is unknown. Figures 1G and 1H are duplicates; and figures 1I and 1J are duplicates. The majority of experimental points reported in figure 3 panel A are duplicated in figure 3 panel B. The collection, description, analysis and presentation of the behavioural data in Figure 3 is inadequate and cannot be relied upon. As a result the editors are retracting the Article. The authors do not agree with the retraction. -- Not a great example really. It was at this point I decided not to give this any further attention.
shusaku 5y ago
> This doesn’t have the same faux objectivity as This is the crux of the issue I suppose. If you don’t maintain this plausible deniability, you get accused of being biased when you report the actual truth.
rapjr9 5y ago
Too often "no evidence" means "no evidence that I know of" which really means "I don't know". Doctors in particular use "no evidence" this way all the time because they are trained to show confidence in diagnosis when the truth is they often don't know and are just trying things to get some clues. If you suggest something they are not familiar with they'll often say "there's no evidence that works". You have to show them evidence to get them to try it. The evidence may exist, they just don't know about it. Even when it is true that there is no evidence, that often just means no one has actually studied the problem. So "no evidence we can cure the common cold" really means "no one has actually tried to cure the common cold because we consider it too complex a problem so we've never tried". Now that the pandemic has forced a lot of scientists to actually try curing one coronavirus they are actually starting to think maybe they could cure the common cold and the flu and even have some drugs in clinical trials. Curing colds could mean combatting many viruses individually, but that doesn't seem as difficult as mapping the human genome. So what if there are 200 viruses that cause colds? The sooner you start curing them all the sooner colds will be gone. It basically just became dogma so you couldn't get grant funding for it. So another interpretation of "no evidence" is "we haven't tried".
tpoacher 5y ago
> In traditional science, you start with a “null hypothesis” along the lines of “this thing doesn’t happen and nothing about it is interesting”. Then you do your study, and if it gets surprising results, you might end up “rejecting the null hypothesis” and concluding that the interesting thing is true; otherwise, you have “no evidence” for anything except the null. Not true. The degree of evidence for either hypothesis would be determined by its respective p-value. The fact that you then take into account the evidence on either side to make a pragmatic binary decision is a different matter altogether. E.g., a p-value of 0.050000001 is not strong evidence for the null. A p-value of 0.99999 on the other hand, is.

77 points by feross 5y ago

deleted 5y ago

[deleted]

Trump has a history of making claims without evidence https://www.usatoday.com/story/news/politics/2017/03/05/trump-has-history-making-claims-without-evidence/98779966/ https://www.usatoday.com/story/news/politics/2017/03/05/trum...

anonnyj 5y ago
He's been out for about a year, let it go man.

deleted 5y ago

[deleted]

ncmncm 5y ago

This is where we get, "Absence of evidence is not evidence of absence". That can also be abused, but it takes some doing. Sometimes absence of evidence really is evidence of absence, if you would expect evidence to have surfaced by now. Nobody knows whether 5G will cause cancer. Nobody knew whether 3G and 4G would, either. We ran the experiment on the general public, and with very large N, it seems like not: no rash of cases seems to have appeared. Absent evidence is considered evidence. But that tells us nothing about 5G. It looks like they are going to run that experiment, too. (Does anybody need 5G? Doesn't seem to matter. One thing we know is you and I will be paying for it, regardless.)

PikachuEXE 5y ago
Logically absence of evidence would never equal to evidence of absence It at most make the theory more and more convinced over time and/or other things However the possibility of presence would not be eliminated by "absence of evidence". Unless there is evidence that also eliminates the possibility of presence.
carlmr 5y ago
>Does anybody need 5G? Doesn't seem to matter. One thing we know is you and I will be paying for it, regardless. Just bought a new phone. I wanted the cheaper non-5G version, since I believe the phone will be broken years or decades before Germany has useful 5G coverage. But the 4G version was sold out.
pjc50 5y ago
Nobody's suggested plausible mechanisms for low-power radio in those bands causing cancer, either. The only EM-induced cancer risk you should be worrying about is from the big EM emitter in the sky.
- ncmncm 5y ago
  There is, in fact, plenty of evidence for nanoampere current causing profound biological effects. I personally attended a talk in 1987 by a biologist who induced insect larvae to grow to 4x normal size by E-M stimulation of such a current. And, there is George Becker's "The Body Electric", which anybody may read, summarizing a whole career, funded by the US Veterans Administration medical program, investigating those effects in detail. Certain modulations of radio-frequency E-M in those bands can induce current of such magnitude, even at power levels such as are produced by cell phones. But the regulatory agencies do not admit effects of modulations; regulations admit only the effect of heating by baseband energy absorption, microwave-oven style. On the plus side, the modulations used for digital transmission are not generally expected to produce coherent currents. But nobody has actually verified any of that. Generally, if you are planning to bathe an entire population in a form of radiation it has never been exposed to at any time in its entire evolutionary history, it should be your responsibility to demonstrate it won't cause any trouble. But, under present FCC rules, it appears not to be. So, instead, the experiment will again be run against the general population, and what happens will happen. Fingers crossed? We have numerous examples from the 20th century of blithely assuming that new physical phenomena are harmless or even beneficial, with subsequent harm. My mother, like many in her generation, had cancer in her nose from (ineffective!) x-radiation treatment for acne 50 years before. That was not caused by heating.
BaronSamedi 5y ago
Thanks to the magic of modus tollens, absence of evidence can indeed be evidence of absence. "A 100kt nuclear bomb just went off in your front yard!" Me, looking out the window, nope. It depends, of course, on the argument being made. It could be true in some cases. As cliches go this one is better than "you can't prove a negative"--not true. At least the "evidence of absence" one makes you think about causality and the nature of evidence.
- ncmncm 5y ago
  If you look out the window, that is evidence. If you are heard to have reported looking out the window, that is itself evidence, because vaporized people don't report. So, not an example. But, actually checking should be expected before proclaiming "no evidence". And, before parroting claims where there really is no evidence. Until as recently as 2000, hospitals were routinely injecting quite large amounts of mercury-based neurotoxic disinfectant (thiomersal) into infants, when they got a dozen simultaneous vaccinations each preserved with the stuff. They stopped putting it into American vaccines about then, without admitting it had caused any harm. AFaIK it is still used in vaccines sent to poor countries. (Thiomersal is, anyway, excreted pretty quickly.)
  - eesmith 5y ago
    > without admitting it had caused any harm There's no evidence it caused harm. For example, quoting https://en.wikipedia.org/wiki/Thiomersal#Toxicology https://en.wikipedia.org/wiki/Thiomersal#Toxicology - "the World Health Organization has concluded that there is no evidence of toxicity from thiomersal in vaccines and no reason on safety grounds to change to more expensive single-dose administration" > They stopped putting it into American vaccines about then Because American worry warts were induced into a false panic by a fraud pushing a false connection between vaccinations and autism, leading to a specific belief that the mercury in thiomersal was the main factor. The US authorities believed the precautionary response of removing thiomersal would increase public confidence in the vaccination system, even without solid evidence that it caused a problem. (The evidence by comparing autism in the US with a country that didn't use thiomersal was that thiomersal did not have a contributing effect.) The US can do this because it has the money that a poorer country does not. However, this precautionary removal caused people like you to believe the authorities were hiding a connection. See https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4826a3.htm https://www.cdc.gov/mmwr/preview/mmwrhtml/mm4826a3.htm for the stated reason. (" There is a significant safety margin incorporated into all the acceptable mercury exposure limits. Furthermore, there are no data or evidence of any harm caused by the level of exposure that some children may have encountered in following the existing immunization schedule. Infants and children who have received thimerosal-containing vaccines do not need to be tested for mercury exposure.") > when they got a dozen simultaneous vaccinations each preserved with the stuff That's exaggerating. https://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolescent.html#birth-15 https://www.cdc.gov/vaccines/schedules/hcp/imz/child-adolesc... lists 7 simultaneous vaccinations for most infants and 11 if including all high-risk groups. However, the CDC link points out "Some but not all of the vaccines recommended routinely for children in the United States contain thimerosal". https://www.cdc.gov/vaccinesafety/concerns/thimerosal/index.html https://www.cdc.gov/vaccinesafety/concerns/thimerosal/index.... says MMR, Varicella, IPV, and pneumococcal conjugate vaccines have never contained thimerosal, which are 3 of the 11.

MichaelBurge 5y ago

I usually use a word like "guess" to describe a prior that sounds plausible but hasn't been confirmed: "Our best guess is that the moon is made of green cheese" or "Our best guess is that jumping out of a plane without a parachute would kill you, but there has not been a formal study"

salawat 5y ago
There actually has been a formal double blind study on whether jumping out of planes with parachutes save lives. The result found no life saving correlation, but suffered from other experimental problems. Study did not get "off the ground" so to speak due to lack of participants agreeing to participate in the at height part due to reluctance to accept the risk of being in the placebo'd control group.

comex 5y ago

There is some distinction if you know where to look. “no evidence so far that the disease can be passed through human contact” “No clear evidence Wuhan coronavirus can spread before people show symptoms” “‘No hard evidence’ new coronavirus strain is more transmissible” “At this time, there is no evidence of human-to-human transmission outside China” These are all statements that were thought at the time to be plausible, perhaps even likely, but not proven. In contrast, most of the unqualified “no evidence” claims are for statements that were thought at the time to be false: Covid is airborne; non-N95 masks are effective; Covid came from a lab (still considered doubtful), booster shots are necessary (would still be debatable if not for Omicron). On the other hand, even the least probable of those statements is still orders of magnitude more probable than “homeopathy works”, so there’s definitely a missing distinction there.

labster 5y ago
SARS-CoV-1 is airborne, so why would an informed person think that SARS-CoV-2 was not likely to be airborne? Masks are effective in original SARS and influenza, so why would an informed person conclude masks would not likely be effective on the new coronavirus? Influenza needs boosters annually due to variants — why would a reasonable person know either way at the beginning that boosters would be unnecessary? I think you drank too much government koolaid in your list of ostensibly false things. Worries about mask shortages, diplomatic relations with China, availability of vaccines in the global south, and inducing people to get vaccinated the first time all led to our governments lying to us and presenting it as science. I don’t think the actions were malicious, but they ended up being both wrong and bad public policy.

seanwilson 5y ago

Another phrase to avoid I think is "statically significant" or "significant". This is used even when the effect size is small, but people generally hear "significant" and think "a large amount". Small effect sizes can come from experimental bias/flaws that disappear when better experiments are done later too. You also see it in the context of "the result wasn't statistically significant but was bigger than all the rest" as if that means something.

vmception 5y ago

Seriously fuck the whole scientific and medical community for absolutely bungling the communication on this the whole time till this very day. Fumbled the ball over some pedantic irrelevant semantical bullshit that means nothing to the general public the one time the general public needs to hear from them. In addition to how “no evidence” is used around reporters, stupidly “Young” doesn't mean 55 years old anywhere, except in ICUs. And half the messaging seemd like its worrying about fat shaming, we all noticed who actually young gets affected and we all notice who is staying home whether the government is involved in that decision or not Fumbled, bungled, tumbled

eesmith 5y ago

> Is there "no evidence" that using a parachute helps prevent injuries when jumping out of planes? This was the conclusion of a cute paper in the BMJ The BMJ paper doesn't say there was no evidence! The paper explicitly says there was anecdotal evidence. Here's the relevant quote from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ : ] The perception that parachutes are a successful intervention is based largely on anecdotal evidence. Observational data have shown that their use is associated with morbidity and mortality, due to both failure of the intervention1,2 and iatrogenic complications.3 In addition, “natural history” studies of free fall indicate that failure to take or deploy a parachute does not inevitably result in an adverse outcome.4 > Their [the BMJ authors'] point was that "evidence" isn't the same thing as "peer-reviewed journal articles". How does Astral Codex Ten draw this conclusion? The phrase "peer-review" doesn't exist in the BMJ article. The point of the paper is that observational studies should not be rejected out-of-hand. Observational studies can be and are peer-reviewed. > Is there "no evidence" for alien abductions? Some of the circled headlines used "no evidence" or even "'Exactly zero' evidence." But others used "no hard evidence" and "no clear evidence", which are higher bars. So, yes, there is evidence of alien abductions, but quoting an article I found using Google Scholar, it's well characterized as: "With so little hard evidence to support anything having to do with aliens on Earth — in fact, essentially no evidence outside personal testimony" - https://cpb-us-e1.wpmucdn.com/journeys.dartmouth.edu/dist/a/1604/files/2020/12/Psych-7-Essay-3.pdf https://cpb-us-e1.wpmucdn.com/journeys.dartmouth.edu/dist/a/... (It then points out other psychological mechanisms which might explain the evidence, ending "Whether their stories are true or not, their trauma is very real, and it deserves to be treated as such.") > Is there "no evidence" homeopathy works? Sure, there's evidence. But there's also counter-evidence. Is there stronger evidence that homeopathy works than there there was evidence that COVID-19 could spread through the air-con system? Probably. So how is this example supposed to help make the author's point? > Here we should reject journal articles because they disagree with informal evidence! Q: What is the basis of the author's informal evidence? A: "a strong theoretical understanding". Q: Where did that strong theoretical understanding come from? A: Published research. So the author is rejecting a journal article because it disagrees with other journal articles, after being watered through public perceptions. That's not informal evidence! > there is literally not a single piece of evidence anywhere in the world that anyone could use in favor of snake oil. The author earlier accepted anecdotal evidence of alien abductions as a type of evidence. How is the author so sure there is no anecdotal evidence that snake oil works? It's because of essays like these I avoid substack links.

ZeroGravitas 5y ago

My take is these two meanings: "We're actively looking for this, because if it's true it would be important, but we've not found it yet so we haven't moved to the next stage that will be required if/when we find it" vs. "People are spreading this lie and we don't want to seem super sure of ourselves because, hey, we might be wrong and we're not wanting to say that such a thing is categorically impossible but currently the experts say no" And yes, it's unfortunate that in English these two meanings coincide to a degree. Language can be hard, especially if your trying to fit it into a tweet or headline and people are actively trying to misunderstand you. But the meta point seems to be, if your political tribe all adopt very stupid opinions on something that you're an expert in, then someone outside the ingroup must be found to blame for that. Of course, if the outgroup all have a stupid opinion then it's proof of their fundamental wrongness and inferiority. But, if we had to lose one, I'd go for replacing the second meaning. I'm quite happy for it to be "Expert consensus" instead, but is that defensible against someone who politically disagrees with you? Apparently not if you follow climate change science. Who is a journalist to decide that 95% of scientists in an area agreeing is consensus. So here we are.

bennyelv 5y ago

This article is a mess. Zooming in on: Is there "no evidence" homeopathy works? No, here's a peer-reviewed study showing that it does. https://pubmed.ncbi.nlm.nih.gov/30202036/ https://pubmed.ncbi.nlm.nih.gov/30202036/ Except the study has been retracted. I quote: Following publication, the journal received criticisms regarding the rationale of this study and the plausibility of its central conclusions. Expert advice was obtained, and the following issues were determined to undermine confidence in the reliability of the study. The in vitro model does not support the main conclusion of the paper that Rhus Tox reduces pain. The qualitative and quantitative composition of the Rhus Tox extract is unknown. Figures 1G and 1H are duplicates; and figures 1I and 1J are duplicates. The majority of experimental points reported in figure 3 panel A are duplicated in figure 3 panel B. The collection, description, analysis and presentation of the behavioural data in Figure 3 is inadequate and cannot be relied upon. As a result the editors are retracting the Article. The authors do not agree with the retraction. -- Not a great example really. It was at this point I decided not to give this any further attention.

shusaku 5y ago

> This doesn’t have the same faux objectivity as This is the crux of the issue I suppose. If you don’t maintain this plausible deniability, you get accused of being biased when you report the actual truth.

rapjr9 5y ago

Too often "no evidence" means "no evidence that I know of" which really means "I don't know". Doctors in particular use "no evidence" this way all the time because they are trained to show confidence in diagnosis when the truth is they often don't know and are just trying things to get some clues. If you suggest something they are not familiar with they'll often say "there's no evidence that works". You have to show them evidence to get them to try it. The evidence may exist, they just don't know about it. Even when it is true that there is no evidence, that often just means no one has actually studied the problem. So "no evidence we can cure the common cold" really means "no one has actually tried to cure the common cold because we consider it too complex a problem so we've never tried". Now that the pandemic has forced a lot of scientists to actually try curing one coronavirus they are actually starting to think maybe they could cure the common cold and the flu and even have some drugs in clinical trials. Curing colds could mean combatting many viruses individually, but that doesn't seem as difficult as mapping the human genome. So what if there are 200 viruses that cause colds? The sooner you start curing them all the sooner colds will be gone. It basically just became dogma so you couldn't get grant funding for it. So another interpretation of "no evidence" is "we haven't tried".

tpoacher 5y ago

> In traditional science, you start with a “null hypothesis” along the lines of “this thing doesn’t happen and nothing about it is interesting”. Then you do your study, and if it gets surprising results, you might end up “rejecting the null hypothesis” and concluding that the interesting thing is true; otherwise, you have “no evidence” for anything except the null. Not true. The degree of evidence for either hypothesis would be determined by its respective p-value. The fact that you then take into account the evidence on either side to make a pragmatic binary decision is a different matter altogether. E.g., a p-value of 0.050000001 is not strong evidence for the null. A p-value of 0.99999 on the other hand, is.