18 ms·

The phrase "no evidence" is a red flag for bad science communication (2021)

294 points by NavinF 3y ago

ndsipa_pomu 3y ago
I'm always suspicious when politicians use the phrase "no evidence" or "there's no intention to". They're essentially meaningless as the "no evidence" could easily be due to no-one investigating the issue although it's slightly better if the amount of investigation has been clarified.
- bnralt 3y ago
  Likewise, when it comes to politics I've found "evidence based" to be a red flag. Most of the people using it mean seem to have come to a particular position first, then cherry picked whatever evidence they could find to back up that position. Ironically enough, it usually signifies someone who's ideologically devoted to a particular position.
  - Hnrobert42 3y ago
    Same with “common sense.”
  - ajross 3y ago
    > Most of the [politicians saying "evidence based"] seem to have come to a particular position first, then cherry picked whatever evidence they could find to back up that position. How is that different than anyone else, for example yahoos arguing on the internet? The idea behind "evidence based" is a recognition that public policy has almost always been opinion-driven, and that we should cite evidence in support of our policies before adopting them. Does that mean the evidence is correct? No. (Though even cherry-picked evidence is better than no evidence!). Does it mean you agree with the policy? Certainly not in all cases. But the idea is sound. I think what you're upset about is not the "evidence based", it's that there's a partisan lean to the use of the term and that your tribe, whichever it is, is still doing the opinion thing and doesn't like citing studies.
  - jwond 3y ago
    This brings to mind a pending bill in California where they are trying to amend the state constitution to allow the state to discriminate based on race, sex, etc. as long as it is "research-based" or "research-informed". In practice, I imagine it would enable the state to cherry-pick or even solicit research that would allow them to discriminate in the way that they want. https://leginfo.legislature.ca.gov/faces/billCompareClient.xhtml?bill_id=202320240ACA7&showamends=false https://leginfo.legislature.ca.gov/faces/billCompareClient.x...
  - hnfong 3y ago
    It can be more sinister. (Disclaimer I'm not a scientist) Apparently in scientific circles, people can shoot down new claims by saying it doesn't have enough evidence, while the "default/fallback position" (held by the "establishment") is backed by even less evidence. I'm not saying it's a common thing, but if you're looking for "evidence based" red flags, this might also be something to look out for too.
    - mrguyorama 3y ago
      >Apparently in scientific circles, people can shoot down new claims by saying it doesn't have enough evidence, while the "default/fallback position" (held by the "establishment") is backed by even less evidence. Please provide an example.
- Eddy_Viscosity2 3y ago
  Politicians usually use the phrase "I haven't seen any evidence' which is an even weaker form.
  - HPsquared 3y ago
    Or "I don't recall seeing any evidence"
pcrh 3y ago
Agreed, especially as articles titled "no evidence..." usually contain evidence in favor of the proposition (except that there is also evidence against it). I get similarly annoyed by the use of the term "significant" in statistics, which is a technical term in the field, not an expression of the importance or magnitude of the finding.
Havoc 3y ago
Isn’t it just an indicator that you’re ready a corporate communications PR bullshit piece?
roenxi 3y ago
The thing is that "No Evidence" is just flat untrue. If Crazy Carl says that aliens abducted his prize turnip, that is in itself evidence that aliens did the deed. It is wildly unpersuasive evidence, but nonetheless evidence. People like to believe that "someone says" isn't evidence, which is wildly disconnected from how human society works. Even alleged facts boil down to "someone says". The vast majority of people who know the constant of gravity never measured it for example, they just work with what someone told them - an authority figure saying something is quite strong evidence. Having evidence for untrue things also turns up naturally because of statistical mirages. "No evidence" is a phrase that gets used in untruthful ways. There usually is evidence, of everything. True, false or irrelevant - there will be evidence of it in some form or another. The question is what standard the evidence reaches, not whether it exists.
- Wowfunhappy 3y ago
  What they really mean is "no scientific evidence. If Crazy Carl says that aliens abducted his prize turnip, there is still no scientific evidence of aliens.
  - roenxi 3y ago
    1. That might rule out psychology as a branch of science. 2. This gets back to the point Scott is making - now define "scientific evidence". For any reasonable definition of scientific evidence, there is a lot of evidence for almost everything. I can find scientific evidence for aliens if I want to - to a terribly low standard, true enough - although what Scott picked on was homeopathy which works a bit better. People underestimate lies, damn lies and statistics. There are facts that back up any worldview, no matter how insane. Carl could find scientific evidence to back up his theory. There'll be anomalous weather station readings, weird sounds, academics papers published by crazy academics that kinda support him. We've all met monomaniacal people; they aren't shy of finding scientific evidence for their case. Still bunk though.
    - wongarsu 3y ago
      It also rules out game theory as a science. Which might be fine if you think that math isn't a science, but then you have to resort to physical experiments to collect scientific evidence that 100 + 2 is indeed 102.
ChrisArchitect 3y ago
(2021)
- ChrisArchitect 3y ago
  Some discussion then: https://news.ycombinator.com/item?id=29586998 https://news.ycombinator.com/item?id=29586998
  - nkurz 3y ago
    Hi Chris. You don't know me, but I'd like to discuss something HN related by email. Could you write me at the address in my profile? No hurry, non-urgent.
Al-Khwarizmi 3y ago
I think the author is too naive. First of all, the authors of the declarations in the example headlines are not "scientists and journalists", they are authorities and journalists. People like CDC officials or the director general of the WHO are of course scientists, but they are not speaking as scientists in those news pieces. They are speaking as politicians. Secondly, I don't think they used the "no evidence" template so much with COVID because they were "fundamentally confused". Most of the headlines in those examples are clearly politically motivated: authorities wanted people not to panic, to keep working, and above all, to keep consuming. Hence the proliferation of "no evidence" headlines for things like the virus being airborne (inconvenient for bars, restaurants, shops, etc. - in fact one headline explicitly mentions bars and gyms); kids spreading the virus (inconvenient for parents taking their kids to school and therefore being able to work as ususal), etc. Each and every one of the headlines was very convenient to keep the wheels of "business as usual" spinning, what a coincidence, right? So no, they were not "fundamentally confused", they just were trying to bias the public towards a certain direction, and a great way to do that is to use the "no evidence" narrative which, as the post author explains, is ambiguous so no one can accuse them of outright lying. Note that I'm not saying there aren't examples where such headlines do stem from confusion. I just don't think it's the case of the particular examples in this post.
- darkerside 3y ago
  Hanlon's Razor being to be applied here. People like to point to conspiracy by politicians, but what I really saw were people trying to get the call right, jumping to conclusions, and then failing to realize it. If there were any semi conscious ulterior motivations, they were probably related to personal success.
  - narrator 3y ago
    Hanlon's razor would lead to the acquittal of almost every criminal defendant if applied consistently. Somehow, it only gets applied consistently to politicians, especially those the person invoking it tends to agree with.
  - titzer 3y ago
lp4vn 3y ago
I have the impression that people nowadays are completely infantilized. They will believe anything that supports their worldview and be heavily skeptical of whatever hurts their sensibilities. It's not even to the point of communication anymore, even with proper communication a large part of the population will behave like a 6 years old boy throwing a tantrum. Quick example, we had creationism before 2010, after 2010 something even worse(terraplanism) appeared.
- SiempreViernes 3y ago
  > terraplanism Thanks for informing me about this term, it's awful and I hate it.
renewiltord 3y ago
In general, I think Officious Phrase Use is usually evidence of bullshit. Applies equally to "no evidence", "some evidence", "stop resisting a lawful order", etc. It's usually a sign someone with little on their side is trying to sound official and legit.
NoPie 3y ago
I understand how public can misunderstand this phrase but scientifically it is clear and justified. Sometimes “no evidence” means that we haven't found compelling evidence yet. Some people are desperate and want all the studies to be done immediately and evaluate the risk if the theory is real or not. But for scientists the desire to reach certain outcome is actually counterproductive as it can introduce bias. Slower and less passionate process can lead to better results. For example: 1) We had no evidence of covid being airborne and then we found this evidence. 2) We had no evidence that masks help and then we found no evidence. Two different theories, two different outcomes. Covid is airborne and it changes our understanding (however, we gradually realized that it is impossible to limit the spread and all the measures ultimately were useless). And that masks most likely had very little effect. The reporters could write better for lay public explaining that “no evidence” means that currently we don't have evidence but it could be found later or that “no evidence” is actually that we have a lot of evidence that is indicating in some other direction and the chance of new evidence that rejects those findings is smaller but still could happen.
- darkerside 3y ago
  Also scientifically justified, "Scientists investigating whether COVID is airborne". Either of those headlines leads people down a path. Also, I think the whole question here is, who decides what qualifies as "compelling" evidence?
  - stevenAthompson 3y ago
    >Also, I think the whole question here is, who decides what qualifies as "compelling" evidence? Qualified experts. Any other answer eventually turns into political gibbering and social media nonsense.
- linuxftw 3y ago
  Sometimes 'no evidence' also means there is evidence, they're just ignoring it. Like myocarditis.
  - 3y ago
seanwilson 3y ago
"Statistically significant" is another misleading one for most people I think. People not familiar with the phrase are going to understand it as "important" or "large effect". Something like "high confidence" would be better, along with reporting the effect size. A lot of the time the effect size is low (and will probably disappear with better controls) so leaving this out and leaning on the term "significant" is good for clickbait.
- silent_cal 3y ago
  Same with "not statistically insignificant". People think for two things to be at all related they must be "statistically significantly" related, but it's not true.
Sporktacular 3y ago
I like that, in this age, science tries to keep statements factual. "No evidence" is an objective statement of fact to counter the millions of examples of baseless conspiracy BS getting algorithmically hyped into people's news diet. It's a good thing. As new evidence appears that fact is corrected, and people can act accordingly. Without positive claims of truth, science can try to stay objective.
- mistermann 3y ago
  Can you link to a scientific article that explicitly explains how science has discovered a way to determine nonexistence perfectly (zero possibility for error) in all cases?
  - SiempreViernes 3y ago
    I have a link to such an article but unfortunately its contents only renders legibly in a browser using a more modern epistemology than the one you are currently using, sorry.
    - mistermann 3y ago
      Playing the comedy card, another[1] popular technique one will notice when studying human behavior during the discussion of ideological matters. Perhaps someone will study this phenomenon some day, and perhaps use the knowledge they have obtained to do something to address the problem. [1]https://news.ycombinator.com/item?id=38912340 https://news.ycombinator.com/item?id=38912340
  - Sporktacular 3y ago
    It's such a weird standard to hold. Why does "there is no evidence" have to be perfectly true? It's a falsifiable claim that anyone is free to disprove, so do your own meta analysis and present the counterclaim. Science has to try to be objective and thorough and that gives it credibility. Compare that to the unfalsifiable claims of conspiracists like "China engineered the virus but covered it up so well we have no evidence". And then the same people accepting suggestions of a much weaker claim that it escaped from a lab, as proof that what they said all along has been proven correct. "No evidence" is a perfectly acceptable scientific claim, as open to refutation as any other.
Sporktacular 3y ago
I think "evidence" of why I should or shouldn't believe something is a bigger possible red flag (and deserving of scrutiny) than "no evidence", to which I can remain skeptical and more confident that I'm not being manipulated.
jl6 3y ago
It's an equal-opportunities offender, being deployed in both the "no evidence that X is beneficial" and "no evidence that X is harmful" forms.
pflenker 3y ago
I think the author is too nit-picky here, as the phrase "no evidence" has a superficially similar, but nuanced meaning based on the context - _and this is fine_. It can't be the burden of whoever writes something to cover all possible edge cases of understanding, depending on the medium you need to be able to rely on certain _a priori_ knowledge. We have a lot of similar shorthands in journalism, the one that comes to my mind immediately is the phrase "the markets reacted", even though there is no living, conscious thing which is "the market" and can have a reaction. Similar to "no evidence", this phrase has a meaning which varies in detail but which is superficially the same and it's up to the reader to understand it.
- dml2135 3y ago
  I think your point is valid, but it is also the job of journalists to effectively communicate to the public, so if a turn of phrase is repeatedly misunderstood by a segment of the public, it may make sense to reevaluate it.
javier_e06 3y ago
The article remind me years ago I was in a meeting where a developer had a presentation of his c++ thread factory that we were supposed to use instead of creating our own threads. After the end of the presentation a developer asked a question about his own use-case that also reflected on the complexity of the proposed tools were asked to use. The developer presenting took a pause looking at him and blurted by looking back at the slide in the screen: "In your case, how to use the library, is evident by self-inspection" The meeting concluded ;)
zvmaz 3y ago
I remember at the start of the pandemic a tweet by a famous biologist and philosopher about masks and the spread of the disease. He stated confidently that "there is no evidence" that masks slow down the transmission of the virus (he most probably changed his mind). I had some respect for this guy, but it's all shaken know. In fact, it's all shaken for most intellectuals.
MDWolinski 3y ago
The problem with society is that we live in a world where everyone demands answers NOW and people in positions that need to provide answers MUST give answers. So regardless if a politician or authority figure knows the answer or not, everyone is unwilling (for whatever reasons) to say “I don’t know.” When I worked at Apple retail, we were trained that if we were asked a question that we didn’t know the answer to was to say, “I don’t know, let’s find out together” and you either did some research on Apple’s site or asked someone who may know with the customer. Because nobody wants to say that they don’t know the answer…yet (and that’s key) they take the current working theory and say there’s no evidence to the contrary.
- pb060 3y ago
  Completely unrelated but Apple telling employees what to answer to customers is what makes interactions at their stores feel so artificial and clumsy.
  - MDWolinski 3y ago
    The point of that answer in specific is so the specialist doesn’t make up an answer if they’re not 100% sure it’s the right answer for the customer. Its goal was to make the specialist feel it’s okay to say you don’t know to the customer but that you’re going to work with them to find the right answer.
    - pb060 3y ago
      I just read my comment and realized how unfriendly it was, sorry about that. Thank you for not replying as rudely as I did.
deleted 3y ago
[deleted]
satisfice 3y ago
The status of something as evidence is partly down to a judgement about relevance. If some random person in 1970 jokingly were to have said "Yeah, I killed JFK." That wouldn't have been considered "evidence" of that person's culpability in JFK's death. But if there were other evidence that established the plausibility of that person having possibly killed JFK (worked at the Book Depository, hated JFK, owned a rifle, expert markmsman, etc...) then the statement becomes a kind of evidence-- a confession. So the status of something as "evidence" and not merely data can change based on the context. Here's another example. There was water all over the floor of my kitchen. Initially, we suspected that I may have spilled some water while filling a humidifier. There was evidence for that in the form of my own memory that I had spilled some, a few hours earlier, and there were water streaks down the side of the humidifier. This seemed to be supprting evidence. But then my wife remembered that she had filled a pan to check for leaks in it and then forgot all about it. That pan was now empty. In light of that evidence, which was very compelling (we even verified that there was indeed a leak in that pan) the earlier "evidence" STOPPED being evidence at all.
andrewclunn 3y ago
"Science communication" is a label adopted by so many shills chasing ad revenue (and thus controlled by elite through algorithmic censorship) or outright direct paid shills by corrupted institutions (such as the WHO). The goal therefore was never to more effectively "communicate science," but rather to use the legitimacy of the label of science to promote approved conclusions. This article assumes a misunderstanding where intentional misrepresentation was involved.
deleted 3y ago
[deleted]
Podgajski 3y ago
“there is no evidence that 5G electromagnetic radiation has non-thermal effects on biological beings” is something I hear constantly.
codeflo 3y ago
Words sometimes have a technical and a colloquial meaning. Colloquially, "no evidence" means the thing that technically should be called "unconvincing evidence" or "little evidence". This is similar to the word "theory", which has a different meaning colloquially than technically. Question: If science communicators use the colloquial meaning instead of the technical one, are they bad science communicators? Or especially good ones? I think it's the latter. I also think if you're smart enough to know the technical definition, you should be smart enough to make this translation without making a stink about it. And given that half of the given examples are about COVID, there's some chance that the author has an axe to grind.
- andrewla 3y ago
  "The author" here is none other than Scott Alexander, formerly of Slate Star Codex. For what it's worth, I wouldn't worry too much about his axe grinding; if there's anyone you can take at face value he would be the one. When he has biases he's pretty clear that he has them. You point out a colloquial meaning of the phrase "no evidence", but look closer at his examples -- these are routinely used in both colloquial and technical ways, and in the end muddy the waters more than necessary. In his words, from the article: > You can see the problem. Science communicators are using the same term - “no evidence” - to mean: > 1. This thing is super plausible, and honestly very likely true, but we haven’t checked yet, so we can’t be sure. > 2. We have hard-and-fast evidence that this is false, stop repeating this easily debunked lie. > This is utterly corrosive to anybody trusting science journalism.
zug_zug 3y ago
I like the article and the proposals. I think "no evidence" is problematic and I think Bayesian reasoning is fundamental to science reporting. Maybe it doesn't even go far enough though, sometimes you need domain expertise. Consider: ● Pesticide A was evaluated by the FDA and was declared safe in USA. ● Pesticide A was evaluated and banned in Europe. ● Pesticide A's molecule is similar to pesticide B, which is declared unsafe in USA. ● Of the last 10,000 pesticides to be declared safe by the USA, it actually turns out that Y% of them end up later being declared unsafe or only safer at a lesser dose. ● Pesticide A has only been evaluated by giving a 10,000x dose and looking for effects within a 1-week period. It has never been evaluated by giving a natural dose and looking for long term effects, nor effects on pregnant mothers. ● State that uses Pesticide A (but also many other products) has a 10x elevation of very rare cancers This is a very realistic picture of the type of datapoints you might start with. --- Or consider a case like this: ● Fighter pilot says he saw an alien spacecraft accelerate 100x the speed of a rocket with no noise, or visible propellant ● 50 other people say they saw strange things in incidents too ● There's good reason to think that faster-than-light-travel violates causality ● Scientists haven't found any radiowaves that indicate intelligent life anywhere they have looked I think the point is: I don't see how you get good answers without a bunch of really good bayesianists in the FDA or in Science
- hnfong 3y ago
  I totally agree, but even good bayesianists still can't deal with the problem that people want concrete answers, and they don't accept "it depends on your priors"... :-/
  - BobaFloutist 3y ago
    Isn't it part of the job of the good Bayesianists to establish their own reasonable set of priors? It seems silly for a layman to ask a statistician if there's any there there and have the statistician say "Well that depends, what do you think?"
    - hnfong 3y ago
      The whole point of priors is that everyone can have different ones. If the priors don't affect the results too much (due to a huge amount of evidence), then you can just get the same results using frequentist methods.
0xTJ 3y ago
In my opinion, this says more about poor education standards regarding critical thinking.
cturner 3y ago
I am not fond of the author's substitute suggestion - "Scientists: Snake Oil Doesn’t Work". This turns scientists into a priesthood of experts, and that is not what science is about. Science is a process, not an authority. We should not "trust the science". We should analyse and question it for ourselves. I think the essential problem with the style of journalism from the article is that it reduces the substance of stories into an appeal-to-authority. It is not a trustworthy authority - there is plenty of history for people employed as scientists to make assertions that now look ridiculous. Journalists should put more focus to the arguments and less to the conclusions.
- taeric 3y ago
  I thought there was some subtlety where snake oil does, in fact, work? The problem was "snake oil" sold by people that were deceptive as to what they were actually selling. Do I remember that history wrong?
- VoodooJuJu 3y ago
  >This turns scientists into a priesthood of experts >We should not "trust the science" Unfortunately, those are accurate descriptions of many people's relationship with science, including many academics and scientists themselves. Whether by malice or subconscious self-deception, the most used and useful part of scientific studies is the conclusion, without regard for the reproducibility and efficacy of the experiments or observations. The veracity of these conclusions is weighted by two things: 1. The existence of data-gathering and experimenting rituals, but not necessarily the soundness of the data or experiments 2. How aligned the conclusion is with the expectations and beliefs of the scientists, their funders, and their target prospects
vjk800 3y ago
There can't be evidence for something for which no evidence was sought. Coming from academic physics background, the state of medical research boggles me. Let's start from so called "meta studies". Why are they needed? You'd think that if the individual studies were done carefully enough, a single study would be enough to establish a fact. Instead, you see meta studies with sentences like "we examined 142 studies on X for Y, out of which 76% showed statistically significant improvement". If we assume that the majority "vote" here is the correct result, what the hell did the 24% of the studies do wrong? Why aren't the people who made them expunged from the community for doing questionable research?
- remus 3y ago
  > what the hell did the 24% of the studies do wrong? Why aren't the people who made them expunged from the community for doing questionable research? I think 'expunging people from the community' is a pretty risky path to go down. Perhaps they're looking for a small effect and doing a single study with high enough power is tricky? The real world is a messy place, there's loads of (perfectly innocent, as well as nefarious) ways those 24% of studies could have come to a different conclusion.
csours 3y ago
'No evidence' is not a satisfying answer. It's very easy to satisfy the human mind. It's very easy to stay satisfied. The human mind wants to be satisfied, it does not want the truth. Consult any list of fallacies to see the difference between facts and satisfaction. The true nature of the universe is very complex and hard to understand. Thus we have science. Science, as a practiced art, is a way to delay satisfaction of the mind. My mind will not be satisfied about question X until certain questions are answered. (Bear with me) It can be very clearly observed that the Sun goes around the Earth. Every day I can observe this. I am satisfied. It is ONLY when other questions come up, that I must become unsatisfied and make more observations, and come up with a new theory. I may observe the phases of the moon, and see that the Sun shines on the lunar surface from a certain angle. This observation is not consistent with the Sun orbiting the Earth, as I know that the Moon is closer to Earth than the Sun; thus the daily Lunar cycle should match Earth's cycle. My observations are not consistent. (There's also a lot of other evidence, but I like this one). In fact, it is just an illusion caused by the Earth spinning round. ==== Science communicators need to talk about tests a lot more - what mental model is an expert using, and how do they test that model? How is evidence incorporated into that model to update it?
hot_gril 3y ago
Even if mainstream news fixed this "no evidence" problem, their headlines and articles are already playing too many games to be a good source of scientific knowledge.
bo1024 3y ago
Another important meaning of the phrase is "we searched for evidence of X and failed to find it". In other words, we failed to reject the null. I think Scott means to wrap that up in the second bullet point ('we have evidence X is false'), but of course, there are important distinctions there. Of which journalists may not be aware, or may not spend much time on.
TimTheTinker 3y ago
What an incredibly good article. I hope science journalists take notice.
a-dub 3y ago
this is succinctly captured by the truth table for logical implication. let p be the existence of evidence and q be the state. if p is false, the expression is true regardless of the value of q.
zeroCalories 3y ago
[dead]
lincpa 3y ago
[dead]
xianshou 3y ago
An excellent example of an anti-shibboleth, also known as a "frisco": https://allthingslinguistic.com/post/45448571632/the-opposite-of-a-shibboleth-is-a-frisco https://allthingslinguistic.com/post/45448571632/the-opposit... These signal words act as a silent, passive test of competence in that once you hear them, you can be assured of the speaker's ignorance and discount their opinions accordingly.
gdubs 3y ago
The thing a lot of people here seem to still be missing is that "no evidence", coming from scientific authorities, carries an implication that there's no evidence. But with science, we don't actually _know_ that to be true unless we look. This is an important distinction because in popular culture, 'no evidence' seems to get a lot more headline space than 'we have an untested hypothesis'. Over the past decade, it feels like the aspect of curiosity and hypothesis have disappeared from science, which has become much more institutionalized, compartmentalized, and formalized. It's 'peer review or gtfo'. It is not literally correct to say there's 'no evidence'. We don't know! There could be! 'No evidence _yet_' at the absolute minimum.
orblivion 3y ago
Before even looking at the content of the article I agree. This phrase drives me nuts. It's so imprecise. How hard did you look for evidence? Did you discount evidence from "known deniers"? Do you count statistical anomalies as evidence? If not, what name should we use for those, because they're not _nothing_. And this ambiguity, imho, is exploited politically all the time.
- orblivion 3y ago
  Having read it, I'll add that it seems like journalists would never adopt these suggestions because they're afraid (for better or worse) of letting the public think for themselves. They want to sound authoritative because they've made the decision for the public.
tou23 3y ago
Instead of saying "there is no evidence that Covid is airborne", they should say "there is no evidence about whether Covid is airborne or not", in other words, there is no evidence one way or the other. Just saying "no evidence that X" omits the crucial detail -- whether there is evidence that (not X).
bilsbie 3y ago
I just assume all science communication is bad. (I think we are at that point) Instead I keep a corral of trusted communicators on X. Since I’ve vetted them I can trust their takes a bit more. Edit: since people seem to be interested. As a practical example, when that latest high temp super conductor news came out I immediately had two or three different deep dives on my feed with balanced takes and solid explanations.
- karaterobot 3y ago
  We're sort of living through an epistemological crisis. Not only from the postmodern "there is no truth" perspective, but from the venal "truth by itself does not get clicks" perspective in media, as well as the related "incentive structures are not aligned with the rigorous, methodical, expensive accretion of truth" perspective in research. Maybe that's a slight overstatement, but is trending truer rather than falser as the days go by. We anxiously await a corrective we hope will come soon.
thomastjeffery 3y ago
I agree up until this bit: > If the story is that nobody has ever investigated snake oil, and you have no strong opinion on it, and for some reason that’s newsworthy, use the words “either way”: “No Evidence Either Way About Whether Snake Oil Works”. You spent how long telling me that "no evidence" is bad writing, and your solution is "no evidence" + weasel words? That's even worse. Just recognize that the thing you have no opinion on is not newsworthy, and just drop it. That's right, less writing is good journalism. The greater problem here is the fear of authority. If there is a position to present, then present it directly. If the position is that snake oil doesn't work, then just say it. If you aren't a good enough authority, then find someone who is. This is where scientific communication has ultimately failed. The strongest authorities we have all use the weakest language that they can! By refusing to represent a strong position, we have left a void. That void will inevitably be filled by the least scrupulous authors. Whatever fills the void will be implicitly held in awe by the lack of direct criticism we had constructed around the void in the first place. Refusing to participate creates an inverse cult. An anti-cult where everyone agrees to never share what they believe. It spreads its anti-message just as quickly, and cements itself just as divisively, as any evangelist organization does. We must recognize that the anti-cult lives on equal footing with the cult. The true power of each exists, not in their core truth claim, but at the boundary of argument. Argument is engagement, and engagement is fuel. To avoid engagement is to starve.
- oharapj 3y ago
  This is a pretty misguided argument. There are some topics that are still being researched, and if people say anything other than ‘There’s not enough evidence either way and so until we know more sit tight.’ or something to that effect, they’re simply being misleading. At the same time, these topics are being sensationalised by various people and so something should be said about them. You can use stronger words to discuss the sensationalisation itself, but when discussing the topic and the evidence itself rational, even handed language should be used as that is the language of science. Unfortunately using (misleading) stronger language to stand out more against unscrupulous actors merely ruins your integrity and message. It’s better to be a source of good information for the people looking for it than to add to the pile of untrustworthy noise
fasterik 3y ago
This seems like a needlessly pedantic parsing of the phrase "no evidence". I agree with Scott that we should take a Bayesian approach to scientific claims. So what does "no evidence" mean to a Bayesian? It means that I don't have have any data that would cause me to update my priors significantly on a given claim. This is consistent with both the "low evidence" and "low probability" sense of the phrase. Words and phrases can have different meanings in different contexts, and that's fine. It sort of reminds me of debates over the term "proof". It's perfectly legitimate to say that something has been "scientifically proven", as long as we understand that we're not talking about proof in the mathematical or logical sense, but in a probabilistic sense.
NoZebra120vClip 3y ago
The place where this gets really entertaining is certain areas of Wikipedia. Sometimes I research natural medicine by consulting the relevant articles there, and you can really tell when a substance is efficacious, because Wikipedia has gone out of its way to flatly deny the existence of evidence for that, and "the lady doth protest too much" about said lack of evidence. Of course there is abundant evidence that herbal/natural remedies work: thousands and thousands of years' worth, but said evidence has systematically been supressed, destroyed, and denied by that which came after it (many times, on behalf of the Catholic Church which sought to erase paganism in all its forms, which included herbalism and natural healing.) For example, the Aztecs used the poinsetta plant as an antipyretic, which may explain its quite widespread cultivation by the time that the Conquistadores arrived, who subsequently used it as a symbol of Christmas. But it's hilarious to read the medical "experts" of Wikipedia hold their breath and stamp their widdle feet as they insist that stuff doesn't work, because nobody's dissected and tested it in a sterile lab over the last 50 years.
- CyberDildonics 3y ago
  Do you know what alternative medicine is called when it is proven to work? Medicine.
function_seven 3y ago
My (I think related) pet peeve: "There is no safe level of..." I hate that phrase. It implies to many people that any level whatsoever of X is a danger to you. Some examples: "There is no safe level of alcohol consumption" and "There is no safe level of lead". These both may be technically true. That even one beer a year is worse for your body than complete abstinence. Or even a single fishing weight making incidental contact with your hand is worse then never having touched one. But c'mon. The amount of risk posed by very low levels of consumption or exposure is so minuscule that we can safely (hah!) declare it okay. I've seen people calling for lead to be banned from roofing flashing applications, and total bans on alcohol, citing these statements that "there is no safe level...". You give me a reasonable definition of "safe", and I bet I can determine the safe levels of each of those things. The safe level of alcohol consumption is not 0.0000 units. It's somewhere above that. Same with lead exposure. You can have minimal exposure that results in no discernible difference in your quality of life. Where's that line?
- onetoo 3y ago
  I think it's to communicate that even if you think it's not that bad, it's never good: A single beer every month probably won't kill you, but there is a non-zero increase in risk. By definition, any amount is a danger. I believe that kind of communication intends to combat the more laissez faire "a beer here and there won't kill you," which glosses over the fact that, yeah, it probably won't, but it's also not without statistical risk. Getting rid of that attitude reduces the risk of alcohol-related health complications across the population as a whole.
mrguyorama 3y ago
All this "science communication is bad" thinking is missing the forest for the trees; The public doesn't seem to know how to educate themselves (in a time with more possibility to actually educate yourself somewhat than ever before), doesn't understand how to read or interpret a piece of media in general (a high school level concept), seems to gleefully ignore things they WERE taught (like basic math), and often explicitly describes science and education in general as "liberal brainwashing" Yeah, it's going to be impossible to "communicate" with someone who filled their ears with concrete and is shouting as loud as possible "I think you are the devil (literally)" This has nothing to do with science communication. Entire swaths of the American public believe "My ignorance is as good as your evidence", or worse, that if they don't personally understand something, then it can't be true.

294 points by NavinF 3y ago

I'm always suspicious when politicians use the phrase "no evidence" or "there's no intention to". They're essentially meaningless as the "no evidence" could easily be due to no-one investigating the issue although it's slightly better if the amount of investigation has been clarified.

bnralt 3y ago
Likewise, when it comes to politics I've found "evidence based" to be a red flag. Most of the people using it mean seem to have come to a particular position first, then cherry picked whatever evidence they could find to back up that position. Ironically enough, it usually signifies someone who's ideologically devoted to a particular position.
- Hnrobert42 3y ago
  Same with “common sense.”
- ajross 3y ago
  > Most of the [politicians saying "evidence based"] seem to have come to a particular position first, then cherry picked whatever evidence they could find to back up that position. How is that different than anyone else, for example yahoos arguing on the internet? The idea behind "evidence based" is a recognition that public policy has almost always been opinion-driven, and that we should cite evidence in support of our policies before adopting them. Does that mean the evidence is correct? No. (Though even cherry-picked evidence is better than no evidence!). Does it mean you agree with the policy? Certainly not in all cases. But the idea is sound. I think what you're upset about is not the "evidence based", it's that there's a partisan lean to the use of the term and that your tribe, whichever it is, is still doing the opinion thing and doesn't like citing studies.
- jwond 3y ago
  This brings to mind a pending bill in California where they are trying to amend the state constitution to allow the state to discriminate based on race, sex, etc. as long as it is "research-based" or "research-informed". In practice, I imagine it would enable the state to cherry-pick or even solicit research that would allow them to discriminate in the way that they want. https://leginfo.legislature.ca.gov/faces/billCompareClient.xhtml?bill_id=202320240ACA7&showamends=false https://leginfo.legislature.ca.gov/faces/billCompareClient.x...
- hnfong 3y ago
  It can be more sinister. (Disclaimer I'm not a scientist) Apparently in scientific circles, people can shoot down new claims by saying it doesn't have enough evidence, while the "default/fallback position" (held by the "establishment") is backed by even less evidence. I'm not saying it's a common thing, but if you're looking for "evidence based" red flags, this might also be something to look out for too.
  - mrguyorama 3y ago
    >Apparently in scientific circles, people can shoot down new claims by saying it doesn't have enough evidence, while the "default/fallback position" (held by the "establishment") is backed by even less evidence. Please provide an example.
Eddy_Viscosity2 3y ago
Politicians usually use the phrase "I haven't seen any evidence' which is an even weaker form.
- HPsquared 3y ago
  Or "I don't recall seeing any evidence"

pcrh 3y ago

Agreed, especially as articles titled "no evidence..." usually contain evidence in favor of the proposition (except that there is also evidence against it). I get similarly annoyed by the use of the term "significant" in statistics, which is a technical term in the field, not an expression of the importance or magnitude of the finding.

Havoc 3y ago

Isn’t it just an indicator that you’re ready a corporate communications PR bullshit piece?

roenxi 3y ago

The thing is that "No Evidence" is just flat untrue. If Crazy Carl says that aliens abducted his prize turnip, that is in itself evidence that aliens did the deed. It is wildly unpersuasive evidence, but nonetheless evidence. People like to believe that "someone says" isn't evidence, which is wildly disconnected from how human society works. Even alleged facts boil down to "someone says". The vast majority of people who know the constant of gravity never measured it for example, they just work with what someone told them - an authority figure saying something is quite strong evidence. Having evidence for untrue things also turns up naturally because of statistical mirages. "No evidence" is a phrase that gets used in untruthful ways. There usually is evidence, of everything. True, false or irrelevant - there will be evidence of it in some form or another. The question is what standard the evidence reaches, not whether it exists.

Wowfunhappy 3y ago
What they really mean is "no scientific evidence. If Crazy Carl says that aliens abducted his prize turnip, there is still no scientific evidence of aliens.
- roenxi 3y ago
  1. That might rule out psychology as a branch of science. 2. This gets back to the point Scott is making - now define "scientific evidence". For any reasonable definition of scientific evidence, there is a lot of evidence for almost everything. I can find scientific evidence for aliens if I want to - to a terribly low standard, true enough - although what Scott picked on was homeopathy which works a bit better. People underestimate lies, damn lies and statistics. There are facts that back up any worldview, no matter how insane. Carl could find scientific evidence to back up his theory. There'll be anomalous weather station readings, weird sounds, academics papers published by crazy academics that kinda support him. We've all met monomaniacal people; they aren't shy of finding scientific evidence for their case. Still bunk though.
  - wongarsu 3y ago
    It also rules out game theory as a science. Which might be fine if you think that math isn't a science, but then you have to resort to physical experiments to collect scientific evidence that 100 + 2 is indeed 102.

ChrisArchitect 3y ago

(2021)

ChrisArchitect 3y ago
Some discussion then: https://news.ycombinator.com/item?id=29586998 https://news.ycombinator.com/item?id=29586998
- nkurz 3y ago
  Hi Chris. You don't know me, but I'd like to discuss something HN related by email. Could you write me at the address in my profile? No hurry, non-urgent.

Al-Khwarizmi 3y ago

I think the author is too naive. First of all, the authors of the declarations in the example headlines are not "scientists and journalists", they are authorities and journalists. People like CDC officials or the director general of the WHO are of course scientists, but they are not speaking as scientists in those news pieces. They are speaking as politicians. Secondly, I don't think they used the "no evidence" template so much with COVID because they were "fundamentally confused". Most of the headlines in those examples are clearly politically motivated: authorities wanted people not to panic, to keep working, and above all, to keep consuming. Hence the proliferation of "no evidence" headlines for things like the virus being airborne (inconvenient for bars, restaurants, shops, etc. - in fact one headline explicitly mentions bars and gyms); kids spreading the virus (inconvenient for parents taking their kids to school and therefore being able to work as ususal), etc. Each and every one of the headlines was very convenient to keep the wheels of "business as usual" spinning, what a coincidence, right? So no, they were not "fundamentally confused", they just were trying to bias the public towards a certain direction, and a great way to do that is to use the "no evidence" narrative which, as the post author explains, is ambiguous so no one can accuse them of outright lying. Note that I'm not saying there aren't examples where such headlines do stem from confusion. I just don't think it's the case of the particular examples in this post.

darkerside 3y ago
Hanlon's Razor being to be applied here. People like to point to conspiracy by politicians, but what I really saw were people trying to get the call right, jumping to conclusions, and then failing to realize it. If there were any semi conscious ulterior motivations, they were probably related to personal success.
- narrator 3y ago
  Hanlon's razor would lead to the acquittal of almost every criminal defendant if applied consistently. Somehow, it only gets applied consistently to politicians, especially those the person invoking it tends to agree with.
- titzer 3y ago

lp4vn 3y ago

I have the impression that people nowadays are completely infantilized. They will believe anything that supports their worldview and be heavily skeptical of whatever hurts their sensibilities. It's not even to the point of communication anymore, even with proper communication a large part of the population will behave like a 6 years old boy throwing a tantrum. Quick example, we had creationism before 2010, after 2010 something even worse(terraplanism) appeared.

SiempreViernes 3y ago
> terraplanism Thanks for informing me about this term, it's awful and I hate it.

renewiltord 3y ago

In general, I think Officious Phrase Use is usually evidence of bullshit. Applies equally to "no evidence", "some evidence", "stop resisting a lawful order", etc. It's usually a sign someone with little on their side is trying to sound official and legit.

NoPie 3y ago

I understand how public can misunderstand this phrase but scientifically it is clear and justified. Sometimes “no evidence” means that we haven't found compelling evidence yet. Some people are desperate and want all the studies to be done immediately and evaluate the risk if the theory is real or not. But for scientists the desire to reach certain outcome is actually counterproductive as it can introduce bias. Slower and less passionate process can lead to better results. For example: 1) We had no evidence of covid being airborne and then we found this evidence. 2) We had no evidence that masks help and then we found no evidence. Two different theories, two different outcomes. Covid is airborne and it changes our understanding (however, we gradually realized that it is impossible to limit the spread and all the measures ultimately were useless). And that masks most likely had very little effect. The reporters could write better for lay public explaining that “no evidence” means that currently we don't have evidence but it could be found later or that “no evidence” is actually that we have a lot of evidence that is indicating in some other direction and the chance of new evidence that rejects those findings is smaller but still could happen.

darkerside 3y ago
Also scientifically justified, "Scientists investigating whether COVID is airborne". Either of those headlines leads people down a path. Also, I think the whole question here is, who decides what qualifies as "compelling" evidence?
- stevenAthompson 3y ago
  >Also, I think the whole question here is, who decides what qualifies as "compelling" evidence? Qualified experts. Any other answer eventually turns into political gibbering and social media nonsense.
linuxftw 3y ago
Sometimes 'no evidence' also means there is evidence, they're just ignoring it. Like myocarditis.
- 3y ago

seanwilson 3y ago

"Statistically significant" is another misleading one for most people I think. People not familiar with the phrase are going to understand it as "important" or "large effect". Something like "high confidence" would be better, along with reporting the effect size. A lot of the time the effect size is low (and will probably disappear with better controls) so leaving this out and leaning on the term "significant" is good for clickbait.

silent_cal 3y ago
Same with "not statistically insignificant". People think for two things to be at all related they must be "statistically significantly" related, but it's not true.

Sporktacular 3y ago

I like that, in this age, science tries to keep statements factual. "No evidence" is an objective statement of fact to counter the millions of examples of baseless conspiracy BS getting algorithmically hyped into people's news diet. It's a good thing. As new evidence appears that fact is corrected, and people can act accordingly. Without positive claims of truth, science can try to stay objective.

mistermann 3y ago
Can you link to a scientific article that explicitly explains how science has discovered a way to determine nonexistence perfectly (zero possibility for error) in all cases?
- SiempreViernes 3y ago
  I have a link to such an article but unfortunately its contents only renders legibly in a browser using a more modern epistemology than the one you are currently using, sorry.
  - mistermann 3y ago
    Playing the comedy card, another[1] popular technique one will notice when studying human behavior during the discussion of ideological matters. Perhaps someone will study this phenomenon some day, and perhaps use the knowledge they have obtained to do something to address the problem. [1]https://news.ycombinator.com/item?id=38912340 https://news.ycombinator.com/item?id=38912340
- Sporktacular 3y ago
  It's such a weird standard to hold. Why does "there is no evidence" have to be perfectly true? It's a falsifiable claim that anyone is free to disprove, so do your own meta analysis and present the counterclaim. Science has to try to be objective and thorough and that gives it credibility. Compare that to the unfalsifiable claims of conspiracists like "China engineered the virus but covered it up so well we have no evidence". And then the same people accepting suggestions of a much weaker claim that it escaped from a lab, as proof that what they said all along has been proven correct. "No evidence" is a perfectly acceptable scientific claim, as open to refutation as any other.

Sporktacular 3y ago

I think "evidence" of why I should or shouldn't believe something is a bigger possible red flag (and deserving of scrutiny) than "no evidence", to which I can remain skeptical and more confident that I'm not being manipulated.

jl6 3y ago

It's an equal-opportunities offender, being deployed in both the "no evidence that X is beneficial" and "no evidence that X is harmful" forms.

pflenker 3y ago

I think the author is too nit-picky here, as the phrase "no evidence" has a superficially similar, but nuanced meaning based on the context - _and this is fine_. It can't be the burden of whoever writes something to cover all possible edge cases of understanding, depending on the medium you need to be able to rely on certain _a priori_ knowledge. We have a lot of similar shorthands in journalism, the one that comes to my mind immediately is the phrase "the markets reacted", even though there is no living, conscious thing which is "the market" and can have a reaction. Similar to "no evidence", this phrase has a meaning which varies in detail but which is superficially the same and it's up to the reader to understand it.

dml2135 3y ago
I think your point is valid, but it is also the job of journalists to effectively communicate to the public, so if a turn of phrase is repeatedly misunderstood by a segment of the public, it may make sense to reevaluate it.

javier_e06 3y ago

The article remind me years ago I was in a meeting where a developer had a presentation of his c++ thread factory that we were supposed to use instead of creating our own threads. After the end of the presentation a developer asked a question about his own use-case that also reflected on the complexity of the proposed tools were asked to use. The developer presenting took a pause looking at him and blurted by looking back at the slide in the screen: "In your case, how to use the library, is evident by self-inspection" The meeting concluded ;)

zvmaz 3y ago

I remember at the start of the pandemic a tweet by a famous biologist and philosopher about masks and the spread of the disease. He stated confidently that "there is no evidence" that masks slow down the transmission of the virus (he most probably changed his mind). I had some respect for this guy, but it's all shaken know. In fact, it's all shaken for most intellectuals.

MDWolinski 3y ago

The problem with society is that we live in a world where everyone demands answers NOW and people in positions that need to provide answers MUST give answers. So regardless if a politician or authority figure knows the answer or not, everyone is unwilling (for whatever reasons) to say “I don’t know.” When I worked at Apple retail, we were trained that if we were asked a question that we didn’t know the answer to was to say, “I don’t know, let’s find out together” and you either did some research on Apple’s site or asked someone who may know with the customer. Because nobody wants to say that they don’t know the answer…yet (and that’s key) they take the current working theory and say there’s no evidence to the contrary.

pb060 3y ago
Completely unrelated but Apple telling employees what to answer to customers is what makes interactions at their stores feel so artificial and clumsy.
- MDWolinski 3y ago
  The point of that answer in specific is so the specialist doesn’t make up an answer if they’re not 100% sure it’s the right answer for the customer. Its goal was to make the specialist feel it’s okay to say you don’t know to the customer but that you’re going to work with them to find the right answer.
  - pb060 3y ago
    I just read my comment and realized how unfriendly it was, sorry about that. Thank you for not replying as rudely as I did.

deleted 3y ago

[deleted]

satisfice 3y ago

The status of something as evidence is partly down to a judgement about relevance. If some random person in 1970 jokingly were to have said "Yeah, I killed JFK." That wouldn't have been considered "evidence" of that person's culpability in JFK's death. But if there were other evidence that established the plausibility of that person having possibly killed JFK (worked at the Book Depository, hated JFK, owned a rifle, expert markmsman, etc...) then the statement becomes a kind of evidence-- a confession. So the status of something as "evidence" and not merely data can change based on the context. Here's another example. There was water all over the floor of my kitchen. Initially, we suspected that I may have spilled some water while filling a humidifier. There was evidence for that in the form of my own memory that I had spilled some, a few hours earlier, and there were water streaks down the side of the humidifier. This seemed to be supprting evidence. But then my wife remembered that she had filled a pan to check for leaks in it and then forgot all about it. That pan was now empty. In light of that evidence, which was very compelling (we even verified that there was indeed a leak in that pan) the earlier "evidence" STOPPED being evidence at all.

andrewclunn 3y ago

"Science communication" is a label adopted by so many shills chasing ad revenue (and thus controlled by elite through algorithmic censorship) or outright direct paid shills by corrupted institutions (such as the WHO). The goal therefore was never to more effectively "communicate science," but rather to use the legitimacy of the label of science to promote approved conclusions. This article assumes a misunderstanding where intentional misrepresentation was involved.

deleted 3y ago

[deleted]

Podgajski 3y ago

“there is no evidence that 5G electromagnetic radiation has non-thermal effects on biological beings” is something I hear constantly.

codeflo 3y ago

Words sometimes have a technical and a colloquial meaning. Colloquially, "no evidence" means the thing that technically should be called "unconvincing evidence" or "little evidence". This is similar to the word "theory", which has a different meaning colloquially than technically. Question: If science communicators use the colloquial meaning instead of the technical one, are they bad science communicators? Or especially good ones? I think it's the latter. I also think if you're smart enough to know the technical definition, you should be smart enough to make this translation without making a stink about it. And given that half of the given examples are about COVID, there's some chance that the author has an axe to grind.

andrewla 3y ago
"The author" here is none other than Scott Alexander, formerly of Slate Star Codex. For what it's worth, I wouldn't worry too much about his axe grinding; if there's anyone you can take at face value he would be the one. When he has biases he's pretty clear that he has them. You point out a colloquial meaning of the phrase "no evidence", but look closer at his examples -- these are routinely used in both colloquial and technical ways, and in the end muddy the waters more than necessary. In his words, from the article: > You can see the problem. Science communicators are using the same term - “no evidence” - to mean: > 1. This thing is super plausible, and honestly very likely true, but we haven’t checked yet, so we can’t be sure. > 2. We have hard-and-fast evidence that this is false, stop repeating this easily debunked lie. > This is utterly corrosive to anybody trusting science journalism.

zug_zug 3y ago

I like the article and the proposals. I think "no evidence" is problematic and I think Bayesian reasoning is fundamental to science reporting. Maybe it doesn't even go far enough though, sometimes you need domain expertise. Consider: ● Pesticide A was evaluated by the FDA and was declared safe in USA. ● Pesticide A was evaluated and banned in Europe. ● Pesticide A's molecule is similar to pesticide B, which is declared unsafe in USA. ● Of the last 10,000 pesticides to be declared safe by the USA, it actually turns out that Y% of them end up later being declared unsafe or only safer at a lesser dose. ● Pesticide A has only been evaluated by giving a 10,000x dose and looking for effects within a 1-week period. It has never been evaluated by giving a natural dose and looking for long term effects, nor effects on pregnant mothers. ● State that uses Pesticide A (but also many other products) has a 10x elevation of very rare cancers This is a very realistic picture of the type of datapoints you might start with. --- Or consider a case like this: ● Fighter pilot says he saw an alien spacecraft accelerate 100x the speed of a rocket with no noise, or visible propellant ● 50 other people say they saw strange things in incidents too ● There's good reason to think that faster-than-light-travel violates causality ● Scientists haven't found any radiowaves that indicate intelligent life anywhere they have looked I think the point is: I don't see how you get good answers without a bunch of really good bayesianists in the FDA or in Science

hnfong 3y ago
I totally agree, but even good bayesianists still can't deal with the problem that people want concrete answers, and they don't accept "it depends on your priors"... :-/
- BobaFloutist 3y ago
  Isn't it part of the job of the good Bayesianists to establish their own reasonable set of priors? It seems silly for a layman to ask a statistician if there's any there there and have the statistician say "Well that depends, what do you think?"
  - hnfong 3y ago
    The whole point of priors is that everyone can have different ones. If the priors don't affect the results too much (due to a huge amount of evidence), then you can just get the same results using frequentist methods.

0xTJ 3y ago

In my opinion, this says more about poor education standards regarding critical thinking.

cturner 3y ago

I am not fond of the author's substitute suggestion - "Scientists: Snake Oil Doesn’t Work". This turns scientists into a priesthood of experts, and that is not what science is about. Science is a process, not an authority. We should not "trust the science". We should analyse and question it for ourselves. I think the essential problem with the style of journalism from the article is that it reduces the substance of stories into an appeal-to-authority. It is not a trustworthy authority - there is plenty of history for people employed as scientists to make assertions that now look ridiculous. Journalists should put more focus to the arguments and less to the conclusions.

taeric 3y ago
I thought there was some subtlety where snake oil does, in fact, work? The problem was "snake oil" sold by people that were deceptive as to what they were actually selling. Do I remember that history wrong?
VoodooJuJu 3y ago
>This turns scientists into a priesthood of experts >We should not "trust the science" Unfortunately, those are accurate descriptions of many people's relationship with science, including many academics and scientists themselves. Whether by malice or subconscious self-deception, the most used and useful part of scientific studies is the conclusion, without regard for the reproducibility and efficacy of the experiments or observations. The veracity of these conclusions is weighted by two things: 1. The existence of data-gathering and experimenting rituals, but not necessarily the soundness of the data or experiments 2. How aligned the conclusion is with the expectations and beliefs of the scientists, their funders, and their target prospects

vjk800 3y ago

There can't be evidence for something for which no evidence was sought. Coming from academic physics background, the state of medical research boggles me. Let's start from so called "meta studies". Why are they needed? You'd think that if the individual studies were done carefully enough, a single study would be enough to establish a fact. Instead, you see meta studies with sentences like "we examined 142 studies on X for Y, out of which 76% showed statistically significant improvement". If we assume that the majority "vote" here is the correct result, what the hell did the 24% of the studies do wrong? Why aren't the people who made them expunged from the community for doing questionable research?

remus 3y ago
> what the hell did the 24% of the studies do wrong? Why aren't the people who made them expunged from the community for doing questionable research? I think 'expunging people from the community' is a pretty risky path to go down. Perhaps they're looking for a small effect and doing a single study with high enough power is tricky? The real world is a messy place, there's loads of (perfectly innocent, as well as nefarious) ways those 24% of studies could have come to a different conclusion.

csours 3y ago

'No evidence' is not a satisfying answer. It's very easy to satisfy the human mind. It's very easy to stay satisfied. The human mind wants to be satisfied, it does not want the truth. Consult any list of fallacies to see the difference between facts and satisfaction. The true nature of the universe is very complex and hard to understand. Thus we have science. Science, as a practiced art, is a way to delay satisfaction of the mind. My mind will not be satisfied about question X until certain questions are answered. (Bear with me) It can be very clearly observed that the Sun goes around the Earth. Every day I can observe this. I am satisfied. It is ONLY when other questions come up, that I must become unsatisfied and make more observations, and come up with a new theory. I may observe the phases of the moon, and see that the Sun shines on the lunar surface from a certain angle. This observation is not consistent with the Sun orbiting the Earth, as I know that the Moon is closer to Earth than the Sun; thus the daily Lunar cycle should match Earth's cycle. My observations are not consistent. (There's also a lot of other evidence, but I like this one). In fact, it is just an illusion caused by the Earth spinning round. ==== Science communicators need to talk about tests a lot more - what mental model is an expert using, and how do they test that model? How is evidence incorporated into that model to update it?

hot_gril 3y ago

Even if mainstream news fixed this "no evidence" problem, their headlines and articles are already playing too many games to be a good source of scientific knowledge.

bo1024 3y ago

Another important meaning of the phrase is "we searched for evidence of X and failed to find it". In other words, we failed to reject the null. I think Scott means to wrap that up in the second bullet point ('we have evidence X is false'), but of course, there are important distinctions there. Of which journalists may not be aware, or may not spend much time on.

TimTheTinker 3y ago

What an incredibly good article. I hope science journalists take notice.

a-dub 3y ago

this is succinctly captured by the truth table for logical implication. let p be the existence of evidence and q be the state. if p is false, the expression is true regardless of the value of q.

zeroCalories 3y ago

[dead]

lincpa 3y ago

[dead]

xianshou 3y ago

An excellent example of an anti-shibboleth, also known as a "frisco": https://allthingslinguistic.com/post/45448571632/the-opposite-of-a-shibboleth-is-a-frisco https://allthingslinguistic.com/post/45448571632/the-opposit... These signal words act as a silent, passive test of competence in that once you hear them, you can be assured of the speaker's ignorance and discount their opinions accordingly.

gdubs 3y ago

The thing a lot of people here seem to still be missing is that "no evidence", coming from scientific authorities, carries an implication that there's no evidence. But with science, we don't actually _know_ that to be true unless we look. This is an important distinction because in popular culture, 'no evidence' seems to get a lot more headline space than 'we have an untested hypothesis'. Over the past decade, it feels like the aspect of curiosity and hypothesis have disappeared from science, which has become much more institutionalized, compartmentalized, and formalized. It's 'peer review or gtfo'. It is not literally correct to say there's 'no evidence'. We don't know! There could be! 'No evidence _yet_' at the absolute minimum.

orblivion 3y ago

Before even looking at the content of the article I agree. This phrase drives me nuts. It's so imprecise. How hard did you look for evidence? Did you discount evidence from "known deniers"? Do you count statistical anomalies as evidence? If not, what name should we use for those, because they're not _nothing_. And this ambiguity, imho, is exploited politically all the time.

orblivion 3y ago
Having read it, I'll add that it seems like journalists would never adopt these suggestions because they're afraid (for better or worse) of letting the public think for themselves. They want to sound authoritative because they've made the decision for the public.

tou23 3y ago

Instead of saying "there is no evidence that Covid is airborne", they should say "there is no evidence about whether Covid is airborne or not", in other words, there is no evidence one way or the other. Just saying "no evidence that X" omits the crucial detail -- whether there is evidence that (not X).

bilsbie 3y ago

I just assume all science communication is bad. (I think we are at that point) Instead I keep a corral of trusted communicators on X. Since I’ve vetted them I can trust their takes a bit more. Edit: since people seem to be interested. As a practical example, when that latest high temp super conductor news came out I immediately had two or three different deep dives on my feed with balanced takes and solid explanations.

karaterobot 3y ago
We're sort of living through an epistemological crisis. Not only from the postmodern "there is no truth" perspective, but from the venal "truth by itself does not get clicks" perspective in media, as well as the related "incentive structures are not aligned with the rigorous, methodical, expensive accretion of truth" perspective in research. Maybe that's a slight overstatement, but is trending truer rather than falser as the days go by. We anxiously await a corrective we hope will come soon.

thomastjeffery 3y ago

I agree up until this bit: > If the story is that nobody has ever investigated snake oil, and you have no strong opinion on it, and for some reason that’s newsworthy, use the words “either way”: “No Evidence Either Way About Whether Snake Oil Works”. You spent how long telling me that "no evidence" is bad writing, and your solution is "no evidence" + weasel words? That's even worse. Just recognize that the thing you have no opinion on is not newsworthy, and just drop it. That's right, less writing is good journalism. The greater problem here is the fear of authority. If there is a position to present, then present it directly. If the position is that snake oil doesn't work, then just say it. If you aren't a good enough authority, then find someone who is. This is where scientific communication has ultimately failed. The strongest authorities we have all use the weakest language that they can! By refusing to represent a strong position, we have left a void. That void will inevitably be filled by the least scrupulous authors. Whatever fills the void will be implicitly held in awe by the lack of direct criticism we had constructed around the void in the first place. Refusing to participate creates an inverse cult. An anti-cult where everyone agrees to never share what they believe. It spreads its anti-message just as quickly, and cements itself just as divisively, as any evangelist organization does. We must recognize that the anti-cult lives on equal footing with the cult. The true power of each exists, not in their core truth claim, but at the boundary of argument. Argument is engagement, and engagement is fuel. To avoid engagement is to starve.

oharapj 3y ago
This is a pretty misguided argument. There are some topics that are still being researched, and if people say anything other than ‘There’s not enough evidence either way and so until we know more sit tight.’ or something to that effect, they’re simply being misleading. At the same time, these topics are being sensationalised by various people and so something should be said about them. You can use stronger words to discuss the sensationalisation itself, but when discussing the topic and the evidence itself rational, even handed language should be used as that is the language of science. Unfortunately using (misleading) stronger language to stand out more against unscrupulous actors merely ruins your integrity and message. It’s better to be a source of good information for the people looking for it than to add to the pile of untrustworthy noise

fasterik 3y ago

This seems like a needlessly pedantic parsing of the phrase "no evidence". I agree with Scott that we should take a Bayesian approach to scientific claims. So what does "no evidence" mean to a Bayesian? It means that I don't have have any data that would cause me to update my priors significantly on a given claim. This is consistent with both the "low evidence" and "low probability" sense of the phrase. Words and phrases can have different meanings in different contexts, and that's fine. It sort of reminds me of debates over the term "proof". It's perfectly legitimate to say that something has been "scientifically proven", as long as we understand that we're not talking about proof in the mathematical or logical sense, but in a probabilistic sense.

NoZebra120vClip 3y ago

The place where this gets really entertaining is certain areas of Wikipedia. Sometimes I research natural medicine by consulting the relevant articles there, and you can really tell when a substance is efficacious, because Wikipedia has gone out of its way to flatly deny the existence of evidence for that, and "the lady doth protest too much" about said lack of evidence. Of course there is abundant evidence that herbal/natural remedies work: thousands and thousands of years' worth, but said evidence has systematically been supressed, destroyed, and denied by that which came after it (many times, on behalf of the Catholic Church which sought to erase paganism in all its forms, which included herbalism and natural healing.) For example, the Aztecs used the poinsetta plant as an antipyretic, which may explain its quite widespread cultivation by the time that the Conquistadores arrived, who subsequently used it as a symbol of Christmas. But it's hilarious to read the medical "experts" of Wikipedia hold their breath and stamp their widdle feet as they insist that stuff doesn't work, because nobody's dissected and tested it in a sterile lab over the last 50 years.

CyberDildonics 3y ago
Do you know what alternative medicine is called when it is proven to work? Medicine.

function_seven 3y ago

My (I think related) pet peeve: "There is no safe level of..." I hate that phrase. It implies to many people that any level whatsoever of X is a danger to you. Some examples: "There is no safe level of alcohol consumption" and "There is no safe level of lead". These both may be technically true. That even one beer a year is worse for your body than complete abstinence. Or even a single fishing weight making incidental contact with your hand is worse then never having touched one. But c'mon. The amount of risk posed by very low levels of consumption or exposure is so minuscule that we can safely (hah!) declare it okay. I've seen people calling for lead to be banned from roofing flashing applications, and total bans on alcohol, citing these statements that "there is no safe level...". You give me a reasonable definition of "safe", and I bet I can determine the safe levels of each of those things. The safe level of alcohol consumption is not 0.0000 units. It's somewhere above that. Same with lead exposure. You can have minimal exposure that results in no discernible difference in your quality of life. Where's that line?

onetoo 3y ago
I think it's to communicate that even if you think it's not that bad, it's never good: A single beer every month probably won't kill you, but there is a non-zero increase in risk. By definition, any amount is a danger. I believe that kind of communication intends to combat the more laissez faire "a beer here and there won't kill you," which glosses over the fact that, yeah, it probably won't, but it's also not without statistical risk. Getting rid of that attitude reduces the risk of alcohol-related health complications across the population as a whole.

mrguyorama 3y ago

All this "science communication is bad" thinking is missing the forest for the trees; The public doesn't seem to know how to educate themselves (in a time with more possibility to actually educate yourself somewhat than ever before), doesn't understand how to read or interpret a piece of media in general (a high school level concept), seems to gleefully ignore things they WERE taught (like basic math), and often explicitly describes science and education in general as "liberal brainwashing" Yeah, it's going to be impossible to "communicate" with someone who filled their ears with concrete and is shouting as loud as possible "I think you are the devil (literally)" This has nothing to do with science communication. Entire swaths of the American public believe "My ignorance is as good as your evidence", or worse, that if they don't personally understand something, then it can't be true.