17 ms·

A Failure, but Not of Prediction

223 points by barry-cotter 7y ago

jajag 7y ago
I thought this was an excellent article, particularly the point about how and (possibly) why the CDC advised against the use of face masks to limit spread of infection during a pandemic. There seems to have been other failings of expert advice in the early days of the pandemic (e.g. see the UK government's apparent change of strategy early on) and they do to me look like failures in academic reasoning, rather than failures of political decision making. Will be interesting to read the findings of the inevitable inquiries after the dust has settled.
- tonyedgecombe 7y ago
  I find it hard to criticise the UK government on that U-turn. The facts changed and they reacted. Prior to this starting a flu-like pandemic was at the top of the governments risk register and they had models and a plan which they started following. There was some initial idiocy of course, like Boris shaking hands with everybody in a hospital he visited. They should probably be criticised more for their general preparedness and tight health care spending in the face of an ageing population.
zuhayeer 7y ago
‪“If you can buy an 80% chance of stopping a deadly pandemic for the cost of having to wear some silly cloth over your face, probably that’s a good deal”‬ The reason the CDC didn’t previously recommend face masks could’ve been because of a lack of RCT. But in order to do that, they would have had to expose subjects to disease without face masks which wouldn’t make sense. Goes to show there has to better ways for judging the viability of something without resorting to a strict binary of “do or do not”
- SilasX 7y ago
  That’s exactly the point he makes by bringing up the lack of RCTs regarding jumping out of a plane with/out a parachute.
- pdonis 7y ago
  > Goes to show there has to better ways for judging the viability of something without resorting to a strict binary of “do or do not” It's not just a question of how to judge but who makes the judgment. Part of the point of the article is that wearing a mask seems like something you should be able to conclude is a good idea based on common sense; you shouldn't need to wait for an expert to tell you, and if the expert is telling you something that seems to conflict with common sense, you shouldn't automatically disregard common sense.
  - tomp 7y ago
    I find conflict of interest a much better indicator of truthfulness than common sense. In this case: doctors wearing masks conflicts with doctors recommend people don’t wear masks.
    - pdonis 7y ago
      Not necessarily. Doctors are exposed to a different set of risks while on the job than ordinary people going about their business. And common sense already tells you that it's a good idea to take precautions when working with sick people, so the fact that doctors wear masks while ordinary people going about their business don't have to should not come as a surprise. That's not to say that doctors (and others) can't have reasons other than "masks don't work" for recommending that people (other than other doctors) don't wear masks. Just that doctors wearing masks themselves, by itself, is not enough to know that their recommendation for ordinary people not to wear masks is unreliable. You need more information.
- seemslegit 7y ago
  Lack of RCT is a crutch (in this case), the practical reason is that as hard as it is to get face masks now it would have been much harder if the entire population was scrambling for them in the early days of the pandemic and given the US size, purchasing power and worldwide visibility it would have made it more difficult world-wide - all Western countries did the same, de-emphasized masks as they were imposing a lockdown and are now emphasizing them when planning for gradual release.
  - williamstein 7y ago
    The article claims to refute your statement by pointing out that "The CDC has been singing the same tune for the past ten years. Swine flu, don’t wear masks. SARS, don’t wear masks. They’ve been really consistent on this point. But why?" This text in the article has several links.
    - seemslegit 7y ago
      https://slatestarcodex.com/blog_images/masks_cdc.png https://slatestarcodex.com/blog_images/masks_cdc.png Seems pretty consistent with my point - if masks weren't believed to be effective they would not be recommended "if unavoidable" , a use / don't use recommendation is not the same as a scientific finding of efficacy - it almost always factors in an implicit cost-benefit analysis of exactly the kind SSC advocates for. Case in point - all countries are now recommending (and in some places legally requiring) masks and not because a bunch of RCTs were conducted in the last two months.
- Karrot_Kream 7y ago
  I disagree. I think the CDC did actually perform a cross-benefit analysis on masks during the initial segment of the outbreak. Given the not-incontrovertible, yet positive, efficacy of masks, combined with the implications of a PPE shortage for healthcare workers meant that there probably was a calculated decision made to not recommend masks. Though at this point, without some more transparency, I'm not sure what we can say about the CDC's decisionmaking here.
  - polemic 7y ago
    Potentially, but then surely a Cost/benefit analysis would strongly encourage the CDC to stockpile them in advance too? Perhaps nations need strategic PPE reserves.
  - roenxi 7y ago
    > not sure what we can say about the CDC's decisionmaking here. We can say "gee, we should appoint more truthful bureaucrats". It isn't a crazy guess that (for this pandemic) a useful mask for a normal everyday person is anything that puts something in front of their face that makes it a little harder to breath and baffles the air currents. There may be a shortage of proven face masks but there is no shortage at all of things that could be sewn up to cover faces and deaden air currents. The cost is tiny vs the risk of it not working. Officials who are more worried about preventing a response to the potential crisis than the potential crisis should not be in charge if there is a potential crisis. We want people to change their behaviors if there is a crisis. Ideally even before the evidence in overwhelming that there is a problem.
    - kd5bjo 7y ago
      The cost of recommending masks is that a large minority of people will decide they need the best mask to protect themselves, and buy them up thus depriving the healthcare workers of them. Yes, in an ideal world, the entire situation would be spelled out and only those people who are truly at higher risk would acquire some of the limited supply of more-effective masks. For that to work, however, requires a sense of community and level of trust in government that has been in short supply in the US in recent years. These factors were also in play for swine flu and the other potential pandemics the article examined CDC rsponses for. In a crisis, officials have to base their decisions on current conditions, not on what they should ideally be. Restoring trust in government is extremely important, but can’t be accomplished in the timescale necessary to be useful for this crisis.
- paganel 7y ago
  Honest question, are there many people out there who find wearing a mask "silly"? Yes, they can be sometimes uncomfortable in terms of ease of breathing but to be honest I don't find them silly at all, they serve a purpose, just like an umbrella serves a purpose, it would be stupid of me to laugh of someone holding an umbrella while it's raining outside.
  - DoofusOfDeath 7y ago
    I haven't noticed any of my acquaintances saying that mask-wearing is silly. But I have noticed many people not observing social distancing and stay-at-home requests. Perhaps if those people ignore the guidance because they're not convinced it's important, those same people would judge mask-wearing to be a waste of effort. Perhaps another issue is people who have heard that wearing cloth masks mostly prevents you from getting others sick, and some of them just can't be bothered to make small sacrifices for the speculative benefit of strangers?
  - lonelappde 7y ago
    It's worse than "silly", it's "Oriental".
- nullc 7y ago
  One thing this analysis misses is the crystals. As in: The CDC is a huge government bureaucracy that employs (and cannot swiftly fire) enormous numbers of people from diverse backgrounds. If RCTs aren't the standard, well then why not also wear crystals to ward off the virus? They're inexpensive, and they couldn't hurt-- you can probably find a poorly controlled study to support it too. The CDC is not human, it is a meat-and-paper-political-machine. You cannot understand it's actions under the same framework you'd use to judge another person. The CDC must contain internal defenses against hokum which are much stronger than any person needs to personally have. So why does the CDC sometimes provide overly conservative advice even when doing so causes death? For the same reason the scorpion stings: it's in its nature. It's far from clear to me that a CDC which did make a more prudent facemask recommendation would actually be a better organization overall. Sure, it would have gotten this one right, but it would have likely given other bad advice as a result.
  - SilasX 7y ago
    >As in: The CDC is a huge government bureaucracy that employs (and cannot swiftly fire) enormous numbers of people from diverse backgrounds. If RCTs aren't the standard, well then why not also wear crystals to ward off the virus? They're inexpensive, and they couldn't hurt-- you can probably find a poorly controlled study to support it too. Alexander's discussion of parachutes gave some good heuristics for when you should trust common sense vs require an RCT, which explains why his advice doesn't generalize to advocating crystals. The heuristic was basically that you should trust common sense when a) there's a solid scientific model of the key dynamics, b) the purported remedy is cheap, and c) the costs of not stopping the threat are disproportionately high. Crystals meet b)[1] and c) but fail a) -- we don't have a validated model of crytal/virus interactions. In contrast, we know that (some) viruses (including this one) spread through unimpeded airflow, and masks impede said airflow. We know that high-speed impacts between your body and the ground are deadly, and parachutes demonstrably slow the falling speed of objects. Thus, you should go with common sense in the absence of an RCT for parachutes and masks but not crystals. [1] They might fail b) depending on the particular charlatan!
    - nullc 7y ago
TheCowboy 7y ago
I like a lot of what is said in this piece. Background: I am currently supporting myself by betting on politics and economics. I was caught jobless when this thing hit, halting my attempt at a career switch into either coding or data science. At the end of February I posted my first forecast for 200k cases by March 20th, giving it a 96% chance. I feel like there exists the possibility of doing better, and there exist people who are already there but no one wants to listen or change. Watching this thing since the start of the year has been like watching a preventable slow motion crash, where many people seemed to be trying their hardest to intentionally fail. Let's not just blame the MSM or "elites" either. I speak to people who confidently assert everything is going to be fine in a year, and when pressed for a probability it's not even in top quartile. Even if it was 90% certain, a 10% chance of a significantly negative outcome is too high. I can open any thread on HN regarding economics and the general level of arrogant ignorance is staggering. We're on target to massively screw up the economic relief, which will make the economic stimulus phase all the more challenging. People need to be more predisposed to having possible intellectual and knowledge blind spots. But I'm not all negative, as I remain very optimistic that people will continue to ignore reasonable warnings.
- starpilot 7y ago
  Where do you bet for money to "support yourself"? Good Judgment Open doesn't have cash awards.
  - graeme 7y ago
    Had they made that bet with out options in mid to late feb they could be a millionaire. For political bets you can bet on betting sites. Much harder to earn an income though.
    - tomp 7y ago
      Although market reaction is fundamentally hard to predict. It caughy many by surprise that markets reacted positively when Trump was elected. Or when there’s 150k people dead from coronavirus.
    - 7y ago
mehrdadn 7y ago
A bit unrelated, but I heard about this 2007 paper recently [1]: > The presence of a large reservoir of SARS-CoV-like viruses in horseshoe bats, together with the culture of eating exotic mammals in southern China, is a time bomb. I know the instant reaction is "people predict things all the time, some are bound to come out right", but this seems so specific that I think it's not just pure chance... we're just not heeding some of the warnings. ------ Now one reaction to the blog post: it does make a lot of good points (especially about masks), but for the part about shutting society at least, it doesn't seem to account for the follow-on effects of being wrong. If governments had shut things down due to a 10% chance of a pandemic, and it was prevented, chances are decent that a lot of not-similarly-reasoning people would've been angry. A lot of people, some for good reason and some for bad, just don't trust their governments (or the media) enough, and of those that do, a fair chunk are going under a lot of hardship to comply. So they don't just say "okay, I don't see it that way, but I trust you that this is for the best, even if it means I can't put bread on the table for my family, so I'll just live with the consequences and stay at home." Now I'm not really saying this is an excuse for governments/media to send poor messages, but it's an understandable human component... surely you can't attribute everything to irrational probabilistic reasoning? [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2176051/#__sec26title https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2176051/#__sec2...
- meowface 7y ago
  But if every government had acted much earlier, when the probability seemed to exceed 10%, there wouldn't have necessarily been any total shutdowns like what we're seeing now (except for the initial containment in China). There would still be some economic harm, but it'd be far, far less.
  - mehrdadn 7y ago
    What would've been there instead of a shutdown and how could that have avoided a shutdown? (Specifically thinking of the US here.)
    - undersuit 7y ago
greendave 7y ago
It's nice to read a well-thought-out piece that both critiques and enlightens. That said, I do think it bears some mention that a substantial amount of the early Coronavirus discussion was being poisoned by people who were not arguing in good faith. That sucked a fair amount of the oxygen out of the debate, and I suspect may have left journalists in less of a position to provide thoughtful context around expert opinions. In other words, flat earthers occupying enough print and mindshare to make it hard to focus on the subtleties of Keplerian motion.
- cma 7y ago
  Sort of like how Trump’s China travel ban announcement was accompanied by a statement saying he wanted to close the Mexican border right away too but they decided against it. Almost to bait the media into accusing him of using the situation for his own ends.
mherdeg 7y ago
I was more or less convinced by the discussion in mid-February at https://virologydownunder.com/past-time-to-tell-the-public-it-will-probably-go-pandemic-and-we-should-all-prepare-now/ https://virologydownunder.com/past-time-to-tell-the-public-i... and https://virologydownunder.com/so-you-think-youve-about-to-be-in-a-pandemic/ https://virologydownunder.com/so-you-think-youve-about-to-be... that there was a reasonable chance of needing to take some modest steps to prepare for a pandemic. The predictions in these blog posts about pandemic best practices were in retrospect fairly accurate (see e.g. discussion under "Social distancing will be important but unpleasant") and have been a pretty good guide for the first month of major international response. We're kind of into uncharted waters now though, and I'm wondering who else will have accurate predictions of how the next year or two will go.
- kd5bjo 7y ago
  > We're kind of into uncharted waters now though, and I'm wondering who else will have accurate predictions of how the next year or two will go. There have been several out-of-control pandemics in human history that we can look at. It’s true that lots of things have changed between the Bubonic plague sweeping through Europe and now, but people are still people; there may be some insights there. You’d have to ask a historian (I’m not one). As different regions get this under control, I expect there to be health screenings required for international travel. In the old days, this was quarantine for inbound travellers, but adding a week or two to travel times is a lot less palatable now. We now have other, faster, ways of determining if someone is ill and I expect them to be deployed to protect areas that have succeeded.
projektfu 7y ago
Sometimes you need to answer the question, what would you pay for that lottery ticket? Sometimes you need to answer, would you pay Russian roulette with that gun? In this case if you had a 90% confidence that Coronavirus wouldn’t escape China or wouldn’t cause catastrophe if it did, then you still have to point that 10-chambered revolver at your head if you choose “don’t prepare”. But buying your way out of the game has a cost, too.
mrfusion 7y ago
Another benefit to thinking in probabilities is that you don’t get as attached to ideas. You don’t feel the need to defend your Opinion past it’s expiration date. People can have better discussions without getting offended.
- lonelappde 7y ago
  Otoh probabilities are hard to check for correctnessso you can weaael out of being wrong by making everything you say a motte-bailey deception (to use a slatestarcodex term).
  - mrfusion 7y ago
    It’s ok to be wrong though.
  - TeMPOraL 7y ago
    That's why it's important to be precise in describing what you mean (which may involve explaining the intended meaning when using a common word), so that you can't back-track on it later. Conversely, when you see someone not doing that, be prepared for the possibility of weaseling out.
deleted 7y ago
[deleted]
jonhohle 7y ago
A major issue, not just with the current pandemic, but all major network news programs, is they are not news, they are entertainment editorials. However, they don’t present themselves as such, and present information as if absolute and unbiased. If there were journalistic standards and not just first to print clickbait, there would have to be nearly as many corrections and retractions as there are news stories. The constitution guarantees freedom to publish, but it would be really nice into have some legal standard for accurate reporting of current events in good faith.
- disgruntledphd2 7y ago
  You could call it a fairness doctrine, or something ;) https://en.wikipedia.org/wiki/FCC_fairness_doctrine https://en.wikipedia.org/wiki/FCC_fairness_doctrine
joe_the_user 7y ago
"First, a bunch of generic smart people on Twitter who got things exactly right... 'Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, and Nicholas Christakis....'" And "...they didn’t beat the experts in epidemiology. Whatever probability of pandemic the experts and prediction markets gave for coronavirus getting really bad, these people didn’t necessarily give a higher probability. They were just better at probabilistic reasoning, so they had different reactions to the same number..." I noticed that. As a math person, things were exceptionally clear at a point, clearer than many uncertain events tbh. Simple exponential growth this, I can understand ... and so could a significant slice of people. But my question is: if epidemiologists aren't experts at THIS, wtf are they experts in? Isn't this exactly the "classic" circumstance any average epidemiologist should be a able to see in a heartbeat. Apparently not, so what do they train these people in? Question maybe half rhetorical but also serious. What's "expert knowledge" in these circumstances.
- __s 7y ago
  So, there's a basis for "Goofus" mentality: If you're trying to compile a set of facts, you very likely want a system which is incomplete, but correct. ie false positives have a significant cost to your models, where you essentially have a technical debt of needing to weed out false facts Policy should follow risk analysis, but research which is not time sensitive has compounding factors which favor building certainty before skipping ahead Think of it like software development: planning/testing/etc have high rewards throughout development, but sometimes you need to sell a rushed MVP
- tlb 7y ago
  Professions exist that effectively have negative knowledge about their field, in the sense that they do worse than smart generalists at making predictions. Closest to home, I'd say Software Engineering mostly fits this description. 90% of people who call themselves that are terrible at actually making software. Or, for example, most of the best fiction is not written by English Literature majors. Perhaps epidemiology is like that. Based on what I've seen in the last few months, I'm not impressed.
  - 7y ago
thaumaturgy 7y ago
There's some shockingly bad thinking in this article and I'm surprised that people seem to be taking it at face value. Picking just a few things that stood out to me: > First, a bunch of generic smart people on Twitter who got things exactly right – there are too many of these people to name, but Scott Aaronson highlights “Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, and Nicholas Christakis.” This is pretty much the definition of cherry-picking. How many equally smart people got it wrong? Was there something in common among these particular people that led them to get it right, or did they hazard a guess on approximately the same information that everyone else had and then end up in the winning group? There's a fun article somewhere in my bookmarks that I can't find right now, but it covers the phenomenon of some bettor -- at the race track, or the stock market, or something -- who has this crazy winning streak and nobody can figure out why. Maybe it's this person's lucky hat, or maybe it's the notes they take, or maybe... who knows. But the most compelling reason is that they just happen to be the 1 member of a very large group who, through probability alone, keeps winning. This other article sort of mentions it in passing: > The prosecution also had an expert witness, an economist named David DeRosa. He didn’t share Heeb’s views about poker. DeRosa used a computer to simulate what might happen if 1,000 people each tossed a coin 10,000 times, assuming a certain outcome — such as tails — was equivalent to a win and the number of times a particular person won the toss was random. But the results were remarkably similar to those Heeb presented: A handful of people appeared to win consistently, and another group seemed to lose a large number of times. This wasn’t evidence that a coin toss involved skill, just that — much like the infinite number of monkeys typing — unlikely events can happen if we look at a large enough group. (https://www.discovermagazine.com/the-sciences/are-the-best-gamblers-skilled-or-just-lucky https://www.discovermagazine.com/the-sciences/are-the-best-g...) Back to the posted article... > parachutes, etc. etc. I mean, all kidding aside, there are enough parachute failures at this point to have a pretty good idea of the grim odds of surviving a fall from an aircraft without one. There are some really good reasons for conservativism in medicine. We have lost many, many lives to cures, treatments, and preventions that were "common sense" and only later discovered to be wrong. Or, sometimes, work, but only accidentally. Scurvy is the historical go-to example here. Its cause and treatment has been discovered, lost, and rediscovered many times, and each time, there was some common-sense treatment for it that was wrong. > Gallant wouldn’t have waited for proof. He would have checked prediction markets and asked top experts for probabilistic judgments. If he heard numbers like 10 or 20 percent, he would have done a cost-benefit analysis and found that putting some tough measures into place, like quarantine and social distancing, would be worthwhile if they had a 10 or 20 percent chance of averting catastrophe. Gallant will spend his life preparing for the next disaster that has a 20% chance of occurring and will eventually end up living in a small off-the-grid cabin deep in the woods, slowly being poisoned by the well water that isn't being tested correctly or often enough. There is an opportunity cost for preparing for disaster. I'm all for being at least a little bit prepared. My favorite hobby involves going out and retrieving people who were unprepared. But I never criticize them for getting into bad situations, because it can happen to any of us. (With the possible exception of people who don't evacuate from hurricane areas. Those people piss me off.) You can't prepare for everything. Most of us have never lived through a situation like we're living through now. How believable would this all have been a year ago? Who could believe that we'd be willing to stop the world economy because of a virus? The irritating nut of this is that this isn't necessarily something you can learn from. Okay, let's say you're one of the people that did the mental math and chose not to react -- a "Goofus", according to this article. So you learn from it, yeah? Next thing comes along with a 10% or 20% chance, you prepare for it. You're in California, so, earthquake: you fill up your garage with supplies. The earthquake never comes. Then there's a 10% or 20% chance of wildfire destroying your community... so you move, except the wildfire never hits that community. Armed robbery. Disease. Storms. All of these have some kind of low probability of happening, but will have super serious consequences if they do, so you -- remembering covid-19 -- prepare for each of them. Decades come and go, and eventually, you decide, hey, you're busy. You can't really deal with that thing right now. And besides, none of the others have happened. Except, this time the floods do come, and you're screwed again, and now some guy is writing about how all those people who didn't properly prepare for the flood had bad mental models for assessing risk. > A few weeks ago, I wrote a blog post on face masks. It reviewed the evidence and found that they probably helped prevent the spread of disease. Then it asked: how did the WHO, CDC, etc get this so wrong? Absent from the blog post he references is that there's uncertainty over whether face masks might actually increase the risk of infection among healthy populations of non-medical professionals. The thinking goes that some number of people don't know how to properly use the masks (which he covers) and that they end up touching their face more as a result, and that the masks that most people are using aren't great filters for this particular virus, so the net change in probability of getting infected is going up. But this isn't a certainty. There isn't enough data to know for sure if this is true or not. Even after this pandemic runs its course, there will probably still be disagreement in the studies to come because of all the confounding factors at play. It's a reasonable position though, and that's why people keep getting conflicting information on this. You can get some treatment of the debate and confusion around this from this discussion: https://old.reddit.com/r/COVID19/comments/fcwcn7/to_mask_or_not_to_mask_the_seemingly_endless/ https://old.reddit.com/r/COVID19/comments/fcwcn7/to_mask_or_... ...people are posting studies supporting one position, then rebutting them, then offering professional advice in one direction, and then others in another. It's a mess, and while that thread isn't rigorously scientific, essentially the same argument is happening at and between the CDC and WHO and others.
ilkan 7y ago
IMHO, the journalists' mistake was lazily relying on easily accessible Western orgs instead of partnering with Eastern-based journalists to examine stories coming out of China. I've been following the South China Morning Post since the first week of January and so far all the observations coming from their experts has been refuted by the CDC first, then confirmed a month or two later. They are just as smart, highly motivated and had at least 10x more casework to examine. Tldr; if you want to make better predictions, compare expert opinion against news closer to the source.
armitron 7y ago
Not even a mention for Taleb who had a paper out in January [1] yet paragraphs for Zeynep and the rest who merely parroted rising consensus? Weak, very weak. [1] https://necsi.edu/systemic-risk-of-pandemic-via-novel-pathogens-coronavirus-a-note https://necsi.edu/systemic-risk-of-pandemic-via-novel-pathog...
angry_octet 7y ago
It is astounding how much D-K this piece exhibits. On what basis does he think that virus-transmissive mask wearing is effective? Absolutely none. (He doesn't mention equally plausible arguments about why it could be worse.) And then he goes on to mis-comprehend the parachute study, and predictably use it to justify doing not what experts recommend, but what non-experts want to believe. Also he really seems to mis-comprehend probability and confidence intervals -- like most people. A 29% chance of Trump winning should have been a five alarm alert that there was a significant chance of catastrophy -- even for Republicans. But we're too focused on the things that are very likely. (Like death, and as an American prelude, bankruptcy from health costs, being screwed by megacorps, taxes, and rich getting unfair tax breaks.) If car accidents weren't common and easily foreseeable, would we have insurance? Only if part of the system made us, based on risk analysis.
- ddxxdd 7y ago
  >On what basis does he think that virus-transmissive mask wearing is effective? His prior post has already proven this beyond a shadow of a doubt. https://slatestarcodex.com/2020/03/23/face-masks-much-more-than-you-wanted-to-know/ https://slatestarcodex.com/2020/03/23/face-masks-much-more-t... Perhaps that post is a prerequisite to this post.
  - angry_octet 7y ago
    Well, we have different concepts of doubt. Also, he's talking about proper procedure masks (ASTM Level 1), not paper or cloth or home made masks. These are hard to come by, and in any case should be reserved for high risk situations. There are no studies that I can find showing woven fibre or paper mask effectiveness, especially in regards to the risk increased contact transmission from touching the mask.
    - ddxxdd 7y ago
      >There are no studies that I can find showing woven fibre or paper mask effectiveness Oh, well you're in luck. I have just the thing for you: https://smartairfilters.com/en/blog/best-materials-make-diy-face-mask-virus/ https://smartairfilters.com/en/blog/best-materials-make-diy-... This image is the quick, all-encompassing money-shot: https://smartairfilters.com/wordpress/wp-content/uploads/2020/03/Mask-Materials-Effectiveness-1-Micron-EN-768x558.jpg https://smartairfilters.com/wordpress/wp-content/uploads/202... You can make the argument that homemade masks won't reach those levels of effectiveness because of a lack of a strong seal around the airways. And there have been no real-world case studies of cotton masks protecting people from getting sick. But there are plenty of arguments to be made that support the idea that if 100% of the population wore a cotton shirt around their face, then the pandemic would be strongly mitigated.
danbmil99 7y ago
Nate silver is a god. When everyone else was massaging their predictions to get them under 5% for Trump, he stuck to his guns. I have a strong suspicion that if we could run the counterfactual, 29% would be right on the money.
- disgruntledphd2 7y ago
  I am honestly more impressed by his original work, building multi-level models for Bush vs Obama. There was a kind of earnestness to his coverage that's been lost (inevitably) since he, and 538, became such a big brand.
- lonelappde 7y ago
  > I have a strong suspicion that if we could run the counterfactual, 29% would be right on the money. Conditioned on what? How would you model that? Go back and change the weather? Redo polls?
User23 7y ago
> The stock market is a giant coordinated attempt to predict the economy This statement is so obviously and absurdly wrong it makes it difficult to appreciate the rest of the article.
- nil-sec 7y ago
  Could you elaborate why you think that? It isn’t obvious to me why this is absurdly wrong.
  - User23 7y ago
    The stock market and the economy are observably decoupled. I’m a trader by profession and if I traded based on the real economy rather than, say, the Fed’s latest shenanigans, I’d be ruined in a heartbeat. Just this past month we’ve seen the most crippling blow to the real economy in over a century and a massive stock market rally.
    - nil-sec 7y ago
      Okay, but in this case a better assumption seems to be, the stock market is a function of the economy and political interventions. Or would you suggest this function does not depend on the economy at all?
      - User23 7y ago
        I wont't say that the stock market doesn't depend on the economy at all, because that's also obviously incorrect. A total economic collapse (think dollars stop being worth money) would obviously affect the stock market just to list a limit case. But on the margins, no, the economy doesn't move the stock market in any appreciable fashion in the time frame a trader operates. In the extremely long term though the health of US capital markets is a function of the size and broad diversification of the US economy. Operationally speaking, the market moves based on the actions of market participants and thus the primary intellectual challenge is predicting those actions. Since currently the Federal Reserve working in concert with the US Treasury is an active market participant with unlimited capitalization that is telegraphing its moves, that is what's moving the market. Literally nobody trades on the reality that thousands of small businesses are in danger of going bust. Personally I think it's awful, but I don't have the capital to fix it.
SergeAx 7y ago
IMO it was not about prediction per se, it was about prediction of magnitude. Will be Wuhan virus bad? Probably so. HOW BAD will it be? That's just another level of prediction. Remember that experiment with smoke in the room with actors sitting and chatting like this is totally fine? [0] Posting a tweet "I am getting ready to 3-6 months quarantine and advise everybody to do so as well" has much more internal pressure. [0] http://www.weirduniverse.net/blog/comments/the_smoke_filled_room http://www.weirduniverse.net/blog/comments/the_smoke_filled_...
- Smaug123 7y ago
  Indeed, Putanumonit used precisely that analogy in https://putanumonit.com/2020/02/27/seeing-the-smoke/ https://putanumonit.com/2020/02/27/seeing-the-smoke/ .
marvin 7y ago
I’m sorry to say it, but the author is wrong in his central point that this was not a failure of prediction. > For that matter, why didn’t you post this – on Facebook, on Twitter, on the comments here? You could have gone down in legend Well, I did. I started hedging against a stock market collapse on Feb. 19 and sold half my tax-exposed holdings on Feb. 28, incurring a really painful skin-in-the-game tax bill. In the first week of March, I made a very alarmist post on my personal social media profile, announcing to all my contacts a call to institute quarantines, work-from-home, reduce public transport use, face masks in public and outlaw crowds until control of the epidemic control was evident. This was after reading anonymous first-hand accounts from doctors in Lombardia, following the obvious exponential trend of spread in every country hit and watching China shut down the economy of a meteopolis. At this point, it was clear what would happen. Maybe not if you struggle to disregard opinions that don’t match the data, but that’s not the point. I never take these kinds of defensive or alarmist measures in times of peace. It was a one-time thing, and I felt like a complete freak for going against the consensus and sticking my head out among educated and respected friends. Had the worst FOMO of my life regarding the tax bill and stock market hedges. But I was right, I am not usually wrong and this is not a post-fact rationalization. This was predictable in early March, at the latest. Maybe the author is right that society would have been unable to heed these warnings, but that’s practically a tautology, given that we know how things played out in most Western countries. My family heeded my warnings, and stayed safe through the most critical phase when the disease was rampant but not obvious.
- empath75 7y ago
  Yeah, the grocery stores emptying out in Korea and Italy is weirdly what set off the ‘this shit is about to get real’ alarms for me. Sold stocks, stocked up on food, told everybody I knew to get ready for 1918.
- lonelappde 7y ago
  > My family heeded my warnings, and stayed safe through the most critical phase when the disease was rampant but not obvious. The vast majority of people "stayed safe" through the most critical phase, regardless of behavior. even a million infections in USA is statistically almost no one. It's an exponential process and the world is big, and still now, deep into the disaster, most people haven't developed symptoms.
throwaway_pdp09 7y ago
> Predicting the coronavirus was equally hard No, it's mainly not. The problem is people assume tomorrow is going to be like today and yesterday (for all problems, not just this one). Add to that that people don't like hearing unpleasant news and will tune it out. Net result: they ignore it and when pushed will start to deny it. You can add on top that rare newspaper editors that do recognise a risk will not want to piss off their readership by telling them the truth (or a or probably future occurrence) that they will get angry at. Give the people what they want, not what they need. For covid, I started stockpiling a bit of food etc. about a month before anyone else. My calculation was, this is likely to happen but may not. If things kick off I may need it. If they don't, I eat it anyway. There's no downside. (Incidentally I did the same ~2007 before the crash for the same reason. Turned out unnecessary then). What happened when I told people about covid usually follows 2 particular patterns 1) "No don't be stupid, it's not that bad" - simple denial. 2) they stare at you full-face on, totally expressionless for about 2 or 3 seconds, they turn their head away, exhale, turn back and start talking about something else as if what you'd said never happened - blanking it. It's a curious, quite consistent response when it happens. We get all the warning we need, the majority of people just don't want to know. QV. climate change, carrington events, epidemics (we had plenty of epidemiologists saying When not If), financial crashes. It's all there if you want to know.

223 points by barry-cotter 7y ago

I thought this was an excellent article, particularly the point about how and (possibly) why the CDC advised against the use of face masks to limit spread of infection during a pandemic. There seems to have been other failings of expert advice in the early days of the pandemic (e.g. see the UK government's apparent change of strategy early on) and they do to me look like failures in academic reasoning, rather than failures of political decision making. Will be interesting to read the findings of the inevitable inquiries after the dust has settled.

tonyedgecombe 7y ago
I find it hard to criticise the UK government on that U-turn. The facts changed and they reacted. Prior to this starting a flu-like pandemic was at the top of the governments risk register and they had models and a plan which they started following. There was some initial idiocy of course, like Boris shaking hands with everybody in a hospital he visited. They should probably be criticised more for their general preparedness and tight health care spending in the face of an ageing population.

zuhayeer 7y ago

‪“If you can buy an 80% chance of stopping a deadly pandemic for the cost of having to wear some silly cloth over your face, probably that’s a good deal”‬ The reason the CDC didn’t previously recommend face masks could’ve been because of a lack of RCT. But in order to do that, they would have had to expose subjects to disease without face masks which wouldn’t make sense. Goes to show there has to better ways for judging the viability of something without resorting to a strict binary of “do or do not”

SilasX 7y ago
That’s exactly the point he makes by bringing up the lack of RCTs regarding jumping out of a plane with/out a parachute.
pdonis 7y ago
> Goes to show there has to better ways for judging the viability of something without resorting to a strict binary of “do or do not” It's not just a question of how to judge but who makes the judgment. Part of the point of the article is that wearing a mask seems like something you should be able to conclude is a good idea based on common sense; you shouldn't need to wait for an expert to tell you, and if the expert is telling you something that seems to conflict with common sense, you shouldn't automatically disregard common sense.
- tomp 7y ago
  I find conflict of interest a much better indicator of truthfulness than common sense. In this case: doctors wearing masks conflicts with doctors recommend people don’t wear masks.
  - pdonis 7y ago
    Not necessarily. Doctors are exposed to a different set of risks while on the job than ordinary people going about their business. And common sense already tells you that it's a good idea to take precautions when working with sick people, so the fact that doctors wear masks while ordinary people going about their business don't have to should not come as a surprise. That's not to say that doctors (and others) can't have reasons other than "masks don't work" for recommending that people (other than other doctors) don't wear masks. Just that doctors wearing masks themselves, by itself, is not enough to know that their recommendation for ordinary people not to wear masks is unreliable. You need more information.
seemslegit 7y ago
Lack of RCT is a crutch (in this case), the practical reason is that as hard as it is to get face masks now it would have been much harder if the entire population was scrambling for them in the early days of the pandemic and given the US size, purchasing power and worldwide visibility it would have made it more difficult world-wide - all Western countries did the same, de-emphasized masks as they were imposing a lockdown and are now emphasizing them when planning for gradual release.
- williamstein 7y ago
  The article claims to refute your statement by pointing out that "The CDC has been singing the same tune for the past ten years. Swine flu, don’t wear masks. SARS, don’t wear masks. They’ve been really consistent on this point. But why?" This text in the article has several links.
  - seemslegit 7y ago
    https://slatestarcodex.com/blog_images/masks_cdc.png https://slatestarcodex.com/blog_images/masks_cdc.png Seems pretty consistent with my point - if masks weren't believed to be effective they would not be recommended "if unavoidable" , a use / don't use recommendation is not the same as a scientific finding of efficacy - it almost always factors in an implicit cost-benefit analysis of exactly the kind SSC advocates for. Case in point - all countries are now recommending (and in some places legally requiring) masks and not because a bunch of RCTs were conducted in the last two months.
Karrot_Kream 7y ago
I disagree. I think the CDC did actually perform a cross-benefit analysis on masks during the initial segment of the outbreak. Given the not-incontrovertible, yet positive, efficacy of masks, combined with the implications of a PPE shortage for healthcare workers meant that there probably was a calculated decision made to not recommend masks. Though at this point, without some more transparency, I'm not sure what we can say about the CDC's decisionmaking here.
- polemic 7y ago
  Potentially, but then surely a Cost/benefit analysis would strongly encourage the CDC to stockpile them in advance too? Perhaps nations need strategic PPE reserves.
- roenxi 7y ago
  > not sure what we can say about the CDC's decisionmaking here. We can say "gee, we should appoint more truthful bureaucrats". It isn't a crazy guess that (for this pandemic) a useful mask for a normal everyday person is anything that puts something in front of their face that makes it a little harder to breath and baffles the air currents. There may be a shortage of proven face masks but there is no shortage at all of things that could be sewn up to cover faces and deaden air currents. The cost is tiny vs the risk of it not working. Officials who are more worried about preventing a response to the potential crisis than the potential crisis should not be in charge if there is a potential crisis. We want people to change their behaviors if there is a crisis. Ideally even before the evidence in overwhelming that there is a problem.
  - kd5bjo 7y ago
    The cost of recommending masks is that a large minority of people will decide they need the best mask to protect themselves, and buy them up thus depriving the healthcare workers of them. Yes, in an ideal world, the entire situation would be spelled out and only those people who are truly at higher risk would acquire some of the limited supply of more-effective masks. For that to work, however, requires a sense of community and level of trust in government that has been in short supply in the US in recent years. These factors were also in play for swine flu and the other potential pandemics the article examined CDC rsponses for. In a crisis, officials have to base their decisions on current conditions, not on what they should ideally be. Restoring trust in government is extremely important, but can’t be accomplished in the timescale necessary to be useful for this crisis.
paganel 7y ago
Honest question, are there many people out there who find wearing a mask "silly"? Yes, they can be sometimes uncomfortable in terms of ease of breathing but to be honest I don't find them silly at all, they serve a purpose, just like an umbrella serves a purpose, it would be stupid of me to laugh of someone holding an umbrella while it's raining outside.
- DoofusOfDeath 7y ago
  I haven't noticed any of my acquaintances saying that mask-wearing is silly. But I have noticed many people not observing social distancing and stay-at-home requests. Perhaps if those people ignore the guidance because they're not convinced it's important, those same people would judge mask-wearing to be a waste of effort. Perhaps another issue is people who have heard that wearing cloth masks mostly prevents you from getting others sick, and some of them just can't be bothered to make small sacrifices for the speculative benefit of strangers?
- lonelappde 7y ago
  It's worse than "silly", it's "Oriental".
nullc 7y ago
One thing this analysis misses is the crystals. As in: The CDC is a huge government bureaucracy that employs (and cannot swiftly fire) enormous numbers of people from diverse backgrounds. If RCTs aren't the standard, well then why not also wear crystals to ward off the virus? They're inexpensive, and they couldn't hurt-- you can probably find a poorly controlled study to support it too. The CDC is not human, it is a meat-and-paper-political-machine. You cannot understand it's actions under the same framework you'd use to judge another person. The CDC must contain internal defenses against hokum which are much stronger than any person needs to personally have. So why does the CDC sometimes provide overly conservative advice even when doing so causes death? For the same reason the scorpion stings: it's in its nature. It's far from clear to me that a CDC which did make a more prudent facemask recommendation would actually be a better organization overall. Sure, it would have gotten this one right, but it would have likely given other bad advice as a result.
- SilasX 7y ago
  >As in: The CDC is a huge government bureaucracy that employs (and cannot swiftly fire) enormous numbers of people from diverse backgrounds. If RCTs aren't the standard, well then why not also wear crystals to ward off the virus? They're inexpensive, and they couldn't hurt-- you can probably find a poorly controlled study to support it too. Alexander's discussion of parachutes gave some good heuristics for when you should trust common sense vs require an RCT, which explains why his advice doesn't generalize to advocating crystals. The heuristic was basically that you should trust common sense when a) there's a solid scientific model of the key dynamics, b) the purported remedy is cheap, and c) the costs of not stopping the threat are disproportionately high. Crystals meet b)[1] and c) but fail a) -- we don't have a validated model of crytal/virus interactions. In contrast, we know that (some) viruses (including this one) spread through unimpeded airflow, and masks impede said airflow. We know that high-speed impacts between your body and the ground are deadly, and parachutes demonstrably slow the falling speed of objects. Thus, you should go with common sense in the absence of an RCT for parachutes and masks but not crystals. [1] They might fail b) depending on the particular charlatan!
  - nullc 7y ago

TheCowboy 7y ago

I like a lot of what is said in this piece. Background: I am currently supporting myself by betting on politics and economics. I was caught jobless when this thing hit, halting my attempt at a career switch into either coding or data science. At the end of February I posted my first forecast for 200k cases by March 20th, giving it a 96% chance. I feel like there exists the possibility of doing better, and there exist people who are already there but no one wants to listen or change. Watching this thing since the start of the year has been like watching a preventable slow motion crash, where many people seemed to be trying their hardest to intentionally fail. Let's not just blame the MSM or "elites" either. I speak to people who confidently assert everything is going to be fine in a year, and when pressed for a probability it's not even in top quartile. Even if it was 90% certain, a 10% chance of a significantly negative outcome is too high. I can open any thread on HN regarding economics and the general level of arrogant ignorance is staggering. We're on target to massively screw up the economic relief, which will make the economic stimulus phase all the more challenging. People need to be more predisposed to having possible intellectual and knowledge blind spots. But I'm not all negative, as I remain very optimistic that people will continue to ignore reasonable warnings.

starpilot 7y ago
Where do you bet for money to "support yourself"? Good Judgment Open doesn't have cash awards.
- graeme 7y ago
  Had they made that bet with out options in mid to late feb they could be a millionaire. For political bets you can bet on betting sites. Much harder to earn an income though.
  - tomp 7y ago
    Although market reaction is fundamentally hard to predict. It caughy many by surprise that markets reacted positively when Trump was elected. Or when there’s 150k people dead from coronavirus.
  - 7y ago

mehrdadn 7y ago

A bit unrelated, but I heard about this 2007 paper recently [1]: > The presence of a large reservoir of SARS-CoV-like viruses in horseshoe bats, together with the culture of eating exotic mammals in southern China, is a time bomb. I know the instant reaction is "people predict things all the time, some are bound to come out right", but this seems so specific that I think it's not just pure chance... we're just not heeding some of the warnings. ------ Now one reaction to the blog post: it does make a lot of good points (especially about masks), but for the part about shutting society at least, it doesn't seem to account for the follow-on effects of being wrong. If governments had shut things down due to a 10% chance of a pandemic, and it was prevented, chances are decent that a lot of not-similarly-reasoning people would've been angry. A lot of people, some for good reason and some for bad, just don't trust their governments (or the media) enough, and of those that do, a fair chunk are going under a lot of hardship to comply. So they don't just say "okay, I don't see it that way, but I trust you that this is for the best, even if it means I can't put bread on the table for my family, so I'll just live with the consequences and stay at home." Now I'm not really saying this is an excuse for governments/media to send poor messages, but it's an understandable human component... surely you can't attribute everything to irrational probabilistic reasoning? [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2176051/#__sec26title https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2176051/#__sec2...

meowface 7y ago
But if every government had acted much earlier, when the probability seemed to exceed 10%, there wouldn't have necessarily been any total shutdowns like what we're seeing now (except for the initial containment in China). There would still be some economic harm, but it'd be far, far less.
- mehrdadn 7y ago
  What would've been there instead of a shutdown and how could that have avoided a shutdown? (Specifically thinking of the US here.)
  - undersuit 7y ago

greendave 7y ago

It's nice to read a well-thought-out piece that both critiques and enlightens. That said, I do think it bears some mention that a substantial amount of the early Coronavirus discussion was being poisoned by people who were not arguing in good faith. That sucked a fair amount of the oxygen out of the debate, and I suspect may have left journalists in less of a position to provide thoughtful context around expert opinions. In other words, flat earthers occupying enough print and mindshare to make it hard to focus on the subtleties of Keplerian motion.

cma 7y ago
Sort of like how Trump’s China travel ban announcement was accompanied by a statement saying he wanted to close the Mexican border right away too but they decided against it. Almost to bait the media into accusing him of using the situation for his own ends.

mherdeg 7y ago

I was more or less convinced by the discussion in mid-February at https://virologydownunder.com/past-time-to-tell-the-public-it-will-probably-go-pandemic-and-we-should-all-prepare-now/ https://virologydownunder.com/past-time-to-tell-the-public-i... and https://virologydownunder.com/so-you-think-youve-about-to-be-in-a-pandemic/ https://virologydownunder.com/so-you-think-youve-about-to-be... that there was a reasonable chance of needing to take some modest steps to prepare for a pandemic. The predictions in these blog posts about pandemic best practices were in retrospect fairly accurate (see e.g. discussion under "Social distancing will be important but unpleasant") and have been a pretty good guide for the first month of major international response. We're kind of into uncharted waters now though, and I'm wondering who else will have accurate predictions of how the next year or two will go.

kd5bjo 7y ago
> We're kind of into uncharted waters now though, and I'm wondering who else will have accurate predictions of how the next year or two will go. There have been several out-of-control pandemics in human history that we can look at. It’s true that lots of things have changed between the Bubonic plague sweeping through Europe and now, but people are still people; there may be some insights there. You’d have to ask a historian (I’m not one). As different regions get this under control, I expect there to be health screenings required for international travel. In the old days, this was quarantine for inbound travellers, but adding a week or two to travel times is a lot less palatable now. We now have other, faster, ways of determining if someone is ill and I expect them to be deployed to protect areas that have succeeded.

projektfu 7y ago

Sometimes you need to answer the question, what would you pay for that lottery ticket? Sometimes you need to answer, would you pay Russian roulette with that gun? In this case if you had a 90% confidence that Coronavirus wouldn’t escape China or wouldn’t cause catastrophe if it did, then you still have to point that 10-chambered revolver at your head if you choose “don’t prepare”. But buying your way out of the game has a cost, too.

mrfusion 7y ago

Another benefit to thinking in probabilities is that you don’t get as attached to ideas. You don’t feel the need to defend your Opinion past it’s expiration date. People can have better discussions without getting offended.

lonelappde 7y ago
Otoh probabilities are hard to check for correctnessso you can weaael out of being wrong by making everything you say a motte-bailey deception (to use a slatestarcodex term).
- mrfusion 7y ago
  It’s ok to be wrong though.
- TeMPOraL 7y ago
  That's why it's important to be precise in describing what you mean (which may involve explaining the intended meaning when using a common word), so that you can't back-track on it later. Conversely, when you see someone not doing that, be prepared for the possibility of weaseling out.

deleted 7y ago

[deleted]

jonhohle 7y ago

A major issue, not just with the current pandemic, but all major network news programs, is they are not news, they are entertainment editorials. However, they don’t present themselves as such, and present information as if absolute and unbiased. If there were journalistic standards and not just first to print clickbait, there would have to be nearly as many corrections and retractions as there are news stories. The constitution guarantees freedom to publish, but it would be really nice into have some legal standard for accurate reporting of current events in good faith.

disgruntledphd2 7y ago
You could call it a fairness doctrine, or something ;) https://en.wikipedia.org/wiki/FCC_fairness_doctrine https://en.wikipedia.org/wiki/FCC_fairness_doctrine

joe_the_user 7y ago

"First, a bunch of generic smart people on Twitter who got things exactly right... 'Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, and Nicholas Christakis....'" And "...they didn’t beat the experts in epidemiology. Whatever probability of pandemic the experts and prediction markets gave for coronavirus getting really bad, these people didn’t necessarily give a higher probability. They were just better at probabilistic reasoning, so they had different reactions to the same number..." I noticed that. As a math person, things were exceptionally clear at a point, clearer than many uncertain events tbh. Simple exponential growth this, I can understand ... and so could a significant slice of people. But my question is: if epidemiologists aren't experts at THIS, wtf are they experts in? Isn't this exactly the "classic" circumstance any average epidemiologist should be a able to see in a heartbeat. Apparently not, so what do they train these people in? Question maybe half rhetorical but also serious. What's "expert knowledge" in these circumstances.

__s 7y ago
So, there's a basis for "Goofus" mentality: If you're trying to compile a set of facts, you very likely want a system which is incomplete, but correct. ie false positives have a significant cost to your models, where you essentially have a technical debt of needing to weed out false facts Policy should follow risk analysis, but research which is not time sensitive has compounding factors which favor building certainty before skipping ahead Think of it like software development: planning/testing/etc have high rewards throughout development, but sometimes you need to sell a rushed MVP
tlb 7y ago
Professions exist that effectively have negative knowledge about their field, in the sense that they do worse than smart generalists at making predictions. Closest to home, I'd say Software Engineering mostly fits this description. 90% of people who call themselves that are terrible at actually making software. Or, for example, most of the best fiction is not written by English Literature majors. Perhaps epidemiology is like that. Based on what I've seen in the last few months, I'm not impressed.
- 7y ago

thaumaturgy 7y ago

There's some shockingly bad thinking in this article and I'm surprised that people seem to be taking it at face value. Picking just a few things that stood out to me: > First, a bunch of generic smart people on Twitter who got things exactly right – there are too many of these people to name, but Scott Aaronson highlights “Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, and Nicholas Christakis.” This is pretty much the definition of cherry-picking. How many equally smart people got it wrong? Was there something in common among these particular people that led them to get it right, or did they hazard a guess on approximately the same information that everyone else had and then end up in the winning group? There's a fun article somewhere in my bookmarks that I can't find right now, but it covers the phenomenon of some bettor -- at the race track, or the stock market, or something -- who has this crazy winning streak and nobody can figure out why. Maybe it's this person's lucky hat, or maybe it's the notes they take, or maybe... who knows. But the most compelling reason is that they just happen to be the 1 member of a very large group who, through probability alone, keeps winning. This other article sort of mentions it in passing: > The prosecution also had an expert witness, an economist named David DeRosa. He didn’t share Heeb’s views about poker. DeRosa used a computer to simulate what might happen if 1,000 people each tossed a coin 10,000 times, assuming a certain outcome — such as tails — was equivalent to a win and the number of times a particular person won the toss was random. But the results were remarkably similar to those Heeb presented: A handful of people appeared to win consistently, and another group seemed to lose a large number of times. This wasn’t evidence that a coin toss involved skill, just that — much like the infinite number of monkeys typing — unlikely events can happen if we look at a large enough group. (https://www.discovermagazine.com/the-sciences/are-the-best-gamblers-skilled-or-just-lucky https://www.discovermagazine.com/the-sciences/are-the-best-g...) Back to the posted article... > parachutes, etc. etc. I mean, all kidding aside, there are enough parachute failures at this point to have a pretty good idea of the grim odds of surviving a fall from an aircraft without one. There are some really good reasons for conservativism in medicine. We have lost many, many lives to cures, treatments, and preventions that were "common sense" and only later discovered to be wrong. Or, sometimes, work, but only accidentally. Scurvy is the historical go-to example here. Its cause and treatment has been discovered, lost, and rediscovered many times, and each time, there was some common-sense treatment for it that was wrong. > Gallant wouldn’t have waited for proof. He would have checked prediction markets and asked top experts for probabilistic judgments. If he heard numbers like 10 or 20 percent, he would have done a cost-benefit analysis and found that putting some tough measures into place, like quarantine and social distancing, would be worthwhile if they had a 10 or 20 percent chance of averting catastrophe. Gallant will spend his life preparing for the next disaster that has a 20% chance of occurring and will eventually end up living in a small off-the-grid cabin deep in the woods, slowly being poisoned by the well water that isn't being tested correctly or often enough. There is an opportunity cost for preparing for disaster. I'm all for being at least a little bit prepared. My favorite hobby involves going out and retrieving people who were unprepared. But I never criticize them for getting into bad situations, because it can happen to any of us. (With the possible exception of people who don't evacuate from hurricane areas. Those people piss me off.) You can't prepare for everything. Most of us have never lived through a situation like we're living through now. How believable would this all have been a year ago? Who could believe that we'd be willing to stop the world economy because of a virus? The irritating nut of this is that this isn't necessarily something you can learn from. Okay, let's say you're one of the people that did the mental math and chose not to react -- a "Goofus", according to this article. So you learn from it, yeah? Next thing comes along with a 10% or 20% chance, you prepare for it. You're in California, so, earthquake: you fill up your garage with supplies. The earthquake never comes. Then there's a 10% or 20% chance of wildfire destroying your community... so you move, except the wildfire never hits that community. Armed robbery. Disease. Storms. All of these have some kind of low probability of happening, but will have super serious consequences if they do, so you -- remembering covid-19 -- prepare for each of them. Decades come and go, and eventually, you decide, hey, you're busy. You can't really deal with that thing right now. And besides, none of the others have happened. Except, this time the floods do come, and you're screwed again, and now some guy is writing about how all those people who didn't properly prepare for the flood had bad mental models for assessing risk. > A few weeks ago, I wrote a blog post on face masks. It reviewed the evidence and found that they probably helped prevent the spread of disease. Then it asked: how did the WHO, CDC, etc get this so wrong? Absent from the blog post he references is that there's uncertainty over whether face masks might actually increase the risk of infection among healthy populations of non-medical professionals. The thinking goes that some number of people don't know how to properly use the masks (which he covers) and that they end up touching their face more as a result, and that the masks that most people are using aren't great filters for this particular virus, so the net change in probability of getting infected is going up. But this isn't a certainty. There isn't enough data to know for sure if this is true or not. Even after this pandemic runs its course, there will probably still be disagreement in the studies to come because of all the confounding factors at play. It's a reasonable position though, and that's why people keep getting conflicting information on this. You can get some treatment of the debate and confusion around this from this discussion: https://old.reddit.com/r/COVID19/comments/fcwcn7/to_mask_or_not_to_mask_the_seemingly_endless/ https://old.reddit.com/r/COVID19/comments/fcwcn7/to_mask_or_... ...people are posting studies supporting one position, then rebutting them, then offering professional advice in one direction, and then others in another. It's a mess, and while that thread isn't rigorously scientific, essentially the same argument is happening at and between the CDC and WHO and others.

ilkan 7y ago

IMHO, the journalists' mistake was lazily relying on easily accessible Western orgs instead of partnering with Eastern-based journalists to examine stories coming out of China. I've been following the South China Morning Post since the first week of January and so far all the observations coming from their experts has been refuted by the CDC first, then confirmed a month or two later. They are just as smart, highly motivated and had at least 10x more casework to examine. Tldr; if you want to make better predictions, compare expert opinion against news closer to the source.

armitron 7y ago

Not even a mention for Taleb who had a paper out in January [1] yet paragraphs for Zeynep and the rest who merely parroted rising consensus? Weak, very weak. [1] https://necsi.edu/systemic-risk-of-pandemic-via-novel-pathogens-coronavirus-a-note https://necsi.edu/systemic-risk-of-pandemic-via-novel-pathog...

angry_octet 7y ago

It is astounding how much D-K this piece exhibits. On what basis does he think that virus-transmissive mask wearing is effective? Absolutely none. (He doesn't mention equally plausible arguments about why it could be worse.) And then he goes on to mis-comprehend the parachute study, and predictably use it to justify doing not what experts recommend, but what non-experts want to believe. Also he really seems to mis-comprehend probability and confidence intervals -- like most people. A 29% chance of Trump winning should have been a five alarm alert that there was a significant chance of catastrophy -- even for Republicans. But we're too focused on the things that are very likely. (Like death, and as an American prelude, bankruptcy from health costs, being screwed by megacorps, taxes, and rich getting unfair tax breaks.) If car accidents weren't common and easily foreseeable, would we have insurance? Only if part of the system made us, based on risk analysis.

ddxxdd 7y ago
>On what basis does he think that virus-transmissive mask wearing is effective? His prior post has already proven this beyond a shadow of a doubt. https://slatestarcodex.com/2020/03/23/face-masks-much-more-than-you-wanted-to-know/ https://slatestarcodex.com/2020/03/23/face-masks-much-more-t... Perhaps that post is a prerequisite to this post.
- angry_octet 7y ago
  Well, we have different concepts of doubt. Also, he's talking about proper procedure masks (ASTM Level 1), not paper or cloth or home made masks. These are hard to come by, and in any case should be reserved for high risk situations. There are no studies that I can find showing woven fibre or paper mask effectiveness, especially in regards to the risk increased contact transmission from touching the mask.
  - ddxxdd 7y ago
    >There are no studies that I can find showing woven fibre or paper mask effectiveness Oh, well you're in luck. I have just the thing for you: https://smartairfilters.com/en/blog/best-materials-make-diy-face-mask-virus/ https://smartairfilters.com/en/blog/best-materials-make-diy-... This image is the quick, all-encompassing money-shot: https://smartairfilters.com/wordpress/wp-content/uploads/2020/03/Mask-Materials-Effectiveness-1-Micron-EN-768x558.jpg https://smartairfilters.com/wordpress/wp-content/uploads/202... You can make the argument that homemade masks won't reach those levels of effectiveness because of a lack of a strong seal around the airways. And there have been no real-world case studies of cotton masks protecting people from getting sick. But there are plenty of arguments to be made that support the idea that if 100% of the population wore a cotton shirt around their face, then the pandemic would be strongly mitigated.

danbmil99 7y ago

Nate silver is a god. When everyone else was massaging their predictions to get them under 5% for Trump, he stuck to his guns. I have a strong suspicion that if we could run the counterfactual, 29% would be right on the money.

disgruntledphd2 7y ago
I am honestly more impressed by his original work, building multi-level models for Bush vs Obama. There was a kind of earnestness to his coverage that's been lost (inevitably) since he, and 538, became such a big brand.
lonelappde 7y ago
> I have a strong suspicion that if we could run the counterfactual, 29% would be right on the money. Conditioned on what? How would you model that? Go back and change the weather? Redo polls?

User23 7y ago

> The stock market is a giant coordinated attempt to predict the economy This statement is so obviously and absurdly wrong it makes it difficult to appreciate the rest of the article.

nil-sec 7y ago
Could you elaborate why you think that? It isn’t obvious to me why this is absurdly wrong.
- User23 7y ago
  The stock market and the economy are observably decoupled. I’m a trader by profession and if I traded based on the real economy rather than, say, the Fed’s latest shenanigans, I’d be ruined in a heartbeat. Just this past month we’ve seen the most crippling blow to the real economy in over a century and a massive stock market rally.
  - nil-sec 7y ago
    Okay, but in this case a better assumption seems to be, the stock market is a function of the economy and political interventions. Or would you suggest this function does not depend on the economy at all?
    - User23 7y ago
      I wont't say that the stock market doesn't depend on the economy at all, because that's also obviously incorrect. A total economic collapse (think dollars stop being worth money) would obviously affect the stock market just to list a limit case. But on the margins, no, the economy doesn't move the stock market in any appreciable fashion in the time frame a trader operates. In the extremely long term though the health of US capital markets is a function of the size and broad diversification of the US economy. Operationally speaking, the market moves based on the actions of market participants and thus the primary intellectual challenge is predicting those actions. Since currently the Federal Reserve working in concert with the US Treasury is an active market participant with unlimited capitalization that is telegraphing its moves, that is what's moving the market. Literally nobody trades on the reality that thousands of small businesses are in danger of going bust. Personally I think it's awful, but I don't have the capital to fix it.

SergeAx 7y ago

IMO it was not about prediction per se, it was about prediction of magnitude. Will be Wuhan virus bad? Probably so. HOW BAD will it be? That's just another level of prediction. Remember that experiment with smoke in the room with actors sitting and chatting like this is totally fine? [0] Posting a tweet "I am getting ready to 3-6 months quarantine and advise everybody to do so as well" has much more internal pressure. [0] http://www.weirduniverse.net/blog/comments/the_smoke_filled_room http://www.weirduniverse.net/blog/comments/the_smoke_filled_...

Smaug123 7y ago
Indeed, Putanumonit used precisely that analogy in https://putanumonit.com/2020/02/27/seeing-the-smoke/ https://putanumonit.com/2020/02/27/seeing-the-smoke/ .

marvin 7y ago

I’m sorry to say it, but the author is wrong in his central point that this was not a failure of prediction. > For that matter, why didn’t you post this – on Facebook, on Twitter, on the comments here? You could have gone down in legend Well, I did. I started hedging against a stock market collapse on Feb. 19 and sold half my tax-exposed holdings on Feb. 28, incurring a really painful skin-in-the-game tax bill. In the first week of March, I made a very alarmist post on my personal social media profile, announcing to all my contacts a call to institute quarantines, work-from-home, reduce public transport use, face masks in public and outlaw crowds until control of the epidemic control was evident. This was after reading anonymous first-hand accounts from doctors in Lombardia, following the obvious exponential trend of spread in every country hit and watching China shut down the economy of a meteopolis. At this point, it was clear what would happen. Maybe not if you struggle to disregard opinions that don’t match the data, but that’s not the point. I never take these kinds of defensive or alarmist measures in times of peace. It was a one-time thing, and I felt like a complete freak for going against the consensus and sticking my head out among educated and respected friends. Had the worst FOMO of my life regarding the tax bill and stock market hedges. But I was right, I am not usually wrong and this is not a post-fact rationalization. This was predictable in early March, at the latest. Maybe the author is right that society would have been unable to heed these warnings, but that’s practically a tautology, given that we know how things played out in most Western countries. My family heeded my warnings, and stayed safe through the most critical phase when the disease was rampant but not obvious.

empath75 7y ago
Yeah, the grocery stores emptying out in Korea and Italy is weirdly what set off the ‘this shit is about to get real’ alarms for me. Sold stocks, stocked up on food, told everybody I knew to get ready for 1918.
lonelappde 7y ago
> My family heeded my warnings, and stayed safe through the most critical phase when the disease was rampant but not obvious. The vast majority of people "stayed safe" through the most critical phase, regardless of behavior. even a million infections in USA is statistically almost no one. It's an exponential process and the world is big, and still now, deep into the disaster, most people haven't developed symptoms.

throwaway_pdp09 7y ago

> Predicting the coronavirus was equally hard No, it's mainly not. The problem is people assume tomorrow is going to be like today and yesterday (for all problems, not just this one). Add to that that people don't like hearing unpleasant news and will tune it out. Net result: they ignore it and when pushed will start to deny it. You can add on top that rare newspaper editors that do recognise a risk will not want to piss off their readership by telling them the truth (or a or probably future occurrence) that they will get angry at. Give the people what they want, not what they need. For covid, I started stockpiling a bit of food etc. about a month before anyone else. My calculation was, this is likely to happen but may not. If things kick off I may need it. If they don't, I eat it anyway. There's no downside. (Incidentally I did the same ~2007 before the crash for the same reason. Turned out unnecessary then). What happened when I told people about covid usually follows 2 particular patterns 1) "No don't be stupid, it's not that bad" - simple denial. 2) they stare at you full-face on, totally expressionless for about 2 or 3 seconds, they turn their head away, exhale, turn back and start talking about something else as if what you'd said never happened - blanking it. It's a curious, quite consistent response when it happens. We get all the warning we need, the majority of people just don't want to know. QV. climate change, carrington events, epidemics (we had plenty of epidemiologists saying When not If), financial crashes. It's all there if you want to know.