16 ms·

This study doesn't appear to suffer from the same methodology problems as the Stanford study of a few days ago. In that study, they recruited people through Fac

by ccleve 6y ago

This study doesn't appear to suffer from the same methodology problems as the Stanford study of a few days ago. In that study, they recruited people through Facebook, and reported an infection rate that was low enough that it could have been caused by false positives. Here, it's closer to a random sample, but more importantly it shows really high rates. Those rates overwhelm any error due to false positives. Also, it shows numbers that are in line with our intuition. It shows higher infections in NYC, and higher infections among blacks. That reflects what the hospitals are seeing. This study may well be really good news.

3fe9a03ccd14ca5 6y ago
What's weird is that the Stanford study was suggested to have exaggerated the results, since those who suspected themselves as having the virus at one time were more likely to volunteer (allegedly). However, those results were nowhere near as big as this (I think like 2%-3% versus the 20%!)
- jdm2212 6y ago
  Roughly 1/1000 New Yorkers have died of coronavirus. At a 1% IFR, that suggests at least 10% of the state has gotten the virus. Probably more, because deaths are undercounted and many people who will die of the virus already have it but haven't died yet. In the Bay Area, almost no one has died of coronavirus so the infection rate should be next to nil.
  - brandmeyer 6y ago
    I was astonished by your 1/1000 figure and had to look it up... If anything, its too low. 15k deaths (proven+probable) in 8M people is almost 2/1000. https://www1.nyc.gov/site/doh/covid/covid-19-data.page https://www1.nyc.gov/site/doh/covid/covid-19-data.page
    - jdm2212 6y ago
      I was referring to the state -- but, yeah, it's even worse in the city :( and 2/1000 people * 1% IFR => 20%
    - chasd00 6y ago
      that's crazy to me, what are the demographics of NYC? Is the population more susceptible to dying of Covid19 somehow? I still can't get my head around why NY has suffered so disproportionately.
      - WrtCdEvrydy 6y ago
        It's a small state with a lot of people packed next to each other. Lots of people use public transit... subways are less than six feet apart.
        nicoburns 6y ago
        If the NY subway is anything like London, people will be less than 6 inches apart at rush hour. I imagine that will have played a big part in the spread in both of those cities.
        credit_guy 6y ago
        Yes, the subway is the correct answer. That's how this virus spread like wildfire in NYC. Second reason could be the elevators in the tall buildings (there are plenty of those).
        rocha 6y ago
        A counter argument is that the same didn't happen (or hasn't happened yet) in cities that also have massive public transportation systems like Tokyo or Seoul. There is probably many confounding factors, such as mask use, no talking on the subway, etc, to make pointing to one particular factor very hard. EDIT: typo
        6y ago
- rubidium 6y ago
  But we already know the actual infection rate is higher in NYC than CA. The main point that the antibody tests help uncover is death rate.
- bhouston 6y ago
  NYC and standord are markably different scenarios. But in each case the percentage of baseline infection to deaths is similarish, even though the percent compared to total pop is quite different.
  - zucker42 6y ago
    Nope, the Stanford study had a much lower prediction of the infections to deaths ratio, but that's because its analysis of the data was just wrong.
- TheBlight 6y ago
  What's interesting is if you look at the rest of NY numbers (-NYC,LI,Westchester). That has infection at 3.6%. Santa Clara was estimated to be 3% and LA county 4%. So it's in strong agreement with those areas.
  - kgwgk 6y ago
    But the number of deaths does not agree.
    - TheBlight 6y ago
      Can you elaborate?
      - kgwgk 6y ago
        Percentage of people with antibodies is similar (if you take those studies at face value). Percentage of people dead is not.
    - toufka 6y ago
      There also appears to be a difference in the strains. The NYC strain seems to mostly come from the European branch, while the CA infection comes directly from China (and likely much earlier in 2020). There appears to be some difference in virulence between the strains. https://www.medrxiv.org/content/10.1101/2020.04.08.20056929v2 https://www.medrxiv.org/content/10.1101/2020.04.08.20056929v... and https://nextstrain.org/ncov/global https://nextstrain.org/ncov/global
      - kgwgk 6y ago
        I’ve never looked into that (I know of the hypothesis, though) but IFR in South Korea also seems to be around 1% (or higher), not 0.1%
- cma 6y ago
  Exaggerated in that the California study predicted a flu like IFR (Ioannidis, one of the authors, has been pushing that as his pet theory for over a month). This shows it is more likely several times that.
  - glofish 6y ago
    it doesn't contradic Ioannidis at all. He might be right in the end, and that would be an incredibly bitter pill to many. We are getting closer and closer to the flu fatality rate. from 5% to 3%, then 1% now 0.5% - smart money (and common sense) would bet that the rate will continue to drop, I would expect that people living in healthier environment than NYC will fare even better. No way NYC IFR is the upper bound for the rest of the country. You could just as well expect it to be 10x higher than other places. the flu rate is 0.1%, thus we already hit the order of magnitude.
    - jdm2212 6y ago
      NYC is a younger, healthier city than the national average. NYC also has one of the best medical systems not just in America, but in the world. Roughly 0.2% of everyone in NYC has already died of COVID-19. So 0.2% is pretty close to a hard lower bound on the IFR for COVID-19. And I don't think anyone serious suggests every NYC resident has had COVID. I don't think anyone seriously suggests even half of NYC has had COVID.
      - glofish 6y ago
        All I am saying people there breath the NYC air every single day. Can't possibly be good for them, especially in the light of some chronic pulmonary inflammation induced deaths. The point on the lower bound of 0.2% is informative. I did not know that.
        timr 6y ago
        That lower-bound argument is overly simplistic. There are huge error bars on that ratio, because "the population of NYC" is not something easily defined, and the death count (at this time) includes a lot of "excess deaths" that almost certainly have nothing to do with the virus (e.g. untreated cardiac arrest).
dwaltrip 6y ago
I agree with everything you are saying. This does look like good news. However, we should remember that this is probably a mild overestimate, as the study population was assembled from those who were out and about (shopping at the grocery store, etc). People who are more strictly staying home -- and thus less likely to have been infected -- wouldn't be included in the survey.
- bob33212 6y ago
  Another way to look at it is that 20% of people most likely to contribute to a high transmission rate already have had it, in turn flattening the curve by a considerable amount
cjhopman 6y ago
Sure, but a person who goes to the store 7 times a week is ~7 times as likely to have been sampled compared to someone who goes only once a week. They are also ~7 times as likely to have contracted covid-19 at the grocery store. It's likely that they are also, in general, isolating less than the other. This sampling would have a (possibly slight, but still unknown) bias towards people who are isolating less (who you would expect to then have a higher than average percent positive for antibodies, assuming isolating is helpful). I don't think anyone knows whether or not this is a more representative sample than the stanford study.
- WrtCdEvrydy 6y ago
  Unfortunately, unless you go door to door knocking... you're gonna get biased data.
  - ineedasername 6y ago
    Or truly random sampling that includes appropriate proportions of sub groups, e.g, socioeconomic status etc. Otherwise you have to know on all of the doors. But it's still a lot: for a 95% confidence level and 2% margin of error, for NYC you would need to test about 2,400 people. Assuming a response rate of 10%, you need to knock on 24,000 doors.
    - austingulati 6y ago
      Wouldn't the ability to refuse taking the test introduce bias? i.e. the 10% that do take it still may not be representative of the entire population. Likely better than the grocery store sample, but still not ideal
      - qqqwerty 6y ago
        Yeah, that is still an issue. You can offer compensation to incentivize participation. And you can pre-select your sample (instead of random door-to-door) and require multiple follow ups with the selected persons to reduce nonresponse bias. You can also do a separate phone survey in addition to surveying the test participants. Questions like "Do you think you have had COVID?" and "How many times per week do you leave the house?". If the responses for the test participants vary significantly from the phone survey participants, you can try and weight your data accordingly.
vkou 6y ago
The major methodology problem with the Stanford study was not that they recruited participants through Facebook. The major problem in the Stanford study was that they ran a test that has a 3% false positive rate, and found that 3% of the test-takers tested positive. (And that apparently the asymptomatic COVID-19 rate is 90+% - which does not square with the Diamond princess data). They could have ran that same study back in 2018, and would have gotten the exact same garbage results.
zucker42 6y ago
Is there a write-up for this study? Or anywhere with data/methodology released? Also, is this particularly good news? Using the raw numbers from the headline and the deaths from here https://www1.nyc.gov/site/doh/covid/covid-19-data.page https://www1.nyc.gov/site/doh/covid/covid-19-data.page we can make coarse fatality rate estimates. I assume population of NYC is 8,400,000. 10290 / (.212 * 8.4e6) = 0.6% Including probable deaths (which makes the death count line up more closely with excess deaths) 15411 / (.212 * 8.4e6) = 0.9%. So it seems consistent with a IFR of 0.5%-1%. However, this doesn't account for the lag between antibody presence and death. Also, we don't know what bias sampling from grocery stores introduces, it could affect the results in either direction. Hopefully, New York releases a paper so we can get more accurate estimates. It's good to see a prevalence study with presumably less flaws.
tunesmith 6y ago
Sorry, can someone spell out how this might be good news? 21% is still a long way from herd immunity, and NYC's hospital system has been severely strained getting to this point. On top of that, the hospitalization rate still seems disproportionate to someone getting the flu - maybe it's five times as fatal, but it's > 5x the flu hospitalization rate. It's not like the virus has become less dangerous, we're just realizing how dangerous it has been, what with the impact we've already experienced. Besides, generally if a virus is less fatal than previously expected, it means it's more contagious, meaning that much harder to get to herd immunity. Good news would be things like: evidence the virus has mutated into something less severe; evidence of an anti-viral treatment that improves outcomes for everyone so it's not as big a deal to catch it; evidence that community spread has halted in an area and the boundaries are controlled so people in that area can feel secure they won't catch it; evidence of an impending vaccine. Is this good news just because we're finally establishing that people have caught it once can't re-catch it for now? I guess I can see that as good news but that is so expected that it's more like it would be horrendously bad news if we found evidence that recovered people didn't have antibodies. But generally I don't really see what policy impact this has, other than identifying a pool of workers that can go work in meat-packing factories without fear of catching it again.
- cm2187 6y ago
  I don't know if the hospitalisation rate is so out of whack vs the flu. It should be higher. But does 20% of the population catches the flu in a matter of weeks?
  - creaghpatr 6y ago
    Much of the population gets a flu vaccine so would be tricky to compare.
    - cm2187 6y ago
      My very point. It contaminates people at a slower pace, for that reason and cross immunity, higher R0, etc.
  - 6y ago
ineedasername 6y ago
I couldn't find any detailed write up about their selection methods for participants, only the basics of where they found them. Without that, it is very hard to determine whether or not there's some flavor of selection bias: If it is voluntary, and they do not screen out people who report having had cold/flu symptoms, then they run the risk of attracting a disproportionate number of people who volunteer because they're curious if their prior symptoms were actually coronavirus. That would make it far from a random sample. The fact that they sampled only people actually leaving their house is also a form of selection bias: these are the people more likely to be exposed and may represent a disproportionately high infection rate as well. So I await further information. That said, even if it's overall 0.6%, that is still 6x higher than flu, and higher than H1N1 which had a CFR around 0.1 for areas with adequate health care. (much higher when there was not adequate care, but that is not dissimilar to Covid.) So no matter what, no one should be walking away from this study saying "See it's no big deal! Just the Flu/Cold etc!"
- Retric 6y ago
  Flu is generally well under 0.1% of those infected on average. If you’re comparing them you want to either include or exclude asymptotic people from both populations. “Symptomatic Illnesses” https://www.cdc.gov/flu/about/burden/index.html https://www.cdc.gov/flu/about/burden/index.html As to NYC deaths, many people currently infected will die in the future. You can make various estimates to account for this but a reasonable first approximation is to double current deaths based on NYC’s infection curve vs South Korea’s.
  - greendave 6y ago
    As an asymptotic* person, I resent your hyperbola. (Sorry, couldn't resist).
    - deleted 6y ago
      [deleted]
  - ineedasername 6y ago
    Flu rates are based on all known knowledge, population testing, etc. We don't have that for covid-19. If you want a good apple-to-apples comparison of lethality of another pandemic, you need to find CFR numbers that were available during the pandemic. That is what I provided in my post. Here is the source I got them from [0] which gives the CFR at the 10-week mark for H1N1, somewhat similar to where we are now. Here's the relevant quote: >"The overall case fatality rate as of 16 July 2009 (10 weeks after the first international alert) with pandemic H1N1 influenza varied from 0.1% to 5.1% depending on the country." [0] https://www.cebm.net/covid-19/global-covid-19-case-fatality-rates/ https://www.cebm.net/covid-19/global-covid-19-case-fatality-...
credit_guy 6y ago
Yes, the study is really good news for NYC. My thinking is that the state of emergency, currently in place until 5/15, will be extended at most once to 6/15. After that we'll be somewhat open for business. I imagine everyone will be advised to wear mask/gloves, to keep the social distance and to keep washing hands. Lots of people will continue to work from home, others will bike to work. Being summer, the virus will not be that contagious. When the November rebound is forecast to happen, the herd immunity will be much higher, hospitals better prepared, the medical world will have learned the most effective ways to treat this disease, so the fatality rate will go down to levels seen in Germany, or lower. Bottom line, I don't think we'll have another shelter-in-place in November-December.
- robocat 6y ago
  > Being summer, the virus will not be that contagious. I think that is still scientifically debatable, good overview: https://www.newscientist.com/article/2239380-will-the-spread-of-covid-19-be-affected-by-changing-seasons/ https://www.newscientist.com/article/2239380-will-the-spread... Certainly it is still transmissible in hot countries.
  - ethbro 6y ago
    It seems reasonable that lipid encapsulated viruses are fairly stable in a variety of temperature ranges, no? On the other hand, I can see the additional environmental UV saturation and vitamin D having a noticeable effect. Even a small decrease in environmental persistence should help, given its infectiousness.
  - credit_guy 6y ago
    Thanks for the link. It's a good read. That said, science is enamored with significance levels, p-values, etc. Most people just do that because that's what they've learned, and think that's how it should be done. They don't think it comes from Bayesian inference, where you put a very low prior probability of a drug being useful (because the very vast majority of chemicals we could put in our bodies are either harmful, or have harmful side-effects). In this case, the Bayesian prior is that common colds in general become less prevalent during the Summer, and Covid19 is caused by a coronavirus, which is related to viruses causing the common cold. The burden of proof should be lowered many, many times, but I nobody who did these statistical surveys did that, because this is a total no-no in the field. So, I'll state my conclusion, after reading your link: the infectiousness of Covid19 is more likely than not to decrease in the Summer, based on the studies that were performed. It will not decrease to zero, so that does not contradict your statement that it is transmissible in hot countries.
godelski 6y ago
Why does recruiting through Facebook invalidate the results? If it is a representative sample it is a representative sample regardless of how people were recruited.
- alteria 6y ago
  One critique was that the segment of the population who would respond to the ad and actually test may be more likely to have experienced COVID-like symptoms.
  - chockablock 6y ago
    Also people reported sharing the Santa Clara link with others who might want the test (due to having had symptoms). In principle you could have the same problem with this survey: call your buddy and say “come on down to Costco—they’re doing free antibody tests”. The obvious fix is to not tell people the results of their own tests. Not sure of the ethics/consequences of that approach.
    - deleted 6y ago
      [deleted]
- zucker42 6y ago
  The Facebook ad essentially said "get tested for COVID-19 here", so very plausibly infected people could be more likely to respond.
- dragonwriter 6y ago
  > Why does recruiting through Facebook invalidate the results? If Because “people who both use Facebook and don't automatically discount every ad or other solicitation on Facebook not from someone they personally know, especially if it invokes a major news story, because of the risk of it being a scam looking to steal personal information or do something similarly nefarious” are not representative of “people”.
  - 6y ago
eanzenberg 6y ago
This study is good news... I'd be interested to project the rate of infections WITHOUT shutting down the economy for what's now going on 6 weeks. Seems like it was all in vain.
8ytecoder 6y ago
(Reposting my comment again here) This one's from a different company (BioMedomics) and it was a random test of people pulled straight off the streets. Happened at an entirely different geographical area (Chelsea, Massachusetts) as well. I don't want to be too optimistic but there are some signs that we are heavily under-counting the actual number of cases (at least in the US). https://www.bostonglobe.com/2020/04/17/business/nearly-third-200-blood-samples-taken-chelsea-show-exposure-coronavirus/ https://www.bostonglobe.com/2020/04/17/business/nearly-third...
mlthoughts2018 6y ago
The rates being this high actually casts doubt on the study. It’s very very implausible, and suggests almost surely a selection bias in the sample towards a population much more likely to have contracted it.
- mlthoughts2018 6y ago
  Actually in reading closer and seeing the data is collected at big box & grocery stores, it’s almost surely very biased. This cohort would skew younger and wealthier, which correlates with better preexisting health and fewer risk factors, and would exclude populations who systematically left the city, or who have known heightened risk factors. Comparing the death rate overall with the infection rate of this skewed sample would be likely to greatly underestimate the actual death rate of the virus.
  - Gibbon1 6y ago
    This weekend San Francisco plans to do a hard test of about 6000 people in a couple of square blocks of the Mission district. They are going door to door to encourage people to get tested. Still biased but probably better than anything so far.
    - mlyle 6y ago
      Yup.. though it's unclear how much we'll clear the test sensitivity hurdle by.
      - Gibbon1 6y ago
        I wonder about that. They are trying to test a neighborhood they think is most effected. One wonders if it's possible to follow up and retest positives to exclude false one. Suspect the Bay Area is moving towards what Korea is doing. Intense testing and contact tracing. Friend who works at a biomedical corp says scuttlebutt is they're separating their workforce into groups based on critical need and how well they can work remotely. Workers like him not expected to return to work onsite until next year.
dboreham 6y ago
Only 60k more deaths to go until herd immunity.
dchichkov 6y ago
What is the percentage of people with innate immunity to COVID-19?
Symmetry 6y ago
And the specificity of the test might be worse than advertised but it wouldn't be credible that it could be making a big different with the 20% positive rate here.
smartmic 6y ago
What I learned from an article of a major German newspaper is that in order to really estimate the quality of testing, two criteria are important: sensitivity and specifity of the applied test. If the real infection rate is still low throughout the population, a random sampling will not lead to reliable results (you could also toss a coin). Although the article is in German, it is worth to have a look on the graphic in the middle of the page, it should be understandable. https://www.sueddeutsche.de/gesundheit/corona-test-antikoerper-1.4883631 https://www.sueddeutsche.de/gesundheit/corona-test-antikoerp...
hsitz 6y ago
> Here, it's closer to a random sample, but more importantly it shows really high rates. Those rates overwhelm any error due to false positives. I'm not seeing that, at least not from what I've seen. Whether false positives skew results significantly is highly dependent on how accurate the antibody test they used is (in addition to how large a subset of the population is positive). This guy has an interesting visual tool that helps you to see how much a study could be affected. [1] Also, he says "Here's an interesting relationship. When a test with 95% sensitivity and 95% specificity is applied to a population with <5% prevalence of disease, MOST of the patients with positive tests are FALSE POSITIVEs." I.e., positive rate shows up as greater than 10%, when it's actually less than 5%. Do we know the sensitivity and specificity accuracies for the antibody tests used in NY? You'd like to think they know what they're doing. But this Bayesian stuff can be tricky, especially if they're rushing something through (esp. regarding testing of accuracy of the antibody tests themselves). And the California studies, although they seem to have some competent people behind them, seem to inflate/exaggerate the lower bound of uncertainty in their projection. [2] [1] https://sites.google.com/view/tgmteststat/home https://sites.google.com/view/tgmteststat/home [2] https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaws-in-stanford-study-of-coronavirus-prevalence/ https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaw...

by ccleve 6y ago

3fe9a03ccd14ca5 6y ago

What's weird is that the Stanford study was suggested to have exaggerated the results, since those who suspected themselves as having the virus at one time were more likely to volunteer (allegedly). However, those results were nowhere near as big as this (I think like 2%-3% versus the 20%!)

jdm2212 6y ago
Roughly 1/1000 New Yorkers have died of coronavirus. At a 1% IFR, that suggests at least 10% of the state has gotten the virus. Probably more, because deaths are undercounted and many people who will die of the virus already have it but haven't died yet. In the Bay Area, almost no one has died of coronavirus so the infection rate should be next to nil.
- brandmeyer 6y ago
  I was astonished by your 1/1000 figure and had to look it up... If anything, its too low. 15k deaths (proven+probable) in 8M people is almost 2/1000. https://www1.nyc.gov/site/doh/covid/covid-19-data.page https://www1.nyc.gov/site/doh/covid/covid-19-data.page
  - jdm2212 6y ago
    I was referring to the state -- but, yeah, it's even worse in the city :( and 2/1000 people * 1% IFR => 20%
  - chasd00 6y ago
    that's crazy to me, what are the demographics of NYC? Is the population more susceptible to dying of Covid19 somehow? I still can't get my head around why NY has suffered so disproportionately.
    - WrtCdEvrydy 6y ago
      It's a small state with a lot of people packed next to each other. Lots of people use public transit... subways are less than six feet apart.
      - nicoburns 6y ago
        If the NY subway is anything like London, people will be less than 6 inches apart at rush hour. I imagine that will have played a big part in the spread in both of those cities.
        credit_guy 6y ago
        Yes, the subway is the correct answer. That's how this virus spread like wildfire in NYC. Second reason could be the elevators in the tall buildings (there are plenty of those).
        rocha 6y ago
        A counter argument is that the same didn't happen (or hasn't happened yet) in cities that also have massive public transportation systems like Tokyo or Seoul. There is probably many confounding factors, such as mask use, no talking on the subway, etc, to make pointing to one particular factor very hard. EDIT: typo
        6y ago
rubidium 6y ago
But we already know the actual infection rate is higher in NYC than CA. The main point that the antibody tests help uncover is death rate.
bhouston 6y ago
NYC and standord are markably different scenarios. But in each case the percentage of baseline infection to deaths is similarish, even though the percent compared to total pop is quite different.
- zucker42 6y ago
  Nope, the Stanford study had a much lower prediction of the infections to deaths ratio, but that's because its analysis of the data was just wrong.
TheBlight 6y ago
What's interesting is if you look at the rest of NY numbers (-NYC,LI,Westchester). That has infection at 3.6%. Santa Clara was estimated to be 3% and LA county 4%. So it's in strong agreement with those areas.
- kgwgk 6y ago
  But the number of deaths does not agree.
  - TheBlight 6y ago
    Can you elaborate?
    - kgwgk 6y ago
      Percentage of people with antibodies is similar (if you take those studies at face value). Percentage of people dead is not.
  - toufka 6y ago
    There also appears to be a difference in the strains. The NYC strain seems to mostly come from the European branch, while the CA infection comes directly from China (and likely much earlier in 2020). There appears to be some difference in virulence between the strains. https://www.medrxiv.org/content/10.1101/2020.04.08.20056929v2 https://www.medrxiv.org/content/10.1101/2020.04.08.20056929v... and https://nextstrain.org/ncov/global https://nextstrain.org/ncov/global
    - kgwgk 6y ago
      I’ve never looked into that (I know of the hypothesis, though) but IFR in South Korea also seems to be around 1% (or higher), not 0.1%
cma 6y ago
Exaggerated in that the California study predicted a flu like IFR (Ioannidis, one of the authors, has been pushing that as his pet theory for over a month). This shows it is more likely several times that.
- glofish 6y ago
  it doesn't contradic Ioannidis at all. He might be right in the end, and that would be an incredibly bitter pill to many. We are getting closer and closer to the flu fatality rate. from 5% to 3%, then 1% now 0.5% - smart money (and common sense) would bet that the rate will continue to drop, I would expect that people living in healthier environment than NYC will fare even better. No way NYC IFR is the upper bound for the rest of the country. You could just as well expect it to be 10x higher than other places. the flu rate is 0.1%, thus we already hit the order of magnitude.
  - jdm2212 6y ago
    NYC is a younger, healthier city than the national average. NYC also has one of the best medical systems not just in America, but in the world. Roughly 0.2% of everyone in NYC has already died of COVID-19. So 0.2% is pretty close to a hard lower bound on the IFR for COVID-19. And I don't think anyone serious suggests every NYC resident has had COVID. I don't think anyone seriously suggests even half of NYC has had COVID.
    - glofish 6y ago
      All I am saying people there breath the NYC air every single day. Can't possibly be good for them, especially in the light of some chronic pulmonary inflammation induced deaths. The point on the lower bound of 0.2% is informative. I did not know that.
      - timr 6y ago
        That lower-bound argument is overly simplistic. There are huge error bars on that ratio, because "the population of NYC" is not something easily defined, and the death count (at this time) includes a lot of "excess deaths" that almost certainly have nothing to do with the virus (e.g. untreated cardiac arrest).

dwaltrip 6y ago

I agree with everything you are saying. This does look like good news. However, we should remember that this is probably a mild overestimate, as the study population was assembled from those who were out and about (shopping at the grocery store, etc). People who are more strictly staying home -- and thus less likely to have been infected -- wouldn't be included in the survey.

bob33212 6y ago
Another way to look at it is that 20% of people most likely to contribute to a high transmission rate already have had it, in turn flattening the curve by a considerable amount

cjhopman 6y ago

Sure, but a person who goes to the store 7 times a week is ~7 times as likely to have been sampled compared to someone who goes only once a week. They are also ~7 times as likely to have contracted covid-19 at the grocery store. It's likely that they are also, in general, isolating less than the other. This sampling would have a (possibly slight, but still unknown) bias towards people who are isolating less (who you would expect to then have a higher than average percent positive for antibodies, assuming isolating is helpful). I don't think anyone knows whether or not this is a more representative sample than the stanford study.

WrtCdEvrydy 6y ago
Unfortunately, unless you go door to door knocking... you're gonna get biased data.
- ineedasername 6y ago
  Or truly random sampling that includes appropriate proportions of sub groups, e.g, socioeconomic status etc. Otherwise you have to know on all of the doors. But it's still a lot: for a 95% confidence level and 2% margin of error, for NYC you would need to test about 2,400 people. Assuming a response rate of 10%, you need to knock on 24,000 doors.
  - austingulati 6y ago
    Wouldn't the ability to refuse taking the test introduce bias? i.e. the 10% that do take it still may not be representative of the entire population. Likely better than the grocery store sample, but still not ideal
    - qqqwerty 6y ago
      Yeah, that is still an issue. You can offer compensation to incentivize participation. And you can pre-select your sample (instead of random door-to-door) and require multiple follow ups with the selected persons to reduce nonresponse bias. You can also do a separate phone survey in addition to surveying the test participants. Questions like "Do you think you have had COVID?" and "How many times per week do you leave the house?". If the responses for the test participants vary significantly from the phone survey participants, you can try and weight your data accordingly.

vkou 6y ago

The major methodology problem with the Stanford study was not that they recruited participants through Facebook. The major problem in the Stanford study was that they ran a test that has a 3% false positive rate, and found that 3% of the test-takers tested positive. (And that apparently the asymptomatic COVID-19 rate is 90+% - which does not square with the Diamond princess data). They could have ran that same study back in 2018, and would have gotten the exact same garbage results.

zucker42 6y ago

Is there a write-up for this study? Or anywhere with data/methodology released? Also, is this particularly good news? Using the raw numbers from the headline and the deaths from here https://www1.nyc.gov/site/doh/covid/covid-19-data.page https://www1.nyc.gov/site/doh/covid/covid-19-data.page we can make coarse fatality rate estimates. I assume population of NYC is 8,400,000. 10290 / (.212 * 8.4e6) = 0.6% Including probable deaths (which makes the death count line up more closely with excess deaths) 15411 / (.212 * 8.4e6) = 0.9%. So it seems consistent with a IFR of 0.5%-1%. However, this doesn't account for the lag between antibody presence and death. Also, we don't know what bias sampling from grocery stores introduces, it could affect the results in either direction. Hopefully, New York releases a paper so we can get more accurate estimates. It's good to see a prevalence study with presumably less flaws.

tunesmith 6y ago

Sorry, can someone spell out how this might be good news? 21% is still a long way from herd immunity, and NYC's hospital system has been severely strained getting to this point. On top of that, the hospitalization rate still seems disproportionate to someone getting the flu - maybe it's five times as fatal, but it's > 5x the flu hospitalization rate. It's not like the virus has become less dangerous, we're just realizing how dangerous it has been, what with the impact we've already experienced. Besides, generally if a virus is less fatal than previously expected, it means it's more contagious, meaning that much harder to get to herd immunity. Good news would be things like: evidence the virus has mutated into something less severe; evidence of an anti-viral treatment that improves outcomes for everyone so it's not as big a deal to catch it; evidence that community spread has halted in an area and the boundaries are controlled so people in that area can feel secure they won't catch it; evidence of an impending vaccine. Is this good news just because we're finally establishing that people have caught it once can't re-catch it for now? I guess I can see that as good news but that is so expected that it's more like it would be horrendously bad news if we found evidence that recovered people didn't have antibodies. But generally I don't really see what policy impact this has, other than identifying a pool of workers that can go work in meat-packing factories without fear of catching it again.

cm2187 6y ago
I don't know if the hospitalisation rate is so out of whack vs the flu. It should be higher. But does 20% of the population catches the flu in a matter of weeks?
- creaghpatr 6y ago
  Much of the population gets a flu vaccine so would be tricky to compare.
  - cm2187 6y ago
    My very point. It contaminates people at a slower pace, for that reason and cross immunity, higher R0, etc.
- 6y ago

ineedasername 6y ago

I couldn't find any detailed write up about their selection methods for participants, only the basics of where they found them. Without that, it is very hard to determine whether or not there's some flavor of selection bias: If it is voluntary, and they do not screen out people who report having had cold/flu symptoms, then they run the risk of attracting a disproportionate number of people who volunteer because they're curious if their prior symptoms were actually coronavirus. That would make it far from a random sample. The fact that they sampled only people actually leaving their house is also a form of selection bias: these are the people more likely to be exposed and may represent a disproportionately high infection rate as well. So I await further information. That said, even if it's overall 0.6%, that is still 6x higher than flu, and higher than H1N1 which had a CFR around 0.1 for areas with adequate health care. (much higher when there was not adequate care, but that is not dissimilar to Covid.) So no matter what, no one should be walking away from this study saying "See it's no big deal! Just the Flu/Cold etc!"

Retric 6y ago
Flu is generally well under 0.1% of those infected on average. If you’re comparing them you want to either include or exclude asymptotic people from both populations. “Symptomatic Illnesses” https://www.cdc.gov/flu/about/burden/index.html https://www.cdc.gov/flu/about/burden/index.html As to NYC deaths, many people currently infected will die in the future. You can make various estimates to account for this but a reasonable first approximation is to double current deaths based on NYC’s infection curve vs South Korea’s.
- greendave 6y ago
  As an asymptotic* person, I resent your hyperbola. (Sorry, couldn't resist).
  - deleted 6y ago
    [deleted]
- ineedasername 6y ago
  Flu rates are based on all known knowledge, population testing, etc. We don't have that for covid-19. If you want a good apple-to-apples comparison of lethality of another pandemic, you need to find CFR numbers that were available during the pandemic. That is what I provided in my post. Here is the source I got them from [0] which gives the CFR at the 10-week mark for H1N1, somewhat similar to where we are now. Here's the relevant quote: >"The overall case fatality rate as of 16 July 2009 (10 weeks after the first international alert) with pandemic H1N1 influenza varied from 0.1% to 5.1% depending on the country." [0] https://www.cebm.net/covid-19/global-covid-19-case-fatality-rates/ https://www.cebm.net/covid-19/global-covid-19-case-fatality-...

credit_guy 6y ago

Yes, the study is really good news for NYC. My thinking is that the state of emergency, currently in place until 5/15, will be extended at most once to 6/15. After that we'll be somewhat open for business. I imagine everyone will be advised to wear mask/gloves, to keep the social distance and to keep washing hands. Lots of people will continue to work from home, others will bike to work. Being summer, the virus will not be that contagious. When the November rebound is forecast to happen, the herd immunity will be much higher, hospitals better prepared, the medical world will have learned the most effective ways to treat this disease, so the fatality rate will go down to levels seen in Germany, or lower. Bottom line, I don't think we'll have another shelter-in-place in November-December.

robocat 6y ago
> Being summer, the virus will not be that contagious. I think that is still scientifically debatable, good overview: https://www.newscientist.com/article/2239380-will-the-spread-of-covid-19-be-affected-by-changing-seasons/ https://www.newscientist.com/article/2239380-will-the-spread... Certainly it is still transmissible in hot countries.
- ethbro 6y ago
  It seems reasonable that lipid encapsulated viruses are fairly stable in a variety of temperature ranges, no? On the other hand, I can see the additional environmental UV saturation and vitamin D having a noticeable effect. Even a small decrease in environmental persistence should help, given its infectiousness.
- credit_guy 6y ago
  Thanks for the link. It's a good read. That said, science is enamored with significance levels, p-values, etc. Most people just do that because that's what they've learned, and think that's how it should be done. They don't think it comes from Bayesian inference, where you put a very low prior probability of a drug being useful (because the very vast majority of chemicals we could put in our bodies are either harmful, or have harmful side-effects). In this case, the Bayesian prior is that common colds in general become less prevalent during the Summer, and Covid19 is caused by a coronavirus, which is related to viruses causing the common cold. The burden of proof should be lowered many, many times, but I nobody who did these statistical surveys did that, because this is a total no-no in the field. So, I'll state my conclusion, after reading your link: the infectiousness of Covid19 is more likely than not to decrease in the Summer, based on the studies that were performed. It will not decrease to zero, so that does not contradict your statement that it is transmissible in hot countries.

godelski 6y ago

Why does recruiting through Facebook invalidate the results? If it is a representative sample it is a representative sample regardless of how people were recruited.

alteria 6y ago
One critique was that the segment of the population who would respond to the ad and actually test may be more likely to have experienced COVID-like symptoms.
- chockablock 6y ago
  Also people reported sharing the Santa Clara link with others who might want the test (due to having had symptoms). In principle you could have the same problem with this survey: call your buddy and say “come on down to Costco—they’re doing free antibody tests”. The obvious fix is to not tell people the results of their own tests. Not sure of the ethics/consequences of that approach.
  - deleted 6y ago
    [deleted]
zucker42 6y ago
The Facebook ad essentially said "get tested for COVID-19 here", so very plausibly infected people could be more likely to respond.
dragonwriter 6y ago
> Why does recruiting through Facebook invalidate the results? If Because “people who both use Facebook and don't automatically discount every ad or other solicitation on Facebook not from someone they personally know, especially if it invokes a major news story, because of the risk of it being a scam looking to steal personal information or do something similarly nefarious” are not representative of “people”.
- 6y ago

eanzenberg 6y ago

This study is good news... I'd be interested to project the rate of infections WITHOUT shutting down the economy for what's now going on 6 weeks. Seems like it was all in vain.

8ytecoder 6y ago

(Reposting my comment again here) This one's from a different company (BioMedomics) and it was a random test of people pulled straight off the streets. Happened at an entirely different geographical area (Chelsea, Massachusetts) as well. I don't want to be too optimistic but there are some signs that we are heavily under-counting the actual number of cases (at least in the US). https://www.bostonglobe.com/2020/04/17/business/nearly-third-200-blood-samples-taken-chelsea-show-exposure-coronavirus/ https://www.bostonglobe.com/2020/04/17/business/nearly-third...

mlthoughts2018 6y ago

The rates being this high actually casts doubt on the study. It’s very very implausible, and suggests almost surely a selection bias in the sample towards a population much more likely to have contracted it.

mlthoughts2018 6y ago
Actually in reading closer and seeing the data is collected at big box & grocery stores, it’s almost surely very biased. This cohort would skew younger and wealthier, which correlates with better preexisting health and fewer risk factors, and would exclude populations who systematically left the city, or who have known heightened risk factors. Comparing the death rate overall with the infection rate of this skewed sample would be likely to greatly underestimate the actual death rate of the virus.
- Gibbon1 6y ago
  This weekend San Francisco plans to do a hard test of about 6000 people in a couple of square blocks of the Mission district. They are going door to door to encourage people to get tested. Still biased but probably better than anything so far.
  - mlyle 6y ago
    Yup.. though it's unclear how much we'll clear the test sensitivity hurdle by.
    - Gibbon1 6y ago
      I wonder about that. They are trying to test a neighborhood they think is most effected. One wonders if it's possible to follow up and retest positives to exclude false one. Suspect the Bay Area is moving towards what Korea is doing. Intense testing and contact tracing. Friend who works at a biomedical corp says scuttlebutt is they're separating their workforce into groups based on critical need and how well they can work remotely. Workers like him not expected to return to work onsite until next year.

dboreham 6y ago

Only 60k more deaths to go until herd immunity.

dchichkov 6y ago

What is the percentage of people with innate immunity to COVID-19?

Symmetry 6y ago

And the specificity of the test might be worse than advertised but it wouldn't be credible that it could be making a big different with the 20% positive rate here.

smartmic 6y ago

What I learned from an article of a major German newspaper is that in order to really estimate the quality of testing, two criteria are important: sensitivity and specifity of the applied test. If the real infection rate is still low throughout the population, a random sampling will not lead to reliable results (you could also toss a coin). Although the article is in German, it is worth to have a look on the graphic in the middle of the page, it should be understandable. https://www.sueddeutsche.de/gesundheit/corona-test-antikoerper-1.4883631 https://www.sueddeutsche.de/gesundheit/corona-test-antikoerp...

hsitz 6y ago

> Here, it's closer to a random sample, but more importantly it shows really high rates. Those rates overwhelm any error due to false positives. I'm not seeing that, at least not from what I've seen. Whether false positives skew results significantly is highly dependent on how accurate the antibody test they used is (in addition to how large a subset of the population is positive). This guy has an interesting visual tool that helps you to see how much a study could be affected. [1] Also, he says "Here's an interesting relationship. When a test with 95% sensitivity and 95% specificity is applied to a population with <5% prevalence of disease, MOST of the patients with positive tests are FALSE POSITIVEs." I.e., positive rate shows up as greater than 10%, when it's actually less than 5%. Do we know the sensitivity and specificity accuracies for the antibody tests used in NY? You'd like to think they know what they're doing. But this Bayesian stuff can be tricky, especially if they're rushing something through (esp. regarding testing of accuracy of the antibody tests themselves). And the California studies, although they seem to have some competent people behind them, seem to inflate/exaggerate the lower bound of uncertainty in their projection. [2] [1] https://sites.google.com/view/tgmteststat/home https://sites.google.com/view/tgmteststat/home [2] https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaws-in-stanford-study-of-coronavirus-prevalence/ https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaw...