4 ms·

OP's whole point is it's not a good sample because people were recruited while grocery shopping...I can't think of a more tech industry-oriented view of the wor

OP's whole point is it's not a good sample because people were recruited while grocery shopping...I can't think of a more tech industry-oriented view of the world; most people work from home on zoom and use instacart right?? No. I'd expect people who order deliveries online and can do their entire job on Zoom/slack all day to be a minority to the point where including them in the study would be covered in a sampling error.

mlthoughts2018 6y ago
You’re completely missing the point. The selection effects would be things like able-bodied / non-immunocompromised / already-recovered people being selected way higher, which would inflate the apparent infected population size estimate and underestimate the death rate among those infected. This is not some snooty tech point of view - this kind of flaw in statistical studies matters a lot. The more important the policy implications are, the more important the validity of statistical methods. It’s anything but parochial.
- mlyle 6y ago
  I don't think anyone disagrees that the sample has bias. On the other hand, since the majority of people have no choice but to grocery shop in person, I do not think the sample is as tilted as you imply. And even if it were--- it's notable that it's this easy to find a subgroup with cleared infections this far above what was anticipated to be found across NY/NYC. And-- it's remarkably consistent with statistical estimates made through a variety of other means, e.g. https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v...
  - mlthoughts2018 6y ago
    > “ since the majority of people have no choice but to grocery shop in person” This is deeply flawed. I live in NYC and have been making donations to have food delivered to elderly people since this started. Just the other day, de Blasio announced that taxi and rideshare drivers would be paid by the city to deliver food - and city agencies would be serving as overflow dispatchers and tech support for it. Many, many people do not have to go to the grocery store, or they have their healthy younger family member go in their place, etc. To boot, as Gelman has written on his blog, these studies do not agree with a wide range of other studies, and in most cases the test specificity is on par with the incidence rate itself, making incidence rate estimates from _all_ of the studies very unreliable for correlated reasons, so that pooling the studies does nothing to overcome the huge uncertainty they suffer in incidence rates.
    - mlyle 6y ago
      > and in most cases the test specificity is on par with the incidence rate itself, ????? We can't exclude that specificity may be 97% (it's unlikely, based on our data, but it's at the edge of the confidence interval), which is unfortunately on par with the studies returning 2-4% positives... but can't exactly explain a return of 21%. Subtract off 4% worst-case false positives from 21%, and where are you? Since you want to appeal to authority with Gelman, this is what he said about this on his blog: "– Those California studies estimating 2% or 4% infection rate were hard to assess because of the false-positive problem: if a test has a false positive rate of 1% and you observe 1.5% positive tests, your estimate’s gonna be super noisy. But if 20% of the tests you observe are positive, then the false-positive rate is less of a big deal." ... "– In any case, the 20% number seems reasonable. It’s hard for me to imagine it’s a lot higher, and, given the number of deaths we’ve seen already, I guess it can’t be much lower either." > so that pooling the studies does nothing to overcome the huge uncertainty they suffer in incidence rates. The case count multiple from the serological study for New York state (and indeed, the California counties) is right in line with what's expected with a variety of statistical measures that were made without relying upon the serological data. So if you even peeked at my source you'd not be making this argument. https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v...
      - mlthoughts2018 6y ago
        From the same Gelman article: > First off, 3% does not sound implausible. If they said 30%, I’d be skeptical, given how everyone’s been hiding out for awhile, but 3%, sure, maybe so.
        mlyle 6y ago
        Yes. 3% does not sound implausible (for the Bay Area). Just like he says 20% does not sound implausible (for New York). Have you ever been diagnosed with reading comprehension issues?
- SubiculumCode 6y ago
  This is a convenience sample no doubt, but convenience is important here, as timeliness is important. It is easy to naysay, but why don't you suggest a less biased convenience sample? Because unless you mandate whole population testing, samples will always be biased in some way. Those who have phones, those who have time, those who are motivated, etc etc. Given all the potential ways it could be biased, choosing a convenience sample at locations almost universally used is not bad at all, imo

mlthoughts2018 6y ago

You’re completely missing the point. The selection effects would be things like able-bodied / non-immunocompromised / already-recovered people being selected way higher, which would inflate the apparent infected population size estimate and underestimate the death rate among those infected. This is not some snooty tech point of view - this kind of flaw in statistical studies matters a lot. The more important the policy implications are, the more important the validity of statistical methods. It’s anything but parochial.

mlyle 6y ago
I don't think anyone disagrees that the sample has bias. On the other hand, since the majority of people have no choice but to grocery shop in person, I do not think the sample is as tilted as you imply. And even if it were--- it's notable that it's this easy to find a subgroup with cleared infections this far above what was anticipated to be found across NY/NYC. And-- it's remarkably consistent with statistical estimates made through a variety of other means, e.g. https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v...
- mlthoughts2018 6y ago
  > “ since the majority of people have no choice but to grocery shop in person” This is deeply flawed. I live in NYC and have been making donations to have food delivered to elderly people since this started. Just the other day, de Blasio announced that taxi and rideshare drivers would be paid by the city to deliver food - and city agencies would be serving as overflow dispatchers and tech support for it. Many, many people do not have to go to the grocery store, or they have their healthy younger family member go in their place, etc. To boot, as Gelman has written on his blog, these studies do not agree with a wide range of other studies, and in most cases the test specificity is on par with the incidence rate itself, making incidence rate estimates from _all_ of the studies very unreliable for correlated reasons, so that pooling the studies does nothing to overcome the huge uncertainty they suffer in incidence rates.
  - mlyle 6y ago
    > and in most cases the test specificity is on par with the incidence rate itself, ????? We can't exclude that specificity may be 97% (it's unlikely, based on our data, but it's at the edge of the confidence interval), which is unfortunately on par with the studies returning 2-4% positives... but can't exactly explain a return of 21%. Subtract off 4% worst-case false positives from 21%, and where are you? Since you want to appeal to authority with Gelman, this is what he said about this on his blog: "– Those California studies estimating 2% or 4% infection rate were hard to assess because of the false-positive problem: if a test has a false positive rate of 1% and you observe 1.5% positive tests, your estimate’s gonna be super noisy. But if 20% of the tests you observe are positive, then the false-positive rate is less of a big deal." ... "– In any case, the 20% number seems reasonable. It’s hard for me to imagine it’s a lot higher, and, given the number of deaths we’ve seen already, I guess it can’t be much lower either." > so that pooling the studies does nothing to overcome the huge uncertainty they suffer in incidence rates. The case count multiple from the serological study for New York state (and indeed, the California counties) is right in line with what's expected with a variety of statistical measures that were made without relying upon the serological data. So if you even peeked at my source you'd not be making this argument. https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.04.18.20070821v...
    - mlthoughts2018 6y ago
      From the same Gelman article: > First off, 3% does not sound implausible. If they said 30%, I’d be skeptical, given how everyone’s been hiding out for awhile, but 3%, sure, maybe so.
      - mlyle 6y ago
        Yes. 3% does not sound implausible (for the Bay Area). Just like he says 20% does not sound implausible (for New York). Have you ever been diagnosed with reading comprehension issues?
SubiculumCode 6y ago
This is a convenience sample no doubt, but convenience is important here, as timeliness is important. It is easy to naysay, but why don't you suggest a less biased convenience sample? Because unless you mandate whole population testing, samples will always be biased in some way. Those who have phones, those who have time, those who are motivated, etc etc. Given all the potential ways it could be biased, choosing a convenience sample at locations almost universally used is not bad at all, imo