4 ms·

We don't have perfect studies and perfect data but calling these tests "bad" seems slightly unfair. They give us an imperfect but useful snapshot of what's goin

We don't have perfect studies and perfect data but calling these tests "bad" seems slightly unfair. They give us an imperfect but useful snapshot of what's going on. But thanks for the condescension. I'd expect nothing less on HN.

CydeWeys 6y ago
The study is dramatically over-sampling exactly the people who have the most potential exposure to COVID-19, and dramatically over-sampling the people who don't. This could easily be way off. It's not testing the people who aren't home at all, and it has a low chance of only testing the people who leave home rarely, only when strictly necessary. It's mainly finding the people who leave home a lot.
- TheBlight 6y ago
  How do you know how often these people are leaving their homes? Everyone has to leave their home to go to the grocery store eventually or someone in their family does. Yes there are delivery services but the vast majority of people aren't using those.
  - kgwgk 6y ago
    The more often they go, the more likely you encounter them.
    - FlyMoreRockets 6y ago
      True, but if you encounter one daily shopper, two weekly shoppers and twenty monthly shoppers, it will skew the results. Same if the numbers are reversed.
      - kgwgk 6y ago
        I don’t get your point. Maybe we agree. If in one city there are 1000 30-year-old daily-shopers, 1000 50-year-old weekly-shoppers and 1000 70-year-old monthly-shoppers it will not be a good idea to estimate the age of the population by sampling people who enters the store. The average age is 50, not 34.
      - jdavis703 6y ago
        Most people in NYC are almost certainly not shopping monthly because of storage constraints. Even bi-weekly shopping is likely to exhaust storage space for city-dwellers in small apartments.
      - CydeWeys 6y ago
        But you won't, the opposite will happen; you'll proportionally encounter way more daily shoppers than infrequent shoppers, for the simple tautological reason that the daily shoppers are going more often. This is a sampling bias, and it's the exact same problem as if you ran a survey trying to ascertain how many people have landlines ... by calling random phone numbers.
- varenc 6y ago
  I think the overwhelming majority of people are visiting grocery stores to buy food. (Or have someone in their household doing it) If the studied presume this then it makes more sense.
  - CydeWeys 6y ago
    You can't assume that. I know plenty of people who haven't left their apartments in a month plus, because they're having groceries ordered in. And you completely missed the point that people who are going outside a lot more often are both (a) more likely to have been exposed to COVID-19, and (b) more likely to be encountered by your survey. If there are e.g. 700 daily shoppers and 700 weekly shoppers going to a given store, then if you go on one day and sample every shopper you're not going to get half and half by population as you'd need to for a true random sample; instead, you're going to get 700 daily shoppers and only 100 weekly shoppers! This is a hugely biased sample!
    - leereeves 6y ago
      Are you describing the actual study or the study as you're imagining it? Because I imagine the statisticians have thought of these concerns and attempted to correct for them. I find it interesting that this study corresponds to an IFR around 0.5%, similar to the IFR arrived at by other studies using very different methods. Perhaps the statisticians who conduct these studies actually know what they're doing better than Hacker News commenters. And perhaps it's time to accept the politically inconvenient scientific truth these studies are revealing.
      - twomoretime 6y ago
        >similar to the IFR arrived at by other studies using very different methods With the wild range of IFR from so many studies, you could trivially cherry pick supporting studies for a bunch of different numbers, especially if you throw in am error range.
    - deleted 6y ago
      [deleted]
- deleted 6y ago
  [deleted]
- akiselev 6y ago
  Couldn't you get a bad study that reverses the sampling bias to get a decent picture of social distancing strategies using the two bad studies? That alone seems far more useful than knowing the IFR because its actionable information.
  - qqqwerty 6y ago
    Or better yet, do a simple survey and ask the participants how frequently they go to the grocery store. And if you see a correlation between frequency and infection rate, then try and adjust your weights accordingly. These bias issues are well known among survey designers. There are ways that you can try and account for them. It seems like maybe the medical community is too used to having control groups in their studies?
- SubiculumCode 6y ago
  Most people go to the grocery store. There seems to be this attitude on HN of immediately dismissing research. No human research is ever perfect, flaws and limitations will always be present, and in the present case, study designs are being expedited in order to be timely. Each study is a piece, and together can provide convergent evidence.

CydeWeys 6y ago

The study is dramatically over-sampling exactly the people who have the most potential exposure to COVID-19, and dramatically over-sampling the people who don't. This could easily be way off. It's not testing the people who aren't home at all, and it has a low chance of only testing the people who leave home rarely, only when strictly necessary. It's mainly finding the people who leave home a lot.

TheBlight 6y ago
How do you know how often these people are leaving their homes? Everyone has to leave their home to go to the grocery store eventually or someone in their family does. Yes there are delivery services but the vast majority of people aren't using those.
- kgwgk 6y ago
  The more often they go, the more likely you encounter them.
  - FlyMoreRockets 6y ago
    True, but if you encounter one daily shopper, two weekly shoppers and twenty monthly shoppers, it will skew the results. Same if the numbers are reversed.
    - kgwgk 6y ago
      I don’t get your point. Maybe we agree. If in one city there are 1000 30-year-old daily-shopers, 1000 50-year-old weekly-shoppers and 1000 70-year-old monthly-shoppers it will not be a good idea to estimate the age of the population by sampling people who enters the store. The average age is 50, not 34.
    - jdavis703 6y ago
      Most people in NYC are almost certainly not shopping monthly because of storage constraints. Even bi-weekly shopping is likely to exhaust storage space for city-dwellers in small apartments.
    - CydeWeys 6y ago
      But you won't, the opposite will happen; you'll proportionally encounter way more daily shoppers than infrequent shoppers, for the simple tautological reason that the daily shoppers are going more often. This is a sampling bias, and it's the exact same problem as if you ran a survey trying to ascertain how many people have landlines ... by calling random phone numbers.
varenc 6y ago
I think the overwhelming majority of people are visiting grocery stores to buy food. (Or have someone in their household doing it) If the studied presume this then it makes more sense.
- CydeWeys 6y ago
  You can't assume that. I know plenty of people who haven't left their apartments in a month plus, because they're having groceries ordered in. And you completely missed the point that people who are going outside a lot more often are both (a) more likely to have been exposed to COVID-19, and (b) more likely to be encountered by your survey. If there are e.g. 700 daily shoppers and 700 weekly shoppers going to a given store, then if you go on one day and sample every shopper you're not going to get half and half by population as you'd need to for a true random sample; instead, you're going to get 700 daily shoppers and only 100 weekly shoppers! This is a hugely biased sample!
  - leereeves 6y ago
    Are you describing the actual study or the study as you're imagining it? Because I imagine the statisticians have thought of these concerns and attempted to correct for them. I find it interesting that this study corresponds to an IFR around 0.5%, similar to the IFR arrived at by other studies using very different methods. Perhaps the statisticians who conduct these studies actually know what they're doing better than Hacker News commenters. And perhaps it's time to accept the politically inconvenient scientific truth these studies are revealing.
    - twomoretime 6y ago
      >similar to the IFR arrived at by other studies using very different methods With the wild range of IFR from so many studies, you could trivially cherry pick supporting studies for a bunch of different numbers, especially if you throw in am error range.
  - deleted 6y ago
    [deleted]
deleted 6y ago
[deleted]
akiselev 6y ago
Couldn't you get a bad study that reverses the sampling bias to get a decent picture of social distancing strategies using the two bad studies? That alone seems far more useful than knowing the IFR because its actionable information.
- qqqwerty 6y ago
  Or better yet, do a simple survey and ask the participants how frequently they go to the grocery store. And if you see a correlation between frequency and infection rate, then try and adjust your weights accordingly. These bias issues are well known among survey designers. There are ways that you can try and account for them. It seems like maybe the medical community is too used to having control groups in their studies?
SubiculumCode 6y ago
Most people go to the grocery store. There seems to be this attitude on HN of immediately dismissing research. No human research is ever perfect, flaws and limitations will always be present, and in the present case, study designs are being expedited in order to be timely. Each study is a piece, and together can provide convergent evidence.