4 ms·

I have not seen a single useful model from professional epidemiologists so far -- at least nothing that would guide me better than "look at China and Italy and

I have not seen a single useful model from professional epidemiologists so far -- at least nothing that would guide me better than "look at China and Italy and consider that it might also happen here."

xzel 6y ago
I would argue that they're all useful but its very tough to be accurate when confounding actions are occurring to the problem. For example, in the US, the increase in social distancing measures and mask usage couldn't be easily baked into the original models. Forecasting accurately is very tough. Look at what's been said about Renaissance and its Medallion fund, they're right a little over 50% (they just leverage very heavily). Think about that, the most consistent and dominate hedgefund might only be right about 51% of the time. The issue with just using historical data is the data might be different. But I'm sure all of their models were using data or at least some domain knowledge from the spread in Italy and China.
- icelancer 6y ago
  >> For example, in the US, the increase in social distancing measures and mask usage couldn't be easily baked into the original models. The IHME model - one of the most cited models - explicitly factored this into their model.
  - mattkrause 6y ago
    "Factored in" covers a lot of ground. Here, for reference is the IHME model preprint: https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v... The parameterization of social distancing is on pages 3-4. They considered four coarse categories of interventions, with the total effect given by a ad hoc score of 1, 0.67, 0.334, or 0. It should be blindingly obvious that can be improved upon with more data and work. Moreover, the effects of social distancing aren't even constant--compliance certainly varies from place to place and over time (enforcement changes, people get restless).
  - archgoon 6y ago
    "Starting April 17, we began using mobile phone data to better assess the impact of social distancing across states and countries. These data revealed that social distancing was happening to a larger degree than previously understood, and even before social distancing mandates went into effect." This suggests that social distancing happened 1-2 weeks sooner than their model anticipated. This will give you a much smaller number. What is the policy you believe should have been followed instead? Are you saying that social distancing should have been abandoned? Why do you believe that this would result in fewer deaths?
    - icelancer 6y ago
      No, I am saying they factored it into their models and also undershot their priors that were readily available. If they looked at OpenTable data, they would have realized people were voluntarily socially distancing before state authorities told them they should. In fact, OpenTable data showed that even on the day that the Mayor of NYC told people to go to sit-down restaurants and see movies in person (in early March), y/o/y data showed a net 30% reduction in reservations for restaurants in NYC. People were voluntarily taking precaution well before our government bureaucracies enforced it, and it was available in public datasets. IHME just didn't look hard enough, apparently. Despite what the media keeps pushing by finding pockets of idiots, people aren't stupid. The most at-risk populations realize this and tend to shelter in place on their own and take precautions. de Blasio on March 11th telling people to go out: https://ny.eater.com/2020/3/11/21175497/coronavirus-nyc-restaurants-safe-dine-out https://ny.eater.com/2020/3/11/21175497/coronavirus-nyc-rest... OpenTable data showing a voluntary reduction of restaurant activity despite de Blasio's encouragements: https://ibb.co/r2R9xnT https://ibb.co/r2R9xnT
      - archgoon 6y ago
        What policies are you saying were misguided and should not have been taken based on the predicted elevated death toll? How does one evaluate the OpenTable data and feed it into the model they were using to estimate the amount of social distancing?
        6y ago
13415 6y ago
May I ask which models you have seen so far? I'm interested in them.
icelancer 6y ago
Correct. This appeal to authority fallacy being touted has given us models like the UW-IHME model which is a complete disaster in both results and in methodology. And domestic policy is being based off these models that have two orders of magnitude (at least) of absolute error, and were outside their 95% CI over half of the time. This is unacceptable. https://twitter.com/ElonBachman/status/1248649785655078914 https://twitter.com/ElonBachman/status/1248649785655078914 https://arxiv.org/abs/2004.04734 https://arxiv.org/abs/2004.04734
- buzzkillington 6y ago
  Remember that Greece's economy will not recover to it's 2008 for 20 years because of an excel spreadsheet with errors in it.
  - icelancer 6y ago
    Japan's Nikkei index is still down from the late 1980s, and likely to stay down for many more years to come. The chronic effects of shutting down the economy are vast. People think money can be regenerated; it can. But what Americans have seemingly forgotten is the past where American autoworkers feared Japanese efficiency and where the Japanese economy looked to crush ours. Today, the Japanese salaryman works at least 50% more hours and his purchasing power is less than his counterpart in the United States (just look at patio11's posts on it). https://www.kalzumeus.com/2014/11/07/doing-business-in-japan/ https://www.kalzumeus.com/2014/11/07/doing-business-in-japan... It's not a popular take because of recency bias (the same thing happened on 9/11 with the PATRIOT Act and always doe when outliers happen), but the data will be pretty clear in the end.
cycomanic 6y ago
What does useful mean? I would argue most models are useless without a good understanding of the parameters going into that model. That is where the domainspecific knowledge comes in. Fitting some curve to data is useless unless you know what the parameters correspond to, and how they influence the parameters of the curve.

xzel 6y ago

I would argue that they're all useful but its very tough to be accurate when confounding actions are occurring to the problem. For example, in the US, the increase in social distancing measures and mask usage couldn't be easily baked into the original models. Forecasting accurately is very tough. Look at what's been said about Renaissance and its Medallion fund, they're right a little over 50% (they just leverage very heavily). Think about that, the most consistent and dominate hedgefund might only be right about 51% of the time. The issue with just using historical data is the data might be different. But I'm sure all of their models were using data or at least some domain knowledge from the spread in Italy and China.

icelancer 6y ago
>> For example, in the US, the increase in social distancing measures and mask usage couldn't be easily baked into the original models. The IHME model - one of the most cited models - explicitly factored this into their model.
- mattkrause 6y ago
  "Factored in" covers a lot of ground. Here, for reference is the IHME model preprint: https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v... The parameterization of social distancing is on pages 3-4. They considered four coarse categories of interventions, with the total effect given by a ad hoc score of 1, 0.67, 0.334, or 0. It should be blindingly obvious that can be improved upon with more data and work. Moreover, the effects of social distancing aren't even constant--compliance certainly varies from place to place and over time (enforcement changes, people get restless).
- archgoon 6y ago
  "Starting April 17, we began using mobile phone data to better assess the impact of social distancing across states and countries. These data revealed that social distancing was happening to a larger degree than previously understood, and even before social distancing mandates went into effect." This suggests that social distancing happened 1-2 weeks sooner than their model anticipated. This will give you a much smaller number. What is the policy you believe should have been followed instead? Are you saying that social distancing should have been abandoned? Why do you believe that this would result in fewer deaths?
  - icelancer 6y ago
    No, I am saying they factored it into their models and also undershot their priors that were readily available. If they looked at OpenTable data, they would have realized people were voluntarily socially distancing before state authorities told them they should. In fact, OpenTable data showed that even on the day that the Mayor of NYC told people to go to sit-down restaurants and see movies in person (in early March), y/o/y data showed a net 30% reduction in reservations for restaurants in NYC. People were voluntarily taking precaution well before our government bureaucracies enforced it, and it was available in public datasets. IHME just didn't look hard enough, apparently. Despite what the media keeps pushing by finding pockets of idiots, people aren't stupid. The most at-risk populations realize this and tend to shelter in place on their own and take precautions. de Blasio on March 11th telling people to go out: https://ny.eater.com/2020/3/11/21175497/coronavirus-nyc-restaurants-safe-dine-out https://ny.eater.com/2020/3/11/21175497/coronavirus-nyc-rest... OpenTable data showing a voluntary reduction of restaurant activity despite de Blasio's encouragements: https://ibb.co/r2R9xnT https://ibb.co/r2R9xnT
    - archgoon 6y ago
      What policies are you saying were misguided and should not have been taken based on the predicted elevated death toll? How does one evaluate the OpenTable data and feed it into the model they were using to estimate the amount of social distancing?
      - 6y ago

13415 6y ago

May I ask which models you have seen so far? I'm interested in them.

icelancer 6y ago

Correct. This appeal to authority fallacy being touted has given us models like the UW-IHME model which is a complete disaster in both results and in methodology. And domestic policy is being based off these models that have two orders of magnitude (at least) of absolute error, and were outside their 95% CI over half of the time. This is unacceptable. https://twitter.com/ElonBachman/status/1248649785655078914 https://twitter.com/ElonBachman/status/1248649785655078914 https://arxiv.org/abs/2004.04734 https://arxiv.org/abs/2004.04734

buzzkillington 6y ago
Remember that Greece's economy will not recover to it's 2008 for 20 years because of an excel spreadsheet with errors in it.
- icelancer 6y ago
  Japan's Nikkei index is still down from the late 1980s, and likely to stay down for many more years to come. The chronic effects of shutting down the economy are vast. People think money can be regenerated; it can. But what Americans have seemingly forgotten is the past where American autoworkers feared Japanese efficiency and where the Japanese economy looked to crush ours. Today, the Japanese salaryman works at least 50% more hours and his purchasing power is less than his counterpart in the United States (just look at patio11's posts on it). https://www.kalzumeus.com/2014/11/07/doing-business-in-japan/ https://www.kalzumeus.com/2014/11/07/doing-business-in-japan... It's not a popular take because of recency bias (the same thing happened on 9/11 with the PATRIOT Act and always doe when outliers happen), but the data will be pretty clear in the end.

cycomanic 6y ago

What does useful mean? I would argue most models are useless without a good understanding of the parameters going into that model. That is where the domainspecific knowledge comes in. Fitting some curve to data is useless unless you know what the parameters correspond to, and how they influence the parameters of the curve.