5 ms·

> what specifically leads to such bold conclusions (e.g. "owe an apology", "wasted time and effort", "statistical error", "screwed up", etc.)? They used a test

by knzhou 6y ago

> what specifically leads to such bold conclusions (e.g. "owe an apology", "wasted time and effort", "statistical error", "screwed up", etc.)? They used a test with a false positive rate of about 2%, and found a positive rate of about 2%. It is impossible to use this information to establish a lower bound on the number infected. However, by screwing up the computation of the confidence interval, they claimed a huge lower bound.

statquontrarian 6y ago
Thanks. By the way, the STAT article links to https://undark.org/2020/04/24/john-ioannidis-covid-19-death-rate-critics/ https://undark.org/2020/04/24/john-ioannidis-covid-19-death-... in which Ioannidis is quoted: > “We have been working very hard to address all the comments raised by many colleagues and I am always grateful to receive constructive criticism,” he said. “While we received thousands of congratulations for this work, I cherish far more the comments of the scientists who criticized our work and I thank them deeply for their comments.”
- knzhou 6y ago
  That's an admirable sentiment, Ionannidis must be such a wonderful guy. However, the damage has already been done. He's already done his media blitz, so no matter what happens next, effectively zero people will hear the correction (or more likely, subsequent retraction). It'll just become yet another piece of zombie information, which will appear in dishonest talking points or gish gallops. The quality of public discourse and the reputation of science will continue its decline, and we'll be talking about the next example of this tomorrow.
  - statquontrarian 6y ago
    If we try to be charitable, Ioannidis was too confident in his research team who made mistakes; for example: > The authors said by email that they used a built-in Stata function and aren't sure themselves how the software used the input weights. I suspect they misapplied that function (too complicated to tweet why) but I don't know Stata well enough to be sure; it seems neither do they. https://twitter.com/wfithian/status/1252692357788479488 https://twitter.com/wfithian/status/1252692357788479488 Then, Ioannidis probably didn't double check the work in depth because of confirmation bias and over-confidence in his protégés.
statquontrarian 6y ago
I checked with a friend who has a PhD in statistics specifically about this Twitter thread (which was referenced in some of the articles about this): https://twitter.com/wfithian/status/1252692357788479488 https://twitter.com/wfithian/status/1252692357788479488 One claim Will Fithian makes is: > The errors are not debatable and can be seen in these two screenshots of the supplement: 0.0034, the standard error meant to measure uncertainty about prevalence pi, is not the square root of 0.039, and the variance of a binomial estimate of proportion depends on the sample size. My statistician friend says: > In fact the math checks out in favor of the authors. Square root of .039, then divide by square root of 3330. .= 0034 just like reported. > So the error is debatable. > Now, sometimes you should be using the standard deviation, not the standard error, but that isn't his claim. The .0034 is absolutely the correct standard error, unless there is a difference in terminology for the field or something.

by knzhou 6y ago

statquontrarian 6y ago

Thanks. By the way, the STAT article links to https://undark.org/2020/04/24/john-ioannidis-covid-19-death-rate-critics/ https://undark.org/2020/04/24/john-ioannidis-covid-19-death-... in which Ioannidis is quoted: > “We have been working very hard to address all the comments raised by many colleagues and I am always grateful to receive constructive criticism,” he said. “While we received thousands of congratulations for this work, I cherish far more the comments of the scientists who criticized our work and I thank them deeply for their comments.”

knzhou 6y ago
That's an admirable sentiment, Ionannidis must be such a wonderful guy. However, the damage has already been done. He's already done his media blitz, so no matter what happens next, effectively zero people will hear the correction (or more likely, subsequent retraction). It'll just become yet another piece of zombie information, which will appear in dishonest talking points or gish gallops. The quality of public discourse and the reputation of science will continue its decline, and we'll be talking about the next example of this tomorrow.
- statquontrarian 6y ago
  If we try to be charitable, Ioannidis was too confident in his research team who made mistakes; for example: > The authors said by email that they used a built-in Stata function and aren't sure themselves how the software used the input weights. I suspect they misapplied that function (too complicated to tweet why) but I don't know Stata well enough to be sure; it seems neither do they. https://twitter.com/wfithian/status/1252692357788479488 https://twitter.com/wfithian/status/1252692357788479488 Then, Ioannidis probably didn't double check the work in depth because of confirmation bias and over-confidence in his protégés.

statquontrarian 6y ago

I checked with a friend who has a PhD in statistics specifically about this Twitter thread (which was referenced in some of the articles about this): https://twitter.com/wfithian/status/1252692357788479488 https://twitter.com/wfithian/status/1252692357788479488 One claim Will Fithian makes is: > The errors are not debatable and can be seen in these two screenshots of the supplement: 0.0034, the standard error meant to measure uncertainty about prevalence pi, is not the square root of 0.039, and the variance of a binomial estimate of proportion depends on the sample size. My statistician friend says: > In fact the math checks out in favor of the authors. Square root of .039, then divide by square root of 3330. .= 0034 just like reported. > So the error is debatable. > Now, sometimes you should be using the standard deviation, not the standard error, but that isn't his claim. The .0034 is absolutely the correct standard error, unless there is a difference in terminology for the field or something.