3 ms·

Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons

Outliers, sure. Exactly like this though? That depends. Since this is a population then assuming a gaussian (which could well be wrong for a variety of reasons I'm just guessing here) what's sigma? If 1/100 is within ~3 sigma then sure. Otherwise no.

bluGill 2mo ago
It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.
- fc417fc802 2mo ago
  You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset. To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check. Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
  - bluGill 2mo ago
    The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.
roenxi 2mo ago
It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value. In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01. [0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...

bluGill 2mo ago

It needs more statistical analysis than I have the background to do. However all signs look to another variation of the replication crisis, which is more an more my default assumption when I see an observational study.

fc417fc802 2mo ago
You're handwaving. The xkcd point only applies if you're within ~3 sigma. This isn't a complicated statistical analysis it's just asking how far out the outlier is versus the standard deviation and size of the dataset. To be clear I'm not claiming in one direction or the other. I feel that if you're going to attempt to discredit a piece of work it's on you to do such a basic check. Really the whole point of that xkcd was to viscerally illustrate the practical effect of binary misclassification, or alternatively to make the point that a p of 0.05 isn't necessarily as rigorous as you might expect. Neither of those things directly applies here.
- bluGill 2mo ago
  The article doesn't give enough information to answer your question. It isn't on me it is on the reporter's who shouldn't allow a handwave like that to pass without question. There are far to many cases of people hacking statistics in the world to accept that.

roenxi 2mo ago

It is an outcome from over 400 data points. We're expecting to see events outside a 3 sigma boundary. In fact, 3 sigma is basically the 1:400 boundary. Strictly it is 1 in 370 [0] for anyone who takes Wikipedia at face value. In the study they seem put the P value of the taxi driver outcome at <0.01 (in Table 2, although I admit I haven't read the thing especially closely). For 400 data points, we expect there to be around 4 values with a P value less than <0.01. [0] https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule#Table_of_numerical_values https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rul...