3 ms·

Unless I'm missing something, that statement appears to be correct. They don't say their result isn't erroneous. They say there is statistically only a very tin

Unless I'm missing something, that statement appears to be correct. They don't say their result isn't erroneous. They say there is statistically only a very tiny chance that the correlation is due to chance. If the correlation is caused by an error in their experiment, their statement is still correct, since the correlation was not due to chance.

whimsicalism 3y ago
Did they hypothesize this effect beforehand? If not, this was a data collection and analysis exercise - not an experiment. How many comparisons did they do before stumbling on this result? Unreported.
- nearbuy 3y ago
  They could be p-hacking, though getting 6 sigmas through p-hacking without outright fraudulent data is actually some pretty impressive level of p-hacking. I'd say it's more likely they either made an error or there really is a correlation.
mustacheemperor 3y ago
At the top of the second page of that pdf: >P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone. It was very informative for me. Worth reading.
- nearbuy 3y ago
  Without getting nitpicky to the point where it's impossible to say anything at all without complaining it's not a perfectly complete and true statement, their statement still seems correct to me. Especially given that this is a statement for non-statisticians and they can't exactly go on a 5 minute tangent to explain exactly how p-values work. Perhaps it would have been more accurate if they phrased it: "which means a chance of less than one in a billion of seeing a correlation this strong if it's due to chance", but I think few readers would even notice the difference.
mjhay 3y ago
>They don't say their result isn't erroneous. They say there is statistically only a very tiny chance that the correlation is due to chance. Again, a p-value is not the chance that a result is erroneous - the statement you made is 100% mathematically incorrect. That was a major point of my previous post that you missed. It's a common misconception that the authors of this work are victims of. The linked PDF from the American Statistical Society explains things in more detail.
- nearbuy 3y ago
  You misread me. I said that no one said a p-value is the chance the result is erroneous. They said nothing about errors whatsoever in the quoted statement. Apologies for the double negative in my last comment.

whimsicalism 3y ago

Did they hypothesize this effect beforehand? If not, this was a data collection and analysis exercise - not an experiment. How many comparisons did they do before stumbling on this result? Unreported.

nearbuy 3y ago
They could be p-hacking, though getting 6 sigmas through p-hacking without outright fraudulent data is actually some pretty impressive level of p-hacking. I'd say it's more likely they either made an error or there really is a correlation.

mustacheemperor 3y ago

At the top of the second page of that pdf: >P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone. It was very informative for me. Worth reading.

nearbuy 3y ago
Without getting nitpicky to the point where it's impossible to say anything at all without complaining it's not a perfectly complete and true statement, their statement still seems correct to me. Especially given that this is a statement for non-statisticians and they can't exactly go on a 5 minute tangent to explain exactly how p-values work. Perhaps it would have been more accurate if they phrased it: "which means a chance of less than one in a billion of seeing a correlation this strong if it's due to chance", but I think few readers would even notice the difference.

mjhay 3y ago

>They don't say their result isn't erroneous. They say there is statistically only a very tiny chance that the correlation is due to chance. Again, a p-value is not the chance that a result is erroneous - the statement you made is 100% mathematically incorrect. That was a major point of my previous post that you missed. It's a common misconception that the authors of this work are victims of. The linked PDF from the American Statistical Society explains things in more detail.

nearbuy 3y ago
You misread me. I said that no one said a p-value is the chance the result is erroneous. They said nothing about errors whatsoever in the quoted statement. Apologies for the double negative in my last comment.