3 ms·
When you have large amounts of data, even if it's for a large number of people eventually youre going to find odd 'correlations' between unconnected variables.
by simonh 9y ago
When you have large amounts of data, even if it's for a large number of people eventually youre going to find odd 'correlations' between unconnected variables. People who play edition X of Dungeons and Dragons are 30% less likely to eat chicken sandwitches than people who play edition Y. If you know a thousand things about everybody, the data is going to be riddled with bogus 'connections' like this.
In this case it's too small a sample for this finding to be significant, but increase the sample size and also increase the number of data points per person and you get similar problems. Determining significance is not straightforward.
- var_explained 9y agoThis is not the reason for the correlation. The p-value of the tabs/spaces connection is about 10^-13 (one in 10 trillion), and it can be found separately within each (large) country. This does not mean the effect isn't confounded with some other factor, but it does mean it's not a multiple hypothesis testing issue. You can explore the code and regression yourself! Take a look: https://github.com/dgrtwo/tabs-spaces-post https://github.com/dgrtwo/tabs-spaces-post
- sigstoat 9y agoyup, the multiple comparisons problem: https://en.wikipedia.org/wiki/Multiple_comparisons_problem https://en.wikipedia.org/wiki/Multiple_comparisons_problem a common reason for a study to be total junk.
- mrkgnao 9y agohttp://www.tylervigen.com/spurious-correlations http://www.tylervigen.com/spurious-correlations