3 ms·
>We should be dumping these whole datasets into machine learning and having computers give us potential links to explore. You're describing P-value hacking, th
by kharms 9y ago
>We should be dumping these whole datasets into machine learning and having computers give us potential links to explore.
You're describing P-value hacking, thus named because hack scientists use this technique to publish papers about nonsense.
- highd 9y agoYou basically just need to reduce the P-value you need to claim significance (0.1 to 0.001 or less) to account for the probability of finding those correlations even in noise. This is part of why particle physics has such high standards - you can find a lot of things in the TBs of data CERN generates. See for example: https://en.wikipedia.org/wiki/Genome-wide_association_study https://en.wikipedia.org/wiki/Genome-wide_association_study There's a figure in there depicting associations with P-values of 1e-8: https://en.wikipedia.org/wiki/Genome-wide_association_study#/media/File:Regional_Association_Plot.png https://en.wikipedia.org/wiki/Genome-wide_association_study#...