2 ms·
The p-value is supposed to measure how likely you were to get a result by pure chance. People use this as a measure of how strong the evidence published in some
by wongarsu 3y ago
The p-value is supposed to measure how likely you were to get a result by pure chance. People use this as a measure of how strong the evidence published in some paper is, for example if you publish a result that has a 1% probability of being produced by pure chance it's probably worth looking into.
The problem is if what you actually did wasn't one experiment, but 1000 simultaneous experiments. That happens quickly if you start looking for correlations in your data. The issue is that if you then use the same 1% threshold for significance, on average 10 experiments will clear that threshold by pure chance. Your statistical test has become worthless. That's fine if you account for it, but if you don't and just publish those 10 results that's called p-hacking. People hate it because it makes purely random results look as if they were significant.