3 ms·
Put simply: if you test a random event often enough, you're going to get lucky eventually. In statistics, the question everybody asks is whether or not a given
by veddox 9y ago
Put simply: if you test a random event often enough, you're going to get lucky eventually.
In statistics, the question everybody asks is whether or not a given result is "significant". To answer this question, one calculates a p-value, which gives you the probability that your data are purely random (i.e. not due to any causal factor). By convention, if p < 0.05, one calls this result "significant".
There are various tests to calculate this p-value, depending on the type of data you have available. One of the most common types are t-tests, which are used when you wish to compare two samples. (For example, you might want to compare the number of butterfly species on mountain meadows with the number of species in vineyards.) The t-test takes various factors into account, including the mean and the variance of your datasets. This makes it a pretty good tool for comparison, much better than looking merely at, say, the arithmetic mean.
The problem is that occasionally, the t-test is going to give you a false positive. That means you end up thinking that two things are causally linked that in reality are completely independent. The probability of such a wrong result rises with the number of tests you perform in a given study. That is why such studies need to adjust for multiple testing. Often this is done with the Bonferroni method, which basically means multiplying your p-value by a/n, where a is your level of significance (0.05) and n is the number of tests you perform.
Hope that made sense :-)
- kerkeslager 9y agoThe concept I was missing was that testing two different random variables at the same time (do meat or vegetables affect telomere length in this group) has the same probability of yielding a false positive for one of the variables as testing the same variable twice (does meat affect telomere length in this group, does meat affect telomere length in that group) has of yielding a false positive for one of the tests.