3 ms·
You test some statistical hypothesis, and you end with some probability that the hypethesis is true. Increasing the number of people increases the number of sam
by lars 14y ago
You test some statistical hypothesis, and you end with some probability that the hypethesis is true. Increasing the number of people increases the number of samples. Loosely speaking, increasing the number of samples will reduce the variance on the variable you are measuring. When the variance is reduced, the confidence in the result is increased.
Say you are measuring IQ before and after taking omega 3. Say your hypthesis is "omega 3 immedieately increases IQ". Assume this is true, even if it probably isn't. After testing this on 20 people you might be able to say "the hypthesis appears to be true, but there is a 46% percent chance these results are due to chance". However after testing 1000 people, you might be able to say "the hypthesis appears to be true, and there is a 0.5% percent chance these results are due to chance". Then you are 99.5% confident in your result. Standards for confidence varies from field to field, but typically 99% confidence or more is called a significant result. Given some assumptions, you can also calculate the number of samples needed to generate a result with confidence >X% with probability Y. According to Daniel Kahneman in Thinking Fast and Slow, most researchers don't take the time to do this, and are consistently surprised when their experiments turn out to not give significant results.
- deleted 14y ago[deleted]
- Evbn 14y agoThis post, like so many others, states the p-value concept backwards. Statistics give you the chance of a false positive observation among a universe of negative realities, not the chance of a false reality in a universe of positive observations.