4 ms·

>Warning Sign D3: Too Few Subjects >The author advertising smart drugs doesn't describe any careful studies, so of course he doesn't mention the number of subje

by Permit 14y ago

>Warning Sign D3: Too Few Subjects >The author advertising smart drugs doesn't describe any careful studies, so of course he doesn't mention the number of subjects in any studies. Most so-called studies of smart drugs have been far too small to have any statistical power. Neither yourself, nor Peter Norvig suggested an appropriate number of subjects for tests like these. So I'm curious, what would be an appropriate number of subjects?

pnathan 14y ago
> So I'm curious, what would be an appropriate number of subjects? This depends on the experiment. Some of the things to account for are the number of variables, the confidence you are looking for and the amount of interdependence. Consult a statistician, particularly one expert in experimental trials, for an evaluation.
stephengillie 14y ago
https://en.wikipedia.org/wiki/Sample_size#Required_sample_sizes_for_hypothesis_tests https://en.wikipedia.org/wiki/Sample_size#Required_sample_si...
tel 14y ago
Rule of thumb, the larger the effect size, the fewer the number of participants. The larger the random variation across people the greater the number of participants. There are also exact formulas for simple tests.
lars 14y ago
You test some statistical hypothesis, and you end with some probability that the hypethesis is true. Increasing the number of people increases the number of samples. Loosely speaking, increasing the number of samples will reduce the variance on the variable you are measuring. When the variance is reduced, the confidence in the result is increased. Say you are measuring IQ before and after taking omega 3. Say your hypthesis is "omega 3 immedieately increases IQ". Assume this is true, even if it probably isn't. After testing this on 20 people you might be able to say "the hypthesis appears to be true, but there is a 46% percent chance these results are due to chance". However after testing 1000 people, you might be able to say "the hypthesis appears to be true, and there is a 0.5% percent chance these results are due to chance". Then you are 99.5% confident in your result. Standards for confidence varies from field to field, but typically 99% confidence or more is called a significant result. Given some assumptions, you can also calculate the number of samples needed to generate a result with confidence >X% with probability Y. According to Daniel Kahneman in Thinking Fast and Slow, most researchers don't take the time to do this, and are consistently surprised when their experiments turn out to not give significant results.
- deleted 14y ago
  [deleted]
- Evbn 14y ago
  This post, like so many others, states the p-value concept backwards. Statistics give you the chance of a false positive observation among a universe of negative realities, not the chance of a false reality in a universe of positive observations.

by Permit 14y ago

pnathan 14y ago

> So I'm curious, what would be an appropriate number of subjects? This depends on the experiment. Some of the things to account for are the number of variables, the confidence you are looking for and the amount of interdependence. Consult a statistician, particularly one expert in experimental trials, for an evaluation.

stephengillie 14y ago

https://en.wikipedia.org/wiki/Sample_size#Required_sample_sizes_for_hypothesis_tests https://en.wikipedia.org/wiki/Sample_size#Required_sample_si...

tel 14y ago

Rule of thumb, the larger the effect size, the fewer the number of participants. The larger the random variation across people the greater the number of participants. There are also exact formulas for simple tests.

lars 14y ago

You test some statistical hypothesis, and you end with some probability that the hypethesis is true. Increasing the number of people increases the number of samples. Loosely speaking, increasing the number of samples will reduce the variance on the variable you are measuring. When the variance is reduced, the confidence in the result is increased. Say you are measuring IQ before and after taking omega 3. Say your hypthesis is "omega 3 immedieately increases IQ". Assume this is true, even if it probably isn't. After testing this on 20 people you might be able to say "the hypthesis appears to be true, but there is a 46% percent chance these results are due to chance". However after testing 1000 people, you might be able to say "the hypthesis appears to be true, and there is a 0.5% percent chance these results are due to chance". Then you are 99.5% confident in your result. Standards for confidence varies from field to field, but typically 99% confidence or more is called a significant result. Given some assumptions, you can also calculate the number of samples needed to generate a result with confidence >X% with probability Y. According to Daniel Kahneman in Thinking Fast and Slow, most researchers don't take the time to do this, and are consistently surprised when their experiments turn out to not give significant results.

deleted 14y ago
[deleted]
Evbn 14y ago
This post, like so many others, states the p-value concept backwards. Statistics give you the chance of a false positive observation among a universe of negative realities, not the chance of a false reality in a universe of positive observations.