3 ms·

It seems important to note that this was a study of 35 people. 35. That seems like a surprisingly low number from which to draw broad conclusions of brain act

by davidu 12y ago

It seems important to note that this was a study of 35 people. 35. That seems like a surprisingly low number from which to draw broad conclusions of brain activity for adults diagnosed with ADHD when they were children.

saosebastiao 12y ago
It definitely feels that way to me too, but I've been told by a neuroscience researcher that a lot of neuroscience research is done with small sample sizes. She even showed me one of her journals, and I didn't find a single paper that had more than 50 participants.
- angersock 12y ago
  ...which makes you wonder about the quality of medicinal and health research as a whole.
  - mreiland 12y ago
    Mathematically, 10 is generally considered enough to make broad conclusions, more is always better though. OTOH, that assumes a fairly uniform random sampling. The reason more is better is because it helps leverage against the non-uniform random sampling.
    - Loughla 12y ago
      I honestly don't understand how 10 people is a large enough sample to make broad conclusions about health and medicine for 7 billion people. Can you please explain the statistics behind that?
      - nerfhammer 12y ago
        If you hypothesize that the odds of a coin flipping heads is 50% and you get heads 9 out of 10 times, the odds of getting that result if the odds were really 50% is like 1/500. You should be pretty confident that if you were the flip the coin billions of times you wouldn't get close to 50% heads based on looking at 10 samples.
        scott_karana 12y ago
        Do you not think selection bias could be a major factor? Hypothetically, if this study was undertaken in small-town Wisconsin, and genetic factors could even remotely be involved, the samples wouldn't be representative in the slightest for most genes in the world.
        mreiland 12y ago
        It absolutely is a factor, which is why I cautioned about a uniform, random sampling. Selection bias implies non-uniformity, and 10, 100, or 1000 samples won't fix selection bias since it is, by definition, non-uniform across the intended subjects. It's good that you question it, it should always be one of the first questions asked when looking at these sorts of statistics. I was just pointing out that the number 35, by itself, is plenty to justify a conclusion, and because perfect uniformity is not completely possible, more tends to be better. If the conclusion is faulty, it isn't typically due to the sample size of 35 being too low.
      - surement 12y ago
        http://en.wikipedia.org/wiki/Statistical_power http://en.wikipedia.org/wiki/Statistical_power
      - derefr 12y ago
        In psychopathology specifically, though, you're not making inferences about 7 billion people; you're just making inferences about the population that actually has the disease. If only three people in the world have ever had Rudolff's Aphasia, then you'll want to study them to help the fourth guy down the line.
bigchewy 12y ago
studies are expensive so they end up with small sample sizes. Add in the push to only publish exciting findings and you start to see why so much research is later invalidated.
mantrax5 12y ago
They wanted to study more people, but got distracted.
thefreeman 12y ago
Another unfortunate issue is we don't have fMRI's of these people when they were kids and originally diagnosed with ADHD. And it is a disease that is known to be / have been overdiagnosed a lot. It's quite possible that a number of these "cured" people were just misdiagnosed in the first place.
sliverstorm 12y ago
Unlike software, it's time-consuming and expensive to test people. Most speculative studies on humans seem to start with a small group, and if they think they found something they move up to larger groups. 35 isn't all that terrible, either. An estimated 10 million adults have ADHD (although that sounds bogus to me), and plugging that into a calculator with a 95% confidence, that gives a confidence interval of 16.5. Not sufficient for definitive proof of anything, but still meaningful.
elchief 12y ago
http://stats.stackexchange.com/questions/37993/is-there-a-minimum-sample-size-required-for-the-t-test-to-be-valid http://stats.stackexchange.com/questions/37993/is-there-a-mi... "Historically, the very first demonstration of the t-test (in "Student"'s 1908 paper) was in an application to sample sizes of size four. Indeed, obtaining improved results for small samples is the test's claim to fame"

by davidu 12y ago

saosebastiao 12y ago

It definitely feels that way to me too, but I've been told by a neuroscience researcher that a lot of neuroscience research is done with small sample sizes. She even showed me one of her journals, and I didn't find a single paper that had more than 50 participants.

angersock 12y ago
...which makes you wonder about the quality of medicinal and health research as a whole.
- mreiland 12y ago
  Mathematically, 10 is generally considered enough to make broad conclusions, more is always better though. OTOH, that assumes a fairly uniform random sampling. The reason more is better is because it helps leverage against the non-uniform random sampling.
  - Loughla 12y ago
    I honestly don't understand how 10 people is a large enough sample to make broad conclusions about health and medicine for 7 billion people. Can you please explain the statistics behind that?
    - nerfhammer 12y ago
      If you hypothesize that the odds of a coin flipping heads is 50% and you get heads 9 out of 10 times, the odds of getting that result if the odds were really 50% is like 1/500. You should be pretty confident that if you were the flip the coin billions of times you wouldn't get close to 50% heads based on looking at 10 samples.
      - scott_karana 12y ago
        Do you not think selection bias could be a major factor? Hypothetically, if this study was undertaken in small-town Wisconsin, and genetic factors could even remotely be involved, the samples wouldn't be representative in the slightest for most genes in the world.
        mreiland 12y ago
        It absolutely is a factor, which is why I cautioned about a uniform, random sampling. Selection bias implies non-uniformity, and 10, 100, or 1000 samples won't fix selection bias since it is, by definition, non-uniform across the intended subjects. It's good that you question it, it should always be one of the first questions asked when looking at these sorts of statistics. I was just pointing out that the number 35, by itself, is plenty to justify a conclusion, and because perfect uniformity is not completely possible, more tends to be better. If the conclusion is faulty, it isn't typically due to the sample size of 35 being too low.
    - surement 12y ago
      http://en.wikipedia.org/wiki/Statistical_power http://en.wikipedia.org/wiki/Statistical_power
    - derefr 12y ago
      In psychopathology specifically, though, you're not making inferences about 7 billion people; you're just making inferences about the population that actually has the disease. If only three people in the world have ever had Rudolff's Aphasia, then you'll want to study them to help the fourth guy down the line.

bigchewy 12y ago

studies are expensive so they end up with small sample sizes. Add in the push to only publish exciting findings and you start to see why so much research is later invalidated.

mantrax5 12y ago

They wanted to study more people, but got distracted.

thefreeman 12y ago

Another unfortunate issue is we don't have fMRI's of these people when they were kids and originally diagnosed with ADHD. And it is a disease that is known to be / have been overdiagnosed a lot. It's quite possible that a number of these "cured" people were just misdiagnosed in the first place.

sliverstorm 12y ago

Unlike software, it's time-consuming and expensive to test people. Most speculative studies on humans seem to start with a small group, and if they think they found something they move up to larger groups. 35 isn't all that terrible, either. An estimated 10 million adults have ADHD (although that sounds bogus to me), and plugging that into a calculator with a 95% confidence, that gives a confidence interval of 16.5. Not sufficient for definitive proof of anything, but still meaningful.

elchief 12y ago

http://stats.stackexchange.com/questions/37993/is-there-a-minimum-sample-size-required-for-the-t-test-to-be-valid http://stats.stackexchange.com/questions/37993/is-there-a-mi... "Historically, the very first demonstration of the t-test (in "Student"'s 1908 paper) was in an application to sample sizes of size four. Indeed, obtaining improved results for small samples is the test's claim to fame"