3 ms·

What does that sample size tell you?

by tinco 6y ago

martin_a 6y ago
Sample size is an important metric in statistics. It defines how many people were tested/asked/whatever. Higher numbers are better because that levels out quirks that might have happened. Asking 59 people is obviously not a big sample size and therefore the results of this study are "not very reliable". To say it friendly.
- Arnt 6y ago
  The sample size isn't a problem here, because it doesn't matter: Even a 59,000-person sample would not overcome the 2-5 year delay. And of course the researchers knew that. They weren't even trying to do what you imply. They weren't looking for reliably anything. Rather, they were looking for something interesting, along the lines of "are these interesting questions to pose in a much more expensive study that avoids the 2-5-year problem?"
  - notafraudster 6y ago
    Agreed with your push-back on the sample size thing being operative (this is a point I've made frequently on HN study threads), and agree that the primary design consideration is the post-"treatment" measurement of the outcome making it more likely that having an autistic kid changes metabolites than that having different metabolites influences chance of having an autistic kid. Still, even with contemporaneous measurement, my guess is that there'd be a very real selection issue here; a non-random subset of people who could receive an autism diagnosis do, and to the extent the confounders into diagnosis (one presumes all the standard SES/race/access to healthcare factors are confounders here) constitutes a selection on the dependent variable, even a contemporaneous sample would not generalize to the population of interest.
- rich_sasha 6y ago
  I don't get the downvotes. It is nigh-impossible to conclude something from a tiny sample, other than it's worth looking at more. But there are tons of things worth looking at more, most of which come out to nothing. Covid produced so many small-sample "promising effect" papers, most of which came to nothing at all. It's not a critique of the study, either. You can't recruit tens of thousands of people for every little study. So to rephrase, the headline could read "we found a relationship that probably doesn't actually hold, but just might".
  - tinco 6y ago
    59 is not a tiny sample, they are being downvoted because they're demonstrating a lack of understanding of significance in statistics. The researchers conclude "Mothers of children with ASD have many significantly different metabolite levels compared to mothers of typically developing children at 2–5 years after birth." And they back that conclusion up by describing their analytical process. If GP wants to dispute this conclusion, then the GP should explain why their method is flawed.

What does that sample size tell you?

by tinco 6y ago

What does that sample size tell you?

Sample size is an important metric in statistics. It defines how many people were tested/asked/whatever. Higher numbers are better because that levels out quirks that might have happened. Asking 59 people is obviously not a big sample size and therefore the results of this study are "not very reliable". To say it friendly.

Arnt 6y ago
The sample size isn't a problem here, because it doesn't matter: Even a 59,000-person sample would not overcome the 2-5 year delay. And of course the researchers knew that. They weren't even trying to do what you imply. They weren't looking for reliably anything. Rather, they were looking for something interesting, along the lines of "are these interesting questions to pose in a much more expensive study that avoids the 2-5-year problem?"
- notafraudster 6y ago
  Agreed with your push-back on the sample size thing being operative (this is a point I've made frequently on HN study threads), and agree that the primary design consideration is the post-"treatment" measurement of the outcome making it more likely that having an autistic kid changes metabolites than that having different metabolites influences chance of having an autistic kid. Still, even with contemporaneous measurement, my guess is that there'd be a very real selection issue here; a non-random subset of people who could receive an autism diagnosis do, and to the extent the confounders into diagnosis (one presumes all the standard SES/race/access to healthcare factors are confounders here) constitutes a selection on the dependent variable, even a contemporaneous sample would not generalize to the population of interest.
rich_sasha 6y ago
I don't get the downvotes. It is nigh-impossible to conclude something from a tiny sample, other than it's worth looking at more. But there are tons of things worth looking at more, most of which come out to nothing. Covid produced so many small-sample "promising effect" papers, most of which came to nothing at all. It's not a critique of the study, either. You can't recruit tens of thousands of people for every little study. So to rephrase, the headline could read "we found a relationship that probably doesn't actually hold, but just might".
- tinco 6y ago
  59 is not a tiny sample, they are being downvoted because they're demonstrating a lack of understanding of significance in statistics. The researchers conclude "Mothers of children with ASD have many significantly different metabolite levels compared to mothers of typically developing children at 2–5 years after birth." And they back that conclusion up by describing their analytical process. If GP wants to dispute this conclusion, then the GP should explain why their method is flawed.