3 ms·

I'm essentially right. Studies with less than 200 people are essentially useless. It is more complicated than I originally let on, but not so much more complica

I'm essentially right. Studies with less than 200 people are essentially useless. It is more complicated than I originally let on, but not so much more complicated to muck up the takeaway. Most of our junk science has to do with these low population studies or with studies simultaneously studying high numbers of attributes.

sov 9y ago
Err, I don't quite agree with your description. The takeaway from your post was that low-n is worthless and high-n is ineffable, full stop. The size of the trial has very little to do with whether or not we are to believe the results insofar as it affects its statistical power. The fact that p-hacking and selection bias exist doesn't immediately imply that low-n studies are wrong. Those aren't problems unique to small sample size studies. For sure, statistical power isn't the catch-all thing to examine either--rather, pre-registration of studies would alleviate a great deal of falsities, but none of any of this has to do strictly with low sample size trials as your post implies. Let's say the drug studied was a guaranteed 100% limb regenerative drug for amputees. Would you really require a 200 person study to prove that Examplinol successfully regenerates limbs? I hope the answer to that is "obviously not; it will be very clear whether or not it works with a low sample size." Of course, how would we change the study if it wasn't supposed to work in 100% of people? What if Sample Pharmaceuticals indicated that it only worked for 50% of people? Or 10%? Or 1%? Would it suffice to use 200 people each trial?