4 ms·
For that matter, very small studies are safer from certain types of false positive. The file-drawer problem with small lab studies is serious, and can't be sol
by Bartweiss 8y ago
For that matter, very small studies are safer from certain types of false positive.
The file-drawer problem with small lab studies is serious, and can't be solved with effect sizes; it's much easier to run a few of these and publish the hits than to do that with large-population studies.
But small lab studies provide a lot of value alongside population studies. They help reveal issues with bad self-report data, which are major concerns for sleep and weight questions. They also offer an efficient way to replicate and extend large studies. If you're spending a decade with 10k+ subjects, gathering a huge range of data is understandable (and I would argue, preferable). But it leaves you with a choice between salami slicing across all your variables, or applying multiple-comparisons corrections that destroy your sensitivity. Using large multi-factor studies as exploratory work, then validating effects with small focused studies is actually a pretty healthy outcome.
A final thought: I have a lot of problems with this study, but file-drawer issues aren't one of them. Looking at a bunch of correlated variables (insulin sensitivity, weight gain, outside-meal eating, etc) and reporting them all is a solid way to mitigate that - we know those things should correlate for basically any real effect, which makes it much harder to stumble into a false positive and publish.