3 ms·
Or more concretely, since these projects are typically running on a hypothesis-testing paradigm, you can do a hybrid power analysis: compute a fully Bayesian an
by cba9 11y ago
Or more concretely, since these projects are typically running on a hypothesis-testing paradigm, you can do a hybrid power analysis: compute a fully Bayesian analysis of the original experiment using other experiments and informative priors which take into account the true distribution of effects in a particular subfield (eg a broad distribution with many large effects if it's related to IQ, or a narrow distribution around zero if it's related to things like priming or stereotype thread) to generate a posterior distribution of effect sizes.
Then you can simulate the results of future experiments with _n_ datapoints: sample 1 effect size from that distribution, generate _n_ datapoints assuming that effect size, run the hypothesis-testing, and return the _p_-value.
The fraction of _p_<=0.05 is your best forecast of whether the future experiment will succeed in reproducing it or not.