That paper is a very different setup --- the hierarchical model Gelman, Hill, and Yajima use implicitly builds multiple comparisons into their process. You coul

That paper is a very different setup --- the hierarchical model Gelman, Hill, and Yajima use implicitly builds multiple comparisons into their process. You could do the same thing here in principle by drawing alpha and beta for different experiments from an underlying distribution, but I'd be cautious about applying that approach mindlessly. If you haven't read it, Gelman and Hill's book is excellent: http://www.stat.columbia.edu/~gelman/arm/ http://www.stat.columbia.edu/~gelman/arm/ and so is Gelman's blog.