3 ms·
By the way, as part of a research project which will (hopefully) become public soon, I've been thoroughly investigating these issues. It's important to note tha
by yummyfajitas 12y ago
By the way, as part of a research project which will (hopefully) become public soon, I've been thoroughly investigating these issues. It's important to note that Bayesian and Sequential testing are orthogonal.
You can have a sequential frequentist test (typically the Sequential Probability Ratio Test), which is standard frequentist hypothesis testing that allows multiple looks.
Similarly, you can use a Bayesian test in a "deadbeat" (run the test out to N=10,000, stop and choose the best) manner - you simply use the posterior to determine which variant is superior (i.e. choose the one with the highest mean).
tl;dr; The core difference between deadbeat and sequential is whether or not you stop the test at a fixed time. The core difference between Bayesian and Frequentist (as described in this article) is whether you maximize a utility function (e.g. revenue) or attempt to determine "truth".