>This was a fairly large change that we, understandably, only wanted to deploy to a small subset of users. This was facilitated by Bengineer Kamen's GAE/Bingo s

>This was a fairly large change that we, understandably, only wanted to deploy to a small subset of users. This was facilitated by Bengineer Kamen's GAE/Bingo split-testing framework for App Engine. I think this method of A/B testing has some faults. I blogged about it A/B testing. Is Khan doing it wrong?http://liveatthewitchtrials.blogspot.com/2011/09/ab-testing-is-khan-doing-it-wrong.html http://liveatthewitchtrials.blogspot.com/2011/09/ab-testing-... and Allen Downey ran some simulations at Repeated tests: how bad can it be?http://allendowney.blogspot.com/2011/10/repeated-tests-how-bad-can-it-be.html http://allendowney.blogspot.com/2011/10/repeated-tests-how-b...