3 ms·
You can take an adaptive approach, but almost universally it is done wrong. https://www.lucidchart.com/blog/2016/10/20/the-fatal-flaw-of-ab-tests-peeking/ https
by paulddraper 10y ago
You can take an adaptive approach, but almost universally it is done wrong. https://www.lucidchart.com/blog/2016/10/20/the-fatal-flaw-of-ab-tests-peeking/ https://www.lucidchart.com/blog/2016/10/20/the-fatal-flaw-of...
FYI, the difference with Google Analytics and most other A/B platforms is that GA uses multi-armed bandit. It adjusts the proportions according to the results. This is nice because it effectively ends the test for you, and you don't have to stress about suboptimal success rates just for the purpose of experimentation. https://support.google.com/analytics/answer/2844870 https://support.google.com/analytics/answer/2844870