4 ms·
A/A Testing
- n4jm4 4y agoOne can commit to a performance metric threshold prior to A/B testing. For example, a rule that states "New, nonmaintenance features must increase revenue for the product by at least 1.0%". An objective rule like that neatly resolves the problem of biasing analysis after the test, and doesn't have to necessarily involve any complicated statistics.
- travelhead 4y agoOur favorite test is A, A, B, B It’s surprising how often a test could be set up incorrectly or have other technical issues which this test can catch. We try to ensure a statistically significant result on both A’s or both B’s before determining a winner. It does take longer to run this test, but should give a much higher degree of accuracy.