3 ms·
When ExP was a thing at Microsoft, we always ran an A/A test before we did experiments. We'd also do an A/A/B test to make sure the actual experiments were work
by RyJones 13y ago
When ExP was a thing at Microsoft, we always ran an A/A test before we did experiments. We'd also do an A/A/B test to make sure the actual experiments were working.
http://www.exp-platform.com/Pages/default.aspx http://www.exp-platform.com/Pages/default.aspx
- darkmighty 13y agoThere are also some complex problems with assumptions that are infrequently addressed, e.g. maybe if a regular user sees a structural/cosmetic change he is more likely to look at that and click it, while that would fade away in steady-state.
- RyJones 13y agoTrue. We did everything we could to account for that, though. Someone chooses to clear cookies every time they load a page - not much we could do.