4 ms·
At least for us, most A/B/.. tests require a stable assignment of test to user as key metrics like retention would obviously skewed by randomly assigning on eac
by NewsAware 7y ago
At least for us, most A/B/.. tests require a stable assignment of test to user as key metrics like retention would obviously skewed by randomly assigning on each visit/page view.
- jfries 7y agoThe user id can be used to seed the random number algorithm to achieve that.
- NewsAware 7y agoHmmm but if you change the distribution, this won't be deterministic anymore unless the randomness only applies to new users entering the test setup.
- leshokunin 7y agoWhich is necessary. You don’t want to compare two wildly different users. This is why you need to define criteria for eligibility.