4 ms·
Getting to decisions faster in A/B tests – part 1: literature review
- travisjungroth 4y agoI didn’t see it linked, but a paper by my coworker at Netflix, Michael Lindon, seems relevant: Anytime-Valid Inference for Multinomial Count Data. https://arxiv.org/abs/2011.03567 https://arxiv.org/abs/2011.03567
- kamicollo 4y agoThanks for sharing - will add to my reading list!
- orasis 4y agoIf you’re worried about having too little data then A/B tests are the wrong tool for the job. It’s better to just use ML scorers and rankers.
- CaveTech 4y agoCan you elaborate on this? How would one convert a classic conversion-rate optimization task to use an ML scorer? Would that mean converting it into a classification problem?
- orasis 4y agoIt becomes a regression problem. You use ML to estimate the conversion rate for each item. You still have to do feature engineering, but fast regressors like XGBoost make this much easier. https://improve.ai https://improve.ai is one such scorer/ranker built on XGBoost.