3 ms·

As others have mentioned, you're referring to Thompson sampling and plenty of testing providers offer this (and if you have any DS on staff, they'll be more tha

As others have mentioned, you're referring to Thompson sampling and plenty of testing providers offer this (and if you have any DS on staff, they'll be more than happy to implement it). My experience is that there's a good reason why this hasn't taken off: the returns for this degree of optimization are far lower than you think. I once worked with a very eager, but junior DS who thought that we should build out a massive internal framework for doing this. He didn't quite understand the math behind it, so I build him a demo to understand the basics. What we realized in running the demo under various conditions is that the total return on adding this complexity to optimization was negligible at the scale we were operating at and required much more complexity than our current set up. This pattern repeats in a lot of DS related optimization in my experience. The difference between a close guess and perfectly optimal is often surprisingly little. Many DS teams perform optimizations on business processes that yield a lower improvement in revenue than the salary of the DS that built it.

brookst 3y ago
Small nit: it’s a bad idea if NPV of future returns is less than the cost. If someone making $100k/yr can produce one $50k/yr optimization that NPV’s out to $260k, it’s worth it. I suspect you meant that, just a battle I have at work a lot with people who only look at single-year returns.
tomfutur 3y ago
Besides complexity, a price you pay with multi-armed bandits is that you learn less about the non-optimal options (because as your confidence grows that an option is not the best, you run fewer samples through it). It turns out the people running these experiments are often not satisfied to learn "A is better than B." They want to know "A is 7% better than B," but a MAB system will only run enough B samples to make the first statement.

brookst 3y ago

Small nit: it’s a bad idea if NPV of future returns is less than the cost. If someone making $100k/yr can produce one $50k/yr optimization that NPV’s out to $260k, it’s worth it. I suspect you meant that, just a battle I have at work a lot with people who only look at single-year returns.

tomfutur 3y ago

Besides complexity, a price you pay with multi-armed bandits is that you learn less about the non-optimal options (because as your confidence grows that an option is not the best, you run fewer samples through it). It turns out the people running these experiments are often not satisfied to learn "A is better than B." They want to know "A is 7% better than B," but a MAB system will only run enough B samples to make the first statement.