3 ms·
Reinforcement approaches are certainly interesting, but one of the things missing here (and in most A/B stuff) is statistical significance and experimental powe
by scottlocklin 11y ago
Reinforcement approaches are certainly interesting, but one of the things missing here (and in most A/B stuff) is statistical significance and experimental power. If you have enough data, there are hand wavey arguments that this will eventually be right, but in the meanwhile, if there is some opportunity cost (say, imagine this is a trading algo trying to profit from bid/ask), you screwed yourself out of some unknown amount of profits. There actually ways of hanging a confidence interval on this approach which virtually nobody outside the signal processing and information theory communities know about. Kind of a shame.
- gburt 11y agoAny chance you could point me to a reference? I'm doing research in this space and currently working on a paper which does exactly this for diagnostics of testing processes.
- scottlocklin 11y agoI'm not sure which thing I mentioned you need a reference on. For p-values on machine learning techniques, http://vovk.net/cp/index.html http://vovk.net/cp/index.html I'll eventually do a blog post on this subject; it's very good math that all ML people should know about, though Vovk, Schaefer and Gammerman write pretty dense articles. For statistical power... "Statistical Power Analysis for the Behavioral Sciences" by Jacob Cohen.
- gburt 11y agoSorry, I intended to quote your last sentence, applying a confidence interval to a reinforcement learning system, especially with respect to multi-armed bandits / adaptive experiments, but if I have to dig in to some signal processing stuff I am happy to do that. It seems the conformal prediction link has some relation. I will dig, thanks.
- scottlocklin 11y agoYes, conformal prediction does exactly this.