3 ms·
Any chance you could point me to a reference? I'm doing research in this space and currently working on a paper which does exactly this for diagnostics of testi
by gburt 11y ago
Any chance you could point me to a reference? I'm doing research in this space and currently working on a paper which does exactly this for diagnostics of testing processes.
- scottlocklin 11y agoI'm not sure which thing I mentioned you need a reference on. For p-values on machine learning techniques, http://vovk.net/cp/index.html http://vovk.net/cp/index.html I'll eventually do a blog post on this subject; it's very good math that all ML people should know about, though Vovk, Schaefer and Gammerman write pretty dense articles. For statistical power... "Statistical Power Analysis for the Behavioral Sciences" by Jacob Cohen.
- gburt 11y agoSorry, I intended to quote your last sentence, applying a confidence interval to a reinforcement learning system, especially with respect to multi-armed bandits / adaptive experiments, but if I have to dig in to some signal processing stuff I am happy to do that. It seems the conformal prediction link has some relation. I will dig, thanks.
- scottlocklin 11y agoYes, conformal prediction does exactly this.