4 ms·
Optimizely rolled out a huge update to the way it handles statistics called Stats Engine last year. That update resolves the issues discussed in this article. Y
by kylerush 11y ago
Optimizely rolled out a huge update to the way it handles statistics called Stats Engine last year. That update resolves the issues discussed in this article. You can read more about Stats Engine here: https://www.optimizely.com/statistics/ https://www.optimizely.com/statistics/
- stdbrouw 11y agoTheir Stats Engine does resolve the issues, but I have to laugh at their marketing materials calling it 21st century statistics, because even the new approach is comically behind the times. Looks like the poor schmucks accidentally outsourced the project to some really old-school one trick pony "let's throw some maximum likelihood theory at this" statisticians. I would've hoped even die-hard frequentists would see the value of a multi-armed bandit and the irrelevance of declaring a winner rather than quantifying the effect of each variant and the uncertainty around it. Cf. the technical paper: http://pages.optimizely.com/rs/optimizely/images/stats_engine_technical_paper.pdf http://pages.optimizely.com/rs/optimizely/images/stats_engin...
- TheLogothete 11y agomulti-armed bandits for finding global maximums? Please refrain from using this arrogant tone, because you only make people laugh.
- stdbrouw 11y agoI can see why my comment could come across as arrogant and in hindsight shouldn't have written it with that tone. However, I wrote it immediately after reading https://www.optimizely.com/statistics/ https://www.optimizely.com/statistics/ which makes a number of incredibly boastful claims about how Optimizely's approach is better than anything you've ever seen, and couldn't help but point out the hubris. If you ever come up with a statistical method for finding the global maximum among an infinite number of unspecified alternatives, let me know.