10 ms·
Here is a simpler back of the envelope solution to the problem of segmentation. Automatically ignore all statistics on segments that are below a certain sample
by btilly 12y ago
Here is a simpler back of the envelope solution to the problem of segmentation.
Automatically ignore all statistics on segments that are below a certain sample size. Say, 4k successful conversions. If there are no real differences, you'll still draw lots of wrong conclusions. But your odds of wrongly drawing significantly harmful decisions will be very low, and some of the conclusions you draw will be very good.
Your procedures may make statisticians curse and swear. But it provides a simple rule of thumb that is easy for non-statisticians to understand. And mitigates the really bad mistakes.
- yummyfajitas 12y agoMy strong disagreement with you here is a good illustration of the practical differences between Bayesians and Frequentists. As a Bayesian, I'm horrified by the idea of doing anything other than building a segment model and letting the fatness of the posterior handle the low sample sizes. But yeah, that is a pretty good way for a marketer to avoid being too stupid. And I can't say I haven't done this a bunch of times, particularly when doing quick exploratory calculations.