4 ms·
Other, related points: - With heavy tails, the sample mean (i.e. the number you can see) is very likely to underestimate the population mean. - With heavy eno
by kqr 7y ago
Other, related points:
- With heavy tails, the sample mean (i.e. the number you can see) is very likely to underestimate the population mean.
- With heavy enough tails, higher moments like variance (and therefore standard deviation) do not exist at all -- they're infinite.
- Critically: With heavy tails, the central limit theorem breaks down. Sums of heavy-tailed samples converge to a normal distribution so slowly it might not realistically ever happen with your finite data. Any computation you do that explicitly or implicitly relies on the CLT will give you junk results!
- wodenokoto 7y agoCan you elaborate on the part about sample vs population mean? The way I see it; in these scenarios you aren’t looking at the sample mean. There is no reason to sample your customer base to get an estimation of your average revenue. You can calculate from the entire population.
- paulsutter 7y agoYour current customer base is just a sample of your total market, for example. Next year’s (larger) customer base will be slightly closer to the whole set (so different) but still just a sample.
- wodenokoto 7y agoSo the point is that the current customer base is a skewed sample of the theoretical customer population. Therefore we shouldn’t look at the current customer mean profit to predict what would happen to profits if we doubled the customer base.
- tomnipotent 7y agoIt's not about the doubling, it's about how the distribution changes as you gather more data and get closer to the actual mean (e.g. if you had EVERY possible customer ever, like cable companies). Take Tesla, for example. Its first customers were enthusiasts, but the mean of that group is not the mean of all current Tesla customers - they would look very different, and had Tesla optimized for the latter we'd be looking at a very different outcome.
- kqr 7y agoExactly. These summaries are often used for prediction, which means historic data is used as a sample of the same distribution as future data. Even when comparing two different historical data sets you have to be careful: if you're doing anything that resembles hypothesis testing (i.e. trying to figure out if something you changed made a difference) you're not really comparing two historical data sets -- you're trying to compare the underlying distributions from which the historical data sets were drawn, but hoping that the historical data are representative samples from those.
- j7ake 7y agoAn example of your infinite variance case, sums of cauchy distributions don't even converge to a normal distribution.
- contravariant 7y agoWith heavy enough tails even the mean might not exist.
- mumblemumble 7y agoEvery non-empty set of numbers has a mean. Perhaps you mean that, in a multimodal distribution, the mean might not resemble any individual member of the population?
- CrazyStat 7y agoA finite set of numbers has a mean, yes. A probability distribution with heavy enough tails does not have a mean (e.g. the Cauchy distribution).
- mumblemumble 7y agoSure, but let's not scope creep the conversation. This is in the context of talking about summary stats calculated against sets of discrete observations, not properties of abstract probability distributions. So we're talking about doing some basic arithmetic on a list of numbers, not taking integrals.
- SiempreViernes 7y agoNo, the point is that for certain sets of numbers, calculating the mean does not give you a summary statistic. It just gives the average of a bunch of numbers, not a number that is representative of your population.
- throwaway41968 7y agoThis is why using Gaussian models to predict something that's fundamentally fat-tailed (e.g. IQ vs. many forms of societal achievemnt) is doomed to failure.
- CrazyStat 7y ago