4 ms·
For better accuracy with small samples you could use the multinomial distribution instead. The covariance matrix for the rating probabilities can be found here
by vcdimension 11y ago
For better accuracy with small samples you could use the multinomial distribution instead. The covariance matrix for the rating probabilities can be found here for example: http://www.math.wsu.edu/faculty/genz/papers/mvnsing/node8.html http://www.math.wsu.edu/faculty/genz/papers/mvnsing/node8.ht...
Then the variance for the expected rating can be calculated as a weighted sum of the values in the covariance matrix.
These companies really should be hiring statistics consultants instead of relying on the intuitions of their programmers.
- stdbrouw 11y agoI'd prefer to just treat scores as continuous and correct using `t_ppf(.975, n-1)` instead of the normal approximation (1.96) but I suppose working from a multinomial distribution would give pretty similar results.
- vcdimension 11y agoYou're still relying on the central limit theorem (i.e. a reasonable amount of data) : using t instead of z just corrects for the fact that you only have sample variances instead of population variances. However, I suppose it's not unreasonable to assume that the ratings are likely to have a bell shaped distribution (which could be checked), so the normal/t approximation is probably going to be OK.
- stdbrouw 11y agoAh yes, true. Let's call it a bias/variance tradeoff ;-)