4 ms·
From the question: “However, it is very important that the uncertainty in the number of trials is taken into account because over-estimating a fraction is a co
by usgroup 1y ago
From the question:
“However, it is very important that the uncertainty in the number of trials is taken into account because over-estimating a fraction is a costly mistake.“
Seems fairly clear to me that you’re supposed to use a lower bound estimate to take into account variance on the fraction due to the number of trials in a way to bounds the chance of over estimation.
Further, there is no need for a heuristic when there a several statistical models for this exact problem with clear properties. Some are given in the answer.
- jldugger 1y agoI guess I struggle to understand why under estimation is worse than over estimation, when the final result is a ranking. It seems like they're equally likely to produce an incorrect ranking!
- thekoma 1y agoOut of context, the expression "the uncertainty in the number of trials" would refer to missing knowledge in terms of how many trials actually ran. In the context of the post this doesn't make sense, so the reader is left to hypothesize what the writer actually meant.
- usgroup 1y agoI agree it could be clearer but as a general rule, if you find an interpretation under which the question doesn’t make sense, try considering another interpretation.
- taylorius 1y agoI think "uncertainty due to the number of trials" would be clearer.
- tomsmeding 1y agoUncertainty in the number of successes due to the number of trials, would be even better.
- jdhwosnhw 1y agoThat would also be an incorrect phrasing. This entire thread is a good illustration of the difficulty of speaking precisely about probabilistic concepts. (The number of successes has zero uncertainty. If you flip a coin 10 times and get 5 heads, there is no uncertainty on the number of heads. In general, for any statistical model the uncertainty is only with respect to an underlying model parameter - in this example, while your number of successes is perfectly known, it can be used to infer a probability of success, p, of 0.5, and there is uncertainty associated with that inferred probability.)