3 ms·

Put more simply: suppose I have a coin that might be biased. I decide to asses this via repeatedly flipping it—if it’s biased, it will disproportionately land o

Put more simply: suppose I have a coin that might be biased. I decide to asses this via repeatedly flipping it—if it’s biased, it will disproportionately land on heads or tails. If I flip 10 times and get 4 heads/6 tails, I don’t have the power to make a confident assessment of any bias. On the other hand, if I flip it 100 times and get 40 heads/60 tails, I am a bit more certain. At 1000 flips, with 400 heads/600 tails, I am extremely confident the coin is biased. Even though the fraction of heads is identically 40% across all three sets of flips, the underlying counts yield very different amounts of confidence on how close to 40% the coin’s bias is. The first question is a way of rigorously quantifying this confidence. I don’t think this is “leetcode for statisticians.” This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. In fact, the first question is directly relevant to voting on this site. When sorting replies by fraction of upvotes, how should the forum software rank a new reply with 1 upvote/0 downvotes, versus an older reply with 4 upvotes/1 downvote? What about an older, more controversial reply with 20 upvotes/7 downvotes? 15 upvotes/2 downvotes?

jldugger 1y ago
> This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. Indeed, I use this technique to sort search results in Splunk, as an extension of TF-IDF. Consider a scenario where us-east-2 is broken but us-east-1 is fine (clearly just a hypothetical!). Split the logs along that good/bad dimension, and then break down by some other pattern; log class, punct, etc. Usually I use a prior of 50:50 to help sort out the "happened once in bad cluster" events.

jldugger 1y ago

> This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. Indeed, I use this technique to sort search results in Splunk, as an extension of TF-IDF. Consider a scenario where us-east-2 is broken but us-east-1 is fine (clearly just a hypothetical!). Split the logs along that good/bad dimension, and then break down by some other pattern; log class, punct, etc. Usually I use a prior of 50:50 to help sort out the "happened once in bad cluster" events.