3 ms·

I wrote the deleted comment you are replying to. The essence of my comment was that this text/test is not for me (one person of the general public) but more li

I wrote the deleted comment you are replying to. The essence of my comment was that this text/test is not for me (one person of the general public) but more like a few leetcode-style questions for statisticians. Your attempt to explain what I didn't understand just proves my point as I don't really understand what you are saying either. And that's ok: this is just not for me! (And that's why I deleted my original comment)

dmurray 1y ago
The problems are really underspecified for statisticians, too. Leetcode is normally very clear on the requirements. > it is very important that the uncertainty in the number of trials is taken into account because over-estimating a fraction is a costly mistake. This is not some precise jargon that is meaningless to the layman but completely clearly specified to a professional statistician. It's more like the specification written by your non-technical product manager for how some technical feature should work. A skilled data scientist will have the experience and the context to figure out what it's probably asking for, but he might write down a few more clarifying details before giving it to a junior on his team to implement. If testing these kind of guess-what-the-stakeholder probably-means skills is the point of this test, it's quite good at it. But that's not what leetcode is for.
MontyCarloHall 1y ago
Put more simply: suppose I have a coin that might be biased. I decide to asses this via repeatedly flipping it—if it’s biased, it will disproportionately land on heads or tails. If I flip 10 times and get 4 heads/6 tails, I don’t have the power to make a confident assessment of any bias. On the other hand, if I flip it 100 times and get 40 heads/60 tails, I am a bit more certain. At 1000 flips, with 400 heads/600 tails, I am extremely confident the coin is biased. Even though the fraction of heads is identically 40% across all three sets of flips, the underlying counts yield very different amounts of confidence on how close to 40% the coin’s bias is. The first question is a way of rigorously quantifying this confidence. I don’t think this is “leetcode for statisticians.” This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. In fact, the first question is directly relevant to voting on this site. When sorting replies by fraction of upvotes, how should the forum software rank a new reply with 1 upvote/0 downvotes, versus an older reply with 4 upvotes/1 downvote? What about an older, more controversial reply with 20 upvotes/7 downvotes? 15 upvotes/2 downvotes?
- jldugger 1y ago
  > This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. Indeed, I use this technique to sort search results in Splunk, as an extension of TF-IDF. Consider a scenario where us-east-2 is broken but us-east-1 is fine (clearly just a hypothetical!). Split the logs along that good/bad dimension, and then break down by some other pattern; log class, punct, etc. Usually I use a prior of 50:50 to help sort out the "happened once in bad cluster" events.
jldugger 1y ago
No worries, I wasn't really trying to explain it anyways, as much as seeking confirmation from the rest of HN that this question is ill specified. Judging from responses, yes, it is. If "binomial distribution" and "confidence interval" are unfamiliar terms then you probably are not prepared to pass OP's "statistical reasoning test" regardless. I think most engineers wouldn't, and I only understood the intent of question 1 because my pandemic lockdown project was reading a stats textbook cover to cover.

dmurray 1y ago

The problems are really underspecified for statisticians, too. Leetcode is normally very clear on the requirements. > it is very important that the uncertainty in the number of trials is taken into account because over-estimating a fraction is a costly mistake. This is not some precise jargon that is meaningless to the layman but completely clearly specified to a professional statistician. It's more like the specification written by your non-technical product manager for how some technical feature should work. A skilled data scientist will have the experience and the context to figure out what it's probably asking for, but he might write down a few more clarifying details before giving it to a junior on his team to implement. If testing these kind of guess-what-the-stakeholder probably-means skills is the point of this test, it's quite good at it. But that's not what leetcode is for.

MontyCarloHall 1y ago

Put more simply: suppose I have a coin that might be biased. I decide to asses this via repeatedly flipping it—if it’s biased, it will disproportionately land on heads or tails. If I flip 10 times and get 4 heads/6 tails, I don’t have the power to make a confident assessment of any bias. On the other hand, if I flip it 100 times and get 40 heads/60 tails, I am a bit more certain. At 1000 flips, with 400 heads/600 tails, I am extremely confident the coin is biased. Even though the fraction of heads is identically 40% across all three sets of flips, the underlying counts yield very different amounts of confidence on how close to 40% the coin’s bias is. The first question is a way of rigorously quantifying this confidence. I don’t think this is “leetcode for statisticians.” This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. In fact, the first question is directly relevant to voting on this site. When sorting replies by fraction of upvotes, how should the forum software rank a new reply with 1 upvote/0 downvotes, versus an older reply with 4 upvotes/1 downvote? What about an older, more controversial reply with 20 upvotes/7 downvotes? 15 upvotes/2 downvotes?

jldugger 1y ago
> This question (and the other two) are all examples of concrete, real-world problems that people across a variety of quantitative disciplines frequently encounter. Indeed, I use this technique to sort search results in Splunk, as an extension of TF-IDF. Consider a scenario where us-east-2 is broken but us-east-1 is fine (clearly just a hypothetical!). Split the logs along that good/bad dimension, and then break down by some other pattern; log class, punct, etc. Usually I use a prior of 50:50 to help sort out the "happened once in bad cluster" events.

jldugger 1y ago

No worries, I wasn't really trying to explain it anyways, as much as seeking confirmation from the rest of HN that this question is ill specified. Judging from responses, yes, it is. If "binomial distribution" and "confidence interval" are unfamiliar terms then you probably are not prepared to pass OP's "statistical reasoning test" regardless. I think most engineers wouldn't, and I only understood the intent of question 1 because my pandemic lockdown project was reading a stats textbook cover to cover.