6 ms·

I like this! In the grand HN tradition of being triggered by a word in the post and going off on a not-quite-but-basically-totally-tangential rant: There’s (a

• youainti 2y ago
  > I’ve just seen a lot of people pick up statistics for the first time and lose a finger. I love this. I've never though of statistics like a power tool or firearm, but the analogy fits really well.
  • ninalanyon 2y ago
    Unfortunately it's usually someone else who loses a finger, not the person wielding the statistics.
• btilly 2y ago
  I strongly agree with this, and particularly point 1. If you ask people to provide estimated ranges for answers that they are 90% confident in, people on average produce roughly 30% confidence intervals instead. Over 90% of people don't even get to 70% confidence intervals. You can test yourself at https://blog.codinghorror.com/how-good-an-estimator-are-you/ https://blog.codinghorror.com/how-good-an-estimator-are-you/.
  • Nevermark 2y ago
    From link: > Heaviest blue whale ever recorded I don't think estimation errors regarding things outside of someone's area of familiarity say much. You could ask a much "easier"" question from the same topic area and still get terrible answers: "What percentage of blue whales are blue?" Or just "Are blue whales blue?" Estimating something often encountered but uncounted seems like a better test. Like how many cars pass in front of my house every day. I could apply arithmetic, soft logic and intuition to that. But that would be a difficult question to grade, given it has no universal answer.
    • yen223 2y ago
      I guess people didn't realise they are allowed to, and in fact are expected to, put very wide ranges for things they are not certain about.
    • kqr 2y ago
      I have no familiarity with blue whales but I would guess they're 1--5 times the mass of lorries, which I guess weigh like 10--20 cars which I in turn estimate at 1.2--2 tonnes, so primitively 12--200 tonnes for a normal blue whale. This also aligns with it being at least twice as large as an elephant, something I estimate at 5 tonnes. The question asks for the heaviest, which I think cannot be more than three times the normal weight, and probably no less than 1.3. That lands me at 15--600 tonnes using primitive arithmetic. The calculator in OP suggests 40--320. The real value is apparently 170, but that doesn't really matter. The process of arriving at an interval that is as wide as necessary but no wider is the point. Estimation is a skill that can be trained. It is a generic skill that does not rely on domain knowledge beyond some common sense.
      • dotaenjoyer 2y ago
        I would say general knowledge in many domains may help with this as you can try and approximate to the nearest thing you know from that domain. How you get good at being a generalist is the tricky part, my best bet is reading and doing a lot of trivia (I found crosswords to be somewhat effective at this, but far from being efficient)
• pertdist 2y ago
  I did a project with non-technical stakeholders modeling likely completion dates for a big GANTT chart. Business stakeholders wanted probabilistic task completion times because some of the tasks were new and impractical to quantify with fixed times. Stakeholders really liked specifying work times as t_i ~ PERT(min, mode, max) because it mimics their thinking and handles typical real-world asymmetrical distributions. [Background: PERT is just a re-parameterized beta distribution that's more user-friendly and intuitive https://rpubs.com/Kraj86186/985700 https://rpubs.com/Kraj86186/985700]
  • baq 2y ago
    arguably this is how it should always be done, fixed durations for any tasks are little more than wishful thinking.
  • kqr 2y ago
    This looks like a much more sophisticated version of PERT than I have seen used. When people around me have claimed to use PERT, they have just added together all the small numbers, all the middle numbers, and all the big numbers. That results in a distribution that is too extreme in both lower and upper bound.
    • baq 2y ago
      that... is not PERT. it's 'I read a tweet about three point estimates' and I'm using a generous interpretation of read
• jrowen 2y ago
  This jives with my general reaction to the post, which was that the added complexity and difficulty of reasoning about the ranges actually made me feel less confident in the result of their example calculation. I liked the $50 result, you can tack on a plus or minus range but generally feel like you're about breakeven. On the other hand, "95% sure the real balance will fall into the -$60 to +$220 range" feels like it's creating a false sense of having more concrete information when you've really just added compounding uncertainties at every step (if we don't know that each one is definitely 95%, or the true min/max, we're just adding more guesses to be potentially wrong about). That's why I don't like the Drake equation, every step is just compounding wild-ass guesses, is it really producing a useful number?
  • kqr 2y ago
    It is producing a useful number. As more truly independent terms are added, error grows with the square root while the point estimation grows linearly. In the aggregate, the error makes up less of the point estimation. This is the reason Fermi estimation works. You can test people on it, and almost universally they get more accurate with this method. If you got less certain of the result in the example, that's probably a good thing. People are default overconfident with their estimated error bars.
    • pests 2y ago
      > People are default overconfident with their estimated error bars. You say this but yet roughly in a top level comment mentions people keep their error bars too close.
      • bigfudge 2y ago
        They are meaning the same thing. The original comment pointed out that people’s qualitative description and mental model of the 95% interval means they are overconfident… they think 95 means ‘pretty sure I’m right’ rather than ‘it would be surprising to be wrong’
      • kqr 2y ago
        Sorry, my comment was phrased confusingly. Being overconfident with error bars means placing them too close to the point estimation, i.e. the error bars are too narrow.
• larodi 2y ago
  Actually using it already after finding it few days ago on HN
• jbjbjbjb 2y ago
  I think to do all that you’d need a full on DSL rather than something pocket calculator like. I think adding a triangular distribution would be good though.
• rssoconnor 2y ago
  Normal distributions are the maximum entropy distributions for a given mean and variance. Therefore, in accordance with the principle of maximum entropy, unless you have some reason to not pick a normal distribution (e.g. you know your values must be non-negative), you should be using a normal distribution.
  • kqr 2y ago
    > you should be using a normal distribution. ...if the only things you know about an uncertain value are its expectation and variance, yes. Often you know other things. Often you don't know expectation and variance with any certainty.
  • tgv 2y ago
    At least also accept a log-normal distribution. Sometimes you need a factor like .2 ~ 5, but that isn't the same as N(2.6, 1.2).
• JKCalhoun 2y ago
  > 2) probability is rarely truly uncorrelated Without having fully digested how the Unsure Calculator computes, it seems to me you could perhaps "weight" the ranges you pass to the calculator. Rather than a standard bell curve the Calculator could apply a more tightly focused — or perhaps skewed curve for that term. If you think your salary will be in the range of 10 to 20, but more likely closer to 10 you could: 10<~20 (not to be confused with less-than) or: 10!~20 (not to be confused with factorial) or even: 10~12~20 to indicate a range of 10 to 20 ... leaning toward 12.
  • roughly 2y ago
    The correlation in this case isn't about the distribution for the individual event, it's about the interactions between them - so, for instance, Rent could be anywhere between 1200 and 1800, and Food could be anywhere between 100 and 150, but if Rent is 1200, it means Food is more likely to be 100, and if Food is 150, it means Rent is more likely to be 1800. Basically, there's a shared factor that's influencing both (local cost of living) that's the actual thing you need to model. So, a realistic modeling isn't 1200~1500 + 100~150, it's (1~1.5)*(1200 + 150) - the "cost of living" distribution applies to both factors.
• gamerDude 2y ago
  Great points. I think the idea of this calculator could just be simply extended to specific use cases to make the statistical calculation simple and take into account additional variables. Moving being one example.
• dawnofdusk 2y ago
  >rent is very likely to be correlated with food costs - so, if rent is high, food costs are also likely to be high Not sure I agree with this. It's reasonable to have a model where the mean rent may be correlated with the mean food cost, but given those two parameters we can model the fluctuations about the mean as uncorrelated. In any case at the point when you want to consider something like this you need to do proper Bayesian statistics anyways. >In general normal distributions are rarer than people think - they tend to require some kind of constraining factor on the values to enforce. I don't know where you're getting this from. One needs uncorrelated errors, but this isn't a "constraint" or "negative feedback".
  • roughly 2y ago
    The family example is a pat example, but take something like project planning - two tasks, each one takes between 2 and 4 weeks - except that they’re both reliant on Jim, and if Jim takes the “over” on task 1, what’s the odds he takes the “under” on task 2? This is why I joked about it as the mortgage derivatives maxim - what happened in 2008 (mathematically, at least - the parts of the crisis that aren’t covered by the famous Upton Sinclair quote) was that the mortgage backed derivatives were modeled as an aggregate of a thousand uncorrelated outcomes (a mortgage going bust), without taking into account that at least a subset of the conditions leading to one mortgage going bust would also lead to a separate unrelated mortgage going bust - the results were not uncorrelated, and treating them as such meant the “1 in a million” outcome was substantially more likely in reality than the model allowed. Re: negative feedback - that’s a separate point from the uncorrelated errors problem above, and a critique of using the normal distribution at all for modeling many different scenarios. Normal distributions rely on some kind of, well, normal scattering of the outcomes, which means there’s some reason why they’d tend to clump around a central value. We see it in natural systems because there’s some constraints on things like height and weight of an organism, etc, but without some form of constraint, you can’t rely on a normal distribution - the classic examples being wealth, income, sales, etc, where the outliers tend to be so much larger than average that they’re effectively precluded by a normal distribution, and yet there they are. To be clear, I’m not saying there are not statistical methods for handling all of the above, I’m noting that the naive approach of modeling several different uncorrelated normally distributed outcomes, which is what the posted tool is doing, has severe flaws which are likely to lead to it underestimating the probability of outlier outcomes.