3 ms·

I have a ton of experience with that too. You have two people try to estimate a thing, and they make different choices and come up with answers that are totally

I have a ton of experience with that too. You have two people try to estimate a thing, and they make different choices and come up with answers that are totally consistent with each other, but aren't the same. Then someone sees that they're different and omg omg omg everything's on fire. It's amazingly hard to communicate "variability" in an answer. Whenever I communicate experimental results, my business partners always want to hone in on the simplest explanation (e.g. "treatment effect somewhere between 1 and 3" becomes "treatment effect = 2"). And that's just the uncertainty part of variability. Then there's the kinds of variability you're talking about. There are a ton of tricks to mitigating this, like reporting confidence intervals instead of point estimates, or giving the right number of digits to communicate the key message, or making super sharable/linkable results to minimize the game of telephone. I'd love someone to write a book on how to communicate this sort of "variability" in a business setting.

weaksauce 5y ago
> like reporting confidence intervals Then there is the problem of how insanely innumerate society at large is to the point where something like this doesn't register for huge swaths of people. might work in the context of engineers or people with some higher education but not all contexts.
- 6gvONxR4sf7o 5y ago
  I don't know, it feels like many people understand ranges (but still not all). They don't need to understand what a CI is to get "somewhere between 1 and 3." The barrier I face is that while they understand ranges, they can't take uncertainty.
conductr 5y ago
In corporate finance something like a ROI is seen as telling the future. I can’t tell you how common it is to revisit something on an “let’s understand why the ROI is off” basis. And them not understand when I point out not a single assumption held true. More often than not aggressive assumptions were used to validate the strategy and gain approvals and it was always going to fall short. In my role, CYA is a huge deal.

weaksauce 5y ago

> like reporting confidence intervals Then there is the problem of how insanely innumerate society at large is to the point where something like this doesn't register for huge swaths of people. might work in the context of engineers or people with some higher education but not all contexts.

6gvONxR4sf7o 5y ago
I don't know, it feels like many people understand ranges (but still not all). They don't need to understand what a CI is to get "somewhere between 1 and 3." The barrier I face is that while they understand ranges, they can't take uncertainty.

conductr 5y ago

In corporate finance something like a ROI is seen as telling the future. I can’t tell you how common it is to revisit something on an “let’s understand why the ROI is off” basis. And them not understand when I point out not a single assumption held true. More often than not aggressive assumptions were used to validate the strategy and gain approvals and it was always going to fall short. In my role, CYA is a huge deal.