5 ms·

What this highlights most to me is the false precision people have come to expect. This is a real problem I've seen as an engineer / physicist working with busi

What this highlights most to me is the false precision people have come to expect. This is a real problem I've seen as an engineer / physicist working with business people. Things that I would consider "the same" because they are well withing the noise of one another get all kinds of "hold on now, I added up the numbers in the columns and got 22.1 but the columns total says 22" from people that don't understand rounding, significant figures, experimental error, etc. I know it's important to hide this from people because they end up getting distracted by it (or to occasionally let slip so someone can feel like they have done a thorough review because they found a column didnt add up to the rounded total). But I think it's more important overall to educate people about what kind of variation is important and what isn't, vs. getting hung up on the definition of a word.

6gvONxR4sf7o 5y ago
I have a ton of experience with that too. You have two people try to estimate a thing, and they make different choices and come up with answers that are totally consistent with each other, but aren't the same. Then someone sees that they're different and omg omg omg everything's on fire. It's amazingly hard to communicate "variability" in an answer. Whenever I communicate experimental results, my business partners always want to hone in on the simplest explanation (e.g. "treatment effect somewhere between 1 and 3" becomes "treatment effect = 2"). And that's just the uncertainty part of variability. Then there's the kinds of variability you're talking about. There are a ton of tricks to mitigating this, like reporting confidence intervals instead of point estimates, or giving the right number of digits to communicate the key message, or making super sharable/linkable results to minimize the game of telephone. I'd love someone to write a book on how to communicate this sort of "variability" in a business setting.
- weaksauce 5y ago
  > like reporting confidence intervals Then there is the problem of how insanely innumerate society at large is to the point where something like this doesn't register for huge swaths of people. might work in the context of engineers or people with some higher education but not all contexts.
  - 6gvONxR4sf7o 5y ago
    I don't know, it feels like many people understand ranges (but still not all). They don't need to understand what a CI is to get "somewhere between 1 and 3." The barrier I face is that while they understand ranges, they can't take uncertainty.
- conductr 5y ago
  In corporate finance something like a ROI is seen as telling the future. I can’t tell you how common it is to revisit something on an “let’s understand why the ROI is off” basis. And them not understand when I point out not a single assumption held true. More often than not aggressive assumptions were used to validate the strategy and gain approvals and it was always going to fall short. In my role, CYA is a huge deal.

6gvONxR4sf7o 5y ago

I have a ton of experience with that too. You have two people try to estimate a thing, and they make different choices and come up with answers that are totally consistent with each other, but aren't the same. Then someone sees that they're different and omg omg omg everything's on fire. It's amazingly hard to communicate "variability" in an answer. Whenever I communicate experimental results, my business partners always want to hone in on the simplest explanation (e.g. "treatment effect somewhere between 1 and 3" becomes "treatment effect = 2"). And that's just the uncertainty part of variability. Then there's the kinds of variability you're talking about. There are a ton of tricks to mitigating this, like reporting confidence intervals instead of point estimates, or giving the right number of digits to communicate the key message, or making super sharable/linkable results to minimize the game of telephone. I'd love someone to write a book on how to communicate this sort of "variability" in a business setting.

weaksauce 5y ago
> like reporting confidence intervals Then there is the problem of how insanely innumerate society at large is to the point where something like this doesn't register for huge swaths of people. might work in the context of engineers or people with some higher education but not all contexts.
- 6gvONxR4sf7o 5y ago
  I don't know, it feels like many people understand ranges (but still not all). They don't need to understand what a CI is to get "somewhere between 1 and 3." The barrier I face is that while they understand ranges, they can't take uncertainty.
conductr 5y ago
In corporate finance something like a ROI is seen as telling the future. I can’t tell you how common it is to revisit something on an “let’s understand why the ROI is off” basis. And them not understand when I point out not a single assumption held true. More often than not aggressive assumptions were used to validate the strategy and gain approvals and it was always going to fall short. In my role, CYA is a huge deal.