2 ms·

> I think the only conclusion one can draw from this is that high school statistics is actually useful in real-life. That isn't the conclusion, you can have a

by roenxi 2y ago

> I think the only conclusion one can draw from this is that high school statistics is actually useful in real-life. That isn't the conclusion, you can have a management team that all technically understand statistics and they still make the same mistakes. It is depressing to watch it play out in real time. The problem is when people don't identify that workers are just doing what they are told and personal effort isn't the major factor determining system outcomes. Going through the article, the important part about the narrative is the misconception that "performance outcomes are being determined by the worker, so managing the workers will get better performance!". The management is being done by statistics throughout even though the quality of that management is bad.

wat10000 2y ago
The management in this game is being done by a deep misunderstanding of statistics. They’re using random noise as a signal and taking action based on it. Actually using statistics would tell you that your numbers are random noise and can’t be used to make sensible decisions. It would tell you that performance outcomes aren’t being determined by the worker. There might be better ways to figure that out (in this case, taking five seconds to look at the process) but statistics will get that job done.
- roenxi 2y ago
  Well you aren't wrong, but that is a question of identifying what model to fit. If the question is "are the workers in control of the outcome" then sure, they weren't using statistics properly. But if management realises that is the question to ask they also probably won't need statistics because the answer is obviously no in almost all contexts. Management was asking the question "given that the workers control the process and we know that there are differences between people, which workers are performing the best?". And they were doing the best they could under pressure with the incomplete information they had. It was a stupid model, but that is why they are going to an Ed Deming talk - the median manager defaults to stupid and needs a bit of help. Training people in high school stats doesn't necessarily get better results at model selection, some extremely educated and capable people are still stupid about that part.
  - wat10000 2y ago
    They were certainly not doing their best to answer that question! Stats will answer that question correctly, answering “they’re all performing the same, within the margin of error.” Yes, actually understanding what the hell is going on is really helpful, but as a backstop, it’s good to have the basic stats knowledge that you can’t come to any conclusion from a handful of samples.
    - roenxi 2y ago
      No you're not seeing it from the management perspective. They've already selected their model and assumed that performance depends on the worker involved. There isn't any "they’re all performing the same within the margin of error" because that would violate a model assumption. It has a probability of ~0%, there isn't any point checking for it. You're working from a model where the performance could be the same and then doing the obvious statistical checks. Typically in the wild management have sufficient knowledge to do those checks if their model allowed for performance to be equal. But they don't have a model where that is a reasonable possibility so there is nothing to check. They're doing the correct statistics for the wrong model. And the model happens to be really stupid but that is managers for you. You can teach them all that theory and they still won't apply it. The problem isn't usually a gap in statistical knowledge. Take another look at how Deming framed his experiment - if you look closely he is imparting statistical knowledge but that isn't where the real focus is. He's actually attacking the idea that the workers have any influence on outcomes.
      - wat10000 2y ago
        I see the management perspective. I’m just saying that perspective is obviously wrong if you have any stats knowledge. If your point is just that management could have that knowledge and fail to apply it, I won’t argue with that. But I definitely wouldn’t call it “doing their best” when that happens.

by roenxi 2y ago

wat10000 2y ago

The management in this game is being done by a deep misunderstanding of statistics. They’re using random noise as a signal and taking action based on it. Actually using statistics would tell you that your numbers are random noise and can’t be used to make sensible decisions. It would tell you that performance outcomes aren’t being determined by the worker. There might be better ways to figure that out (in this case, taking five seconds to look at the process) but statistics will get that job done.

roenxi 2y ago
Well you aren't wrong, but that is a question of identifying what model to fit. If the question is "are the workers in control of the outcome" then sure, they weren't using statistics properly. But if management realises that is the question to ask they also probably won't need statistics because the answer is obviously no in almost all contexts. Management was asking the question "given that the workers control the process and we know that there are differences between people, which workers are performing the best?". And they were doing the best they could under pressure with the incomplete information they had. It was a stupid model, but that is why they are going to an Ed Deming talk - the median manager defaults to stupid and needs a bit of help. Training people in high school stats doesn't necessarily get better results at model selection, some extremely educated and capable people are still stupid about that part.
- wat10000 2y ago
  They were certainly not doing their best to answer that question! Stats will answer that question correctly, answering “they’re all performing the same, within the margin of error.” Yes, actually understanding what the hell is going on is really helpful, but as a backstop, it’s good to have the basic stats knowledge that you can’t come to any conclusion from a handful of samples.
  - roenxi 2y ago
    No you're not seeing it from the management perspective. They've already selected their model and assumed that performance depends on the worker involved. There isn't any "they’re all performing the same within the margin of error" because that would violate a model assumption. It has a probability of ~0%, there isn't any point checking for it. You're working from a model where the performance could be the same and then doing the obvious statistical checks. Typically in the wild management have sufficient knowledge to do those checks if their model allowed for performance to be equal. But they don't have a model where that is a reasonable possibility so there is nothing to check. They're doing the correct statistics for the wrong model. And the model happens to be really stupid but that is managers for you. You can teach them all that theory and they still won't apply it. The problem isn't usually a gap in statistical knowledge. Take another look at how Deming framed his experiment - if you look closely he is imparting statistical knowledge but that isn't where the real focus is. He's actually attacking the idea that the workers have any influence on outcomes.
    - wat10000 2y ago
      I see the management perspective. I’m just saying that perspective is obviously wrong if you have any stats knowledge. If your point is just that management could have that knowledge and fail to apply it, I won’t argue with that. But I definitely wouldn’t call it “doing their best” when that happens.