6 ms·

I think the only conclusion one can draw from this is that high school statistics is actually useful in real-life. If the company is crazy enough to put people

by theamk 2y ago

I think the only conclusion one can draw from this is that high school statistics is actually useful in real-life. If the company is crazy enough to put people on PIP based on 2 low-precision data points, and fire them based on 4 data points... it's not going to be a good place to work at, and nothing will get done. If you are looking only at the numbers, then at least use proper statistical methods. Or stop treating people like numbers and treat them as humans, this'll give you much more data to work with.

roenxi 2y ago
> I think the only conclusion one can draw from this is that high school statistics is actually useful in real-life. That isn't the conclusion, you can have a management team that all technically understand statistics and they still make the same mistakes. It is depressing to watch it play out in real time. The problem is when people don't identify that workers are just doing what they are told and personal effort isn't the major factor determining system outcomes. Going through the article, the important part about the narrative is the misconception that "performance outcomes are being determined by the worker, so managing the workers will get better performance!". The management is being done by statistics throughout even though the quality of that management is bad.
- wat10000 2y ago
  The management in this game is being done by a deep misunderstanding of statistics. They’re using random noise as a signal and taking action based on it. Actually using statistics would tell you that your numbers are random noise and can’t be used to make sensible decisions. It would tell you that performance outcomes aren’t being determined by the worker. There might be better ways to figure that out (in this case, taking five seconds to look at the process) but statistics will get that job done.
  - roenxi 2y ago
    Well you aren't wrong, but that is a question of identifying what model to fit. If the question is "are the workers in control of the outcome" then sure, they weren't using statistics properly. But if management realises that is the question to ask they also probably won't need statistics because the answer is obviously no in almost all contexts. Management was asking the question "given that the workers control the process and we know that there are differences between people, which workers are performing the best?". And they were doing the best they could under pressure with the incomplete information they had. It was a stupid model, but that is why they are going to an Ed Deming talk - the median manager defaults to stupid and needs a bit of help. Training people in high school stats doesn't necessarily get better results at model selection, some extremely educated and capable people are still stupid about that part.
    - wat10000 2y ago
      They were certainly not doing their best to answer that question! Stats will answer that question correctly, answering “they’re all performing the same, within the margin of error.” Yes, actually understanding what the hell is going on is really helpful, but as a backstop, it’s good to have the basic stats knowledge that you can’t come to any conclusion from a handful of samples.
      - roenxi 2y ago
        No you're not seeing it from the management perspective. They've already selected their model and assumed that performance depends on the worker involved. There isn't any "they’re all performing the same within the margin of error" because that would violate a model assumption. It has a probability of ~0%, there isn't any point checking for it. You're working from a model where the performance could be the same and then doing the obvious statistical checks. Typically in the wild management have sufficient knowledge to do those checks if their model allowed for performance to be equal. But they don't have a model where that is a reasonable possibility so there is nothing to check. They're doing the correct statistics for the wrong model. And the model happens to be really stupid but that is managers for you. You can teach them all that theory and they still won't apply it. The problem isn't usually a gap in statistical knowledge. Take another look at how Deming framed his experiment - if you look closely he is imparting statistical knowledge but that isn't where the real focus is. He's actually attacking the idea that the workers have any influence on outcomes.
        wat10000 2y ago
        I see the management perspective. I’m just saying that perspective is obviously wrong if you have any stats knowledge. If your point is just that management could have that knowledge and fail to apply it, I won’t argue with that. But I definitely wouldn’t call it “doing their best” when that happens.
sowbug 2y ago
The paddle is a random-number generator. Stats are irrelevant, except to measure luck.
- atoav 2y ago
  Yeah, but anybody with statistical knowledge might have been able to guess that this is a random process and that not evetything is down to individual performance. The point of the thought experiment is to make us think of what the system contributes to team/individual performance. In reality every procedd might have a random component, but it might also have a feedback component — e.g. you give a project with many red beats to a team that did well before only to have them (to us: predictably) underperform, then you give it to another team etc. The lesson here is to understand this underperformance doesn't necessarily mean they got worse, it just means they got the harder project to pull off and other teams might have underperformed even more. If you as a manager focus on the totally fictional underperformance of one team vs the other you optimize things that had no bearing on the outcome. In fact you will make things worse as you make yourself believe the (in reality worse) team that only handled red-bead-free projects before will perform better.
  - sowbug 2y ago
    A grounding in statistics shouldn't be needed to see that taking a scoop of mixed beads is likely to be mixed beads. A variant of Gell-Manning amnesia arises when real managers see this demonstration, go back to their real jobs, and think "of course, my team's processes aren't as stochastic as that." But then again, managers are also beads.
- deepnet 2y ago
  The workers control the paddle and the scooping of the beads. A smart worker will eyeball their draw for red and redraw a few times until they get a good draw. There may be other ways to flick off red beads by tilting and tapping. None of this is excluded behaviour at least in this article.
  - wat10000 2y ago

by theamk 2y ago

roenxi 2y ago

> I think the only conclusion one can draw from this is that high school statistics is actually useful in real-life. That isn't the conclusion, you can have a management team that all technically understand statistics and they still make the same mistakes. It is depressing to watch it play out in real time. The problem is when people don't identify that workers are just doing what they are told and personal effort isn't the major factor determining system outcomes. Going through the article, the important part about the narrative is the misconception that "performance outcomes are being determined by the worker, so managing the workers will get better performance!". The management is being done by statistics throughout even though the quality of that management is bad.

wat10000 2y ago
The management in this game is being done by a deep misunderstanding of statistics. They’re using random noise as a signal and taking action based on it. Actually using statistics would tell you that your numbers are random noise and can’t be used to make sensible decisions. It would tell you that performance outcomes aren’t being determined by the worker. There might be better ways to figure that out (in this case, taking five seconds to look at the process) but statistics will get that job done.
- roenxi 2y ago
  Well you aren't wrong, but that is a question of identifying what model to fit. If the question is "are the workers in control of the outcome" then sure, they weren't using statistics properly. But if management realises that is the question to ask they also probably won't need statistics because the answer is obviously no in almost all contexts. Management was asking the question "given that the workers control the process and we know that there are differences between people, which workers are performing the best?". And they were doing the best they could under pressure with the incomplete information they had. It was a stupid model, but that is why they are going to an Ed Deming talk - the median manager defaults to stupid and needs a bit of help. Training people in high school stats doesn't necessarily get better results at model selection, some extremely educated and capable people are still stupid about that part.
  - wat10000 2y ago
    They were certainly not doing their best to answer that question! Stats will answer that question correctly, answering “they’re all performing the same, within the margin of error.” Yes, actually understanding what the hell is going on is really helpful, but as a backstop, it’s good to have the basic stats knowledge that you can’t come to any conclusion from a handful of samples.
    - roenxi 2y ago
      No you're not seeing it from the management perspective. They've already selected their model and assumed that performance depends on the worker involved. There isn't any "they’re all performing the same within the margin of error" because that would violate a model assumption. It has a probability of ~0%, there isn't any point checking for it. You're working from a model where the performance could be the same and then doing the obvious statistical checks. Typically in the wild management have sufficient knowledge to do those checks if their model allowed for performance to be equal. But they don't have a model where that is a reasonable possibility so there is nothing to check. They're doing the correct statistics for the wrong model. And the model happens to be really stupid but that is managers for you. You can teach them all that theory and they still won't apply it. The problem isn't usually a gap in statistical knowledge. Take another look at how Deming framed his experiment - if you look closely he is imparting statistical knowledge but that isn't where the real focus is. He's actually attacking the idea that the workers have any influence on outcomes.
      - wat10000 2y ago
        I see the management perspective. I’m just saying that perspective is obviously wrong if you have any stats knowledge. If your point is just that management could have that knowledge and fail to apply it, I won’t argue with that. But I definitely wouldn’t call it “doing their best” when that happens.

sowbug 2y ago

The paddle is a random-number generator. Stats are irrelevant, except to measure luck.

atoav 2y ago
Yeah, but anybody with statistical knowledge might have been able to guess that this is a random process and that not evetything is down to individual performance. The point of the thought experiment is to make us think of what the system contributes to team/individual performance. In reality every procedd might have a random component, but it might also have a feedback component — e.g. you give a project with many red beats to a team that did well before only to have them (to us: predictably) underperform, then you give it to another team etc. The lesson here is to understand this underperformance doesn't necessarily mean they got worse, it just means they got the harder project to pull off and other teams might have underperformed even more. If you as a manager focus on the totally fictional underperformance of one team vs the other you optimize things that had no bearing on the outcome. In fact you will make things worse as you make yourself believe the (in reality worse) team that only handled red-bead-free projects before will perform better.
- sowbug 2y ago
  A grounding in statistics shouldn't be needed to see that taking a scoop of mixed beads is likely to be mixed beads. A variant of Gell-Manning amnesia arises when real managers see this demonstration, go back to their real jobs, and think "of course, my team's processes aren't as stochastic as that." But then again, managers are also beads.
deepnet 2y ago
The workers control the paddle and the scooping of the beads. A smart worker will eyeball their draw for red and redraw a few times until they get a good draw. There may be other ways to flick off red beads by tilting and tapping. None of this is excluded behaviour at least in this article.
- wat10000 2y ago