3 ms·

I don't get it. This "experiment" could have been replicated by a simple computer simulation, given that worker output is entirely random. The supposed moral of

I don't get it. This "experiment" could have been replicated by a simple computer simulation, given that worker output is entirely random. The supposed moral of the story is that system design defines outcome, not individual performance but how does that even count as "science" when you don't have control and experimental group. He designed a system with inherent flaws and, surprise, it has flaws. We can see there is variance in "productivity" but we have no idea how this same variance would have affected output if workers actually had agency.

function_seven 5y ago
That's the point. So first, not all science requires an RCT. Dividing expiremental subjects into study and control groups is one way of doing science. It's not the only way. In this case, this is a concrete demonstration of just how much variance can emerge from a "statistically neutral" process. The systemic flaws are part of the demonstration. What appear at first glance to be identical tools, inputs, and processes are in fact subtly different. The demo shows management types that their charts and graphs cannot always be relied upon to differentiate performance levels among staff. The system itself must also be scrutinized. If Bob's ad campaigns are outperforming Alice's by 20% in the first quarter, it doesn't necessarily mean Bob is a marketing genius and Alice needs a PIP. A computer simulation would not have nearly as powerful effect on most people as a live demonstration using real beads. And the imperfections in the paddles is something that naturally arises when they're physically made, but would have to be tuned by the programmer building the simulation. Which would lead to questions about "just how did they decide what variances would come into play?"
- pakitan 5y ago
  I think I get it now. The point of the experiment is to ELI5 the concept of variance to management types who skipped statistics classes :) Could be useful for some bosses I had :)
  - function_seven 5y ago
    So I'm watching a video of this right now[0], and it's even more enlightening than I figured it would be! Deming makes comments throughout the demonstration that I swear I've heard in the real world. For example, one worker—whose previous results put him on probation (he had 12 red beads)—managed to have only 6 the next day. "Looks like probation worked". Meanwhile another worker—previously scoring 5, and getting a merit-based raise from it—did poorly with 12. The remark: "That raise went to his head. He's getting lazy". So yeah, the value of this is in the actual doing of it. [0] https://www.youtube.com/watch?v=7pXu0qxtWPg https://www.youtube.com/watch?v=7pXu0qxtWPg
  - ignoramous 5y ago
    Not just variance but also the concept of essential complexity: One cannot possibly hope to control all the variables, and so one must strive to continually seek to improve the system ("monitoring/alerting") as defects emerge ("hidden factories"), conduct experiments to evaluate predictions against actual outcomes ("a/b tests"), to not erroneously change systems and processes to fix "special-causes" [but understand that most causes of quality degradation are "commons-causes", which has to be fixed by changing systems and processes ("root-cause analysis")], to not drive down costs for the sake of it but rather seek to drive down long-term costs ("ecosystem"), to have a more humanist approach to management... among a host of other concepts. If you are curious (and have the stomach for some MBA-ness), this article (four parts) should be well worth your time: https://archive.is/tXJhw https://archive.is/tXJhw
- jiggawatts 5y ago
  A real world example is that I do programming on a high-end workstation laptop. My coworkers use old budget laptops. This is not in their control — they’re victims of corporate policy. Does it influence quality in complex and hard to quantify ways? Most assuredly…
advisedwang 5y ago
This experiment is something he did in classes etc so people could _experience_ the obvious idiocy in trying to manage individuals for system behaviour. His point is that ALL actual work is also dominated by system behaviour, just more subtly, and managers must focus on the systems and not worker performance.
- pakitan 5y ago
  > His point is that ALL actual work is also *dominated* by system behaviour If that's his point, it seems obviously wrong. Some work, like in the experiment is dominated by system behavior. For others, system would play a much smaller role. For example, 2 people cranking code in a startup. No matter what system you apply, if they are not good programmers, nothing of value will come out.
  - kqr 5y ago
    That's also missing the point somewhat. Put the best two programmers in the world in a shitty system that rewards them for the wrong things and they will produce garbage. Put mediocre programmers in a fantastic system that brings out the best in their collaboration and you might actually get to market sooner and better than the other group.
  - salawat 5y ago
    Something will most certainly come out. You're just not defining the system. If I define the system as "one programmer is responsible for looking at the desired product and writing specifications" and the other programmer is to translate specification into programming code, and never shall one do the other's job, I assure you, the best programmers in the world will produce shit over time. Randomly swap in two new actors with different life experiences into the same spots to do the same work, and you'll still get shit. If in the unlikely event, you get amazing work, it's not that the people doing it were special; it's just anothe outlier in the data stream. Add in the emotional toll of working as hard as possible to succeed but never being able to meet prescribed quality levels? A system is perfectly tuned to produce the results it does. Want different results? Change the system. That is Deming's point. We have a tendency to blame variance in a system on the human actors immediately proximal, instead of paying attention to the actual significant constraints. This is an important lesson to management types, as they are to process/system what a programmer is to a computer. The planners cast the dice for downstream long before downstream can do anything about it, and in many corporate setups, top down works just fine, but bottom up never gets any attention.
lupire 5y ago
It's a demo experiment, like whe the physics teacher swinga a heavy pendulum at their own nose, or shoots a BB gun at a falling toy.

function_seven 5y ago

That's the point. So first, not all science requires an RCT. Dividing expiremental subjects into study and control groups is one way of doing science. It's not the only way. In this case, this is a concrete demonstration of just how much variance can emerge from a "statistically neutral" process. The systemic flaws are part of the demonstration. What appear at first glance to be identical tools, inputs, and processes are in fact subtly different. The demo shows management types that their charts and graphs cannot always be relied upon to differentiate performance levels among staff. The system itself must also be scrutinized. If Bob's ad campaigns are outperforming Alice's by 20% in the first quarter, it doesn't necessarily mean Bob is a marketing genius and Alice needs a PIP. A computer simulation would not have nearly as powerful effect on most people as a live demonstration using real beads. And the imperfections in the paddles is something that naturally arises when they're physically made, but would have to be tuned by the programmer building the simulation. Which would lead to questions about "just how did they decide what variances would come into play?"

pakitan 5y ago
I think I get it now. The point of the experiment is to ELI5 the concept of variance to management types who skipped statistics classes :) Could be useful for some bosses I had :)
- function_seven 5y ago
  So I'm watching a video of this right now[0], and it's even more enlightening than I figured it would be! Deming makes comments throughout the demonstration that I swear I've heard in the real world. For example, one worker—whose previous results put him on probation (he had 12 red beads)—managed to have only 6 the next day. "Looks like probation worked". Meanwhile another worker—previously scoring 5, and getting a merit-based raise from it—did poorly with 12. The remark: "That raise went to his head. He's getting lazy". So yeah, the value of this is in the actual doing of it. [0] https://www.youtube.com/watch?v=7pXu0qxtWPg https://www.youtube.com/watch?v=7pXu0qxtWPg
- ignoramous 5y ago
  Not just variance but also the concept of essential complexity: One cannot possibly hope to control all the variables, and so one must strive to continually seek to improve the system ("monitoring/alerting") as defects emerge ("hidden factories"), conduct experiments to evaluate predictions against actual outcomes ("a/b tests"), to not erroneously change systems and processes to fix "special-causes" [but understand that most causes of quality degradation are "commons-causes", which has to be fixed by changing systems and processes ("root-cause analysis")], to not drive down costs for the sake of it but rather seek to drive down long-term costs ("ecosystem"), to have a more humanist approach to management... among a host of other concepts. If you are curious (and have the stomach for some MBA-ness), this article (four parts) should be well worth your time: https://archive.is/tXJhw https://archive.is/tXJhw
jiggawatts 5y ago
A real world example is that I do programming on a high-end workstation laptop. My coworkers use old budget laptops. This is not in their control — they’re victims of corporate policy. Does it influence quality in complex and hard to quantify ways? Most assuredly…

advisedwang 5y ago

This experiment is something he did in classes etc so people could _experience_ the obvious idiocy in trying to manage individuals for system behaviour. His point is that ALL actual work is also dominated by system behaviour, just more subtly, and managers must focus on the systems and not worker performance.

pakitan 5y ago
> His point is that ALL actual work is also *dominated* by system behaviour If that's his point, it seems obviously wrong. Some work, like in the experiment is dominated by system behavior. For others, system would play a much smaller role. For example, 2 people cranking code in a startup. No matter what system you apply, if they are not good programmers, nothing of value will come out.
- kqr 5y ago
  That's also missing the point somewhat. Put the best two programmers in the world in a shitty system that rewards them for the wrong things and they will produce garbage. Put mediocre programmers in a fantastic system that brings out the best in their collaboration and you might actually get to market sooner and better than the other group.
- salawat 5y ago
  Something will most certainly come out. You're just not defining the system. If I define the system as "one programmer is responsible for looking at the desired product and writing specifications" and the other programmer is to translate specification into programming code, and never shall one do the other's job, I assure you, the best programmers in the world will produce shit over time. Randomly swap in two new actors with different life experiences into the same spots to do the same work, and you'll still get shit. If in the unlikely event, you get amazing work, it's not that the people doing it were special; it's just anothe outlier in the data stream. Add in the emotional toll of working as hard as possible to succeed but never being able to meet prescribed quality levels? A system is perfectly tuned to produce the results it does. Want different results? Change the system. That is Deming's point. We have a tendency to blame variance in a system on the human actors immediately proximal, instead of paying attention to the actual significant constraints. This is an important lesson to management types, as they are to process/system what a programmer is to a computer. The planners cast the dice for downstream long before downstream can do anything about it, and in many corporate setups, top down works just fine, but bottom up never gets any attention.

lupire 5y ago

It's a demo experiment, like whe the physics teacher swinga a heavy pendulum at their own nose, or shoots a BB gun at a falling toy.