4 ms·
That's the point. So first, not all science requires an RCT. Dividing expiremental subjects into study and control groups is one way of doing science. It's not
by function_seven 5y ago
That's the point.
So first, not all science requires an RCT. Dividing expiremental subjects into study and control groups is one way of doing science. It's not the only way.
In this case, this is a concrete demonstration of just how much variance can emerge from a "statistically neutral" process. The systemic flaws are part of the demonstration. What appear at first glance to be identical tools, inputs, and processes are in fact subtly different. The demo shows management types that their charts and graphs cannot always be relied upon to differentiate performance levels among staff. The system itself must also be scrutinized. If Bob's ad campaigns are outperforming Alice's by 20% in the first quarter, it doesn't necessarily mean Bob is a marketing genius and Alice needs a PIP.
A computer simulation would not have nearly as powerful effect on most people as a live demonstration using real beads. And the imperfections in the paddles is something that naturally arises when they're physically made, but would have to be tuned by the programmer building the simulation. Which would lead to questions about "just how did they decide what variances would come into play?"
- pakitan 5y agoI think I get it now. The point of the experiment is to ELI5 the concept of variance to management types who skipped statistics classes :) Could be useful for some bosses I had :)
- function_seven 5y agoSo I'm watching a video of this right now[0], and it's even more enlightening than I figured it would be! Deming makes comments throughout the demonstration that I swear I've heard in the real world. For example, one worker—whose previous results put him on probation (he had 12 red beads)—managed to have only 6 the next day. "Looks like probation worked". Meanwhile another worker—previously scoring 5, and getting a merit-based raise from it—did poorly with 12. The remark: "That raise went to his head. He's getting lazy". So yeah, the value of this is in the actual doing of it. [0] https://www.youtube.com/watch?v=7pXu0qxtWPg https://www.youtube.com/watch?v=7pXu0qxtWPg
- ignoramous 5y agoNot just variance but also the concept of essential complexity: One cannot possibly hope to control all the variables, and so one must strive to continually seek to improve the system ("monitoring/alerting") as defects emerge ("hidden factories"), conduct experiments to evaluate predictions against actual outcomes ("a/b tests"), to not erroneously change systems and processes to fix "special-causes" [but understand that most causes of quality degradation are "commons-causes", which has to be fixed by changing systems and processes ("root-cause analysis")], to not drive down costs for the sake of it but rather seek to drive down long-term costs ("ecosystem"), to have a more humanist approach to management... among a host of other concepts. If you are curious (and have the stomach for some MBA-ness), this article (four parts) should be well worth your time: https://archive.is/tXJhw https://archive.is/tXJhw
- jiggawatts 5y agoA real world example is that I do programming on a high-end workstation laptop. My coworkers use old budget laptops. This is not in their control — they’re victims of corporate policy. Does it influence quality in complex and hard to quantify ways? Most assuredly…