7 ms·
Statistical process control after W. Edwards Deming (2020)
- neverartful 5y agoMy first job out of college (undergraduate) was working for a small consulting firm that did a lot of work in this area for the petrochemical industry and their suppliers. The owner had a PhD in decision sciences (statistics) and he did consulting work on statistical process control (SPC). This was my first exposure to this field and was so fascinated by it that after working there for a year, I quit my job so that I could go back to school full time and pursue a Masters degree in this area, but with focus on management information systems (MIS). Ironically, my work after graduate school has never touched on SPC or quality control. It's unfortunate that Deming (and many others) never really got the attention that their methods deserved.
- cafard 5y agoTwenty-five or thirty years ago, one hear a great deal about total quality management (TQM).
- neverartful 5y agoYep, it was back in the early 1990s.
- mch82 5y agoUntil recently, there hasn’t been enough data to apply statistical process control in a lot of fields. Chemical process companies (like Kodak used to be) have used SPC. “Web scale” companies commonly talk about running experiments now. Lots of people talk about A/B testing, which is an extremely simplistic example of SPC. I’ve been anticipating more advanced SPC tools to replace A/B testing for years, but am not aware of any. Maybe I should have created one. Now I wonder if Machine Learning might pass up SPC as the primary method by which people identify and control key process variables.
- neverartful 5y agoActually, it does not take a huge amount of data to make use of statistical process control. In fact, it's so little that someone should be able to do it with paper, pencil, and calculator. The statistical technique I think that's better able to displace A/B testing is design of experiments (DOE). This approach also does not require that much data to implement. There was an excellent youtube series on it where the presenter used popping popcorn for his experiments. I can't seem to find it now.
- analog31 5y agoI've observed DOE in use. It's a statistical method at its core, and as such potentially suffers from the replication crisis.
- Guthur 5y agoI'm hoping that will change. I'm a big fan of Deming and Drucker, they've both got really insightful thoughts on business. My goal is to work more of Deming into knowledge work (software) to develop meaningful improvement into the process and product while leveraging Drucker for better effectiveness of the individual.
- larrydag 5y agoGE and other companies tried to implement Demings approach. They called it Six Sigma. Those that used Deming's concepts were successful. Yet too many misunderstood the idea of SPC and used in areas that was not meant for quality control. For instance trying to use SPC in the managerial process itself.
- mch82 5y agoSix Sigma is to the field of Statistical Process Control what PowerPoint templates are to the field of graphic design. The biggest problem with Six Sigma is many of the people who roll it out understand how to follow the instructions in a Six Sigma book, but don’t understand the underlying statistics well enough to recognize when the templates will fail or need to be changed.
- hencq 5y agoAmen. The principles behind Six Sigma are solid. They're just a repackaging of Deming's SPC principles after all. However, too often it devolves into a box ticking exercise. That being said, in my experience just educating everyone in your organization on the basic principles (e.g. what Six Sigma would call yellow belt) tends to be quite valuable. Even without a lot of statistics, teaching people to first properly define a problem statement and emphasizing finding root causes before taking action tends to lead to better decision making.
- eplanit 5y agoGawd, Six Sigma was an ever-present factor in all our work when I was at IBM in the late 80's to early 90's. I have complete respect for Deming, but when anything becomes "religion" it gets damaged or mis-used a lot, as you say. In our software group, we were constantly battling with management that wanted to start to look at software development as a factory-like process, and to apply factory QA processes.
- devonkim 5y agoCargo culting one’s way to attempt obtaining results seems like the default approach of leadership unable to really adapt the proper lessons from other organizations and apply them to their own. If I as a person look up to a world class bodybuilder as a scrawny weakling, the solution to achieve that goal isn’t to just eat like a bodybuilder but to train like one and that usually means dropping other activities like playing a bunch of video games (not that it’s mutually exclusive - see Henry Cavill). But we as humans do the same thing all the time (how many people think of rich people in terms of consumption or some sets of personalities from popular media rather than to look at the data fully). However, there is a lot to learn and enjoy in the process, and companies that do well learn to have clear goals and processes that seem genuine and that employees can connect with. When people stop feeling this connection, they simply stop doing the habits at the individual (not working out, relapsing, etc) or disengage in organizations (burn out, disillusionment, etc)
- curiouscats 5y agoHere are some online resources on Deming's ideas https://curiouscat.com/management/deming/online
- deleted 5y ago[deleted]
- jihadjihad 5y agoW. Edwards Deming is the source of one of my favorite quotes: "In God we trust; all others must bring data." One of the fascinating things about Deming's work is that Japan's automobiles went from being referred to pejoratively as "Jap Crap" to symbols of quality (Toyota, Honda, Subaru, Nissan, Mazda, etc.) within a short few decades--quite a feat for a nation struggling to regain its industrial footing after WWII.
- Tomte 5y agoDeming used the quote, but did not coin it: https://quoteinvestigator.com/2017/12/29/god-data/ https://quoteinvestigator.com/2017/12/29/god-data/
- heymijo 5y agoI'm curious what makes that your favorite quote. It gives me pause, mainly because I've seen it abused, ironically, often in contrast to some of Deming's core ideas. YMMV
- xyzzy21 5y agoThe products I design/sell generate the inputs for this kind of thing for the semiconductor industry. It always makes cocktail party conversation complicated.
- mch82 5y agoOut of curiosity, do you mean that your products generates template forms for collecting statistical sample data (like Minitab), or that your products collect measurements that are fed into tools like Minitab as data?
- shitmonger 5y agoFuck you vax. Fuck your vax pass. Fuck you.
- WWWWH 5y agoThis is a great article, but is there a good modern source (say an introductory text book) on these ideas? I've a vague memory of doing this as a first year undergraduate, but that was a long time ago. And stupidly, I didn't think I'd ever need them...
- Zigurd 5y agoDeming wrote very accessible books. Out of the Crisis is a general audience book but it covers statistical quality analysis in enough depth to understand how it works.
- wheelinsupial 5y agoWhich parts of the concepts are you looking for? Douglas Montgomery has a textbook called Statistical Process Control that covers control charts and many other statistical techniques.
- Witoso 5y agoI recommend "Understanding Variation: The Key to Managing Chaos" by Donald J. Wheeler, colleague of Deming. Good intro to SPC.
- WWWWH 5y agoThanks for the tips guys. I’ll check them out. I also pulled a copy of Box’s essays off the shelf, and while he is obviously mainly interested in experimental design he does drop in a few references to oldish books on SPC.
- s-luv-j 5y ago>"You can not inspect quality into a product." ~Deming Yep. This applies double to software. Putting software under load and trending health measures to look for two standard deviations variance is very similar to how Deming instrumented production lines. The neat thing that software enables for both automobile manufacturing and software products themselves is that those exact same health measures can be trended and analyzed after they get into the hands of consumers. If your quality apparatus is purely inspection based, you will live your life one escalation to the next.
- onion2k 5y agoI've spent much of the past few years trying to work out if this means I should stop relying on pull requests to catch problems, and put something in place ahead of writing code instead, but so far I've not figured out an alternative that works. If anyone has any ideas, please share them.
- s-luv-j 5y agoOne of the reasons why pushing change straight to production with A/B testing works so well is that you can expose your new change to statistically significant real user load and, assuming your have decent instrumentation, find the problems. This does mean that a subset of your users get exposed to those problems. A more user friendly way would be to have a scaled down replica of production setup with simulators that mimic user behavior. Assuming the development team actually believes in and tends this environment, they will find the lion's share of bugs that commonly slip through peer review.
- afarrell 5y agoPair-drafting pseudocode with a colleague into a Dropbox Paper document is a great way to evaluate tradeoffs.
- Zigurd 5y agoDeming is talking about repeatable manufacturing processes. The point of Statistical Process Control is that you sample metrics that tell you if the process will produce products reliably within tolerances. Software is completely different, it is hardly ever a repeatable process to create software. Making the same software over and over is nonsensical, but making a billion identical contact lenses is exactly what you want. There are forms of software "inspection," like pair programming, that amount to what Deming says you should not do in manufacturing: Inspect everything. But pair programming is pretty rare. Other practices like code review have been called into question. So it may still be true that you can't inspect your way to quality in software, but not for the same reasons that it is a bad idea in many cases in manufacturing.
- User23 5y agoDeming gets the credit, but in Japan it was Homer Sarasohn[1][2] who did the foundational work[3]. In fact, he handed the reins off to Deming. One particularly interesting thing is that, at least once upon a time, Canon didn't even bother inspecting their copiers and printers, because the variance was so low it was a waste of time[4]. [1] https://en.wikipedia.org/wiki/Homer_Sarasohn https://en.wikipedia.org/wiki/Homer_Sarasohn [2] https://honoringhomer.net/ https://honoringhomer.net/ [3] https://apnews.com/article/e2908339e16a48dc8efe37d77a905daa https://apnews.com/article/e2908339e16a48dc8efe37d77a905daa [4] http://qualiticien.over-blog.com/article-679016.html http://qualiticien.over-blog.com/article-679016.html
- rramadass 5y agoThis is great; had never known about Homer Sarasohn before. So many amazing people have been lost to the vagaries of history that it behooves us to research and rediscover our "lost" heroes. Thanks for all the links; can you recommend some introductory books on the subject?
- User23 5y agoOn the subject of "lost" heroes? Not really, I've just come across a few in the course of my life. It would certainly make a great book though!
- rramadass 5y agoI was asking about books on Statistical Process/Quality Control though a book on our "lost heroes" would make a more enjoyable reading :-) It seems Japan's resurgence after WWII owes a lot to the enlightened policies of a "few" Americans and their willingness to actually follow through with concrete actions. I wonder how many Japanese really know about their own history and what they really owe the US.
- winslett 5y agoI stumbled upon Deming's "Out of the Crisis" when wandering through the library in school. It has changed how I thought about system improvement ever since. I use the things I learned for marketing, product, and engineering systems. Want to improve a system? Before doing anything, collect data and measure. A system must be stable, else you cannot improve it. Marketing campaigns. Funnel optimizations. Performance improvement. Most "stability" is achieved by simple things like squashing bugs (campaign bugs, performance bugs, or UX bugs). Once you achieve stability of output (i.e. low standard deviation), then improve the system. Simple, uh?
- marcosdumay 5y agoWell, keep in mind that "improvement" here implies non-transformative change (we usually call this "optimization" on informatics). If you are not sure you have the correct system running, stabilizing it is the last thing you want.
- avs733 5y agoI think that may be a too narrow vision of “transformative” improvement. I worked in a factory where one of the biggest, and hardest, improvements we made was coaching the managers and engineers to leave the technicians alone when they were doing maintenance. We collected data on how often they were interrupted, how much time it cost, and how interruptions could lead to RIT changes in procedures that affected quality. Previously the engineers and managers saw the checking as helping because they could se ethe status of things and observe and answer questions, many would even lend a hand. Oversight and engagement was seem as a path to improved quality. To let go of that was a huge transformation. It started with data - a lot of data - that allowed us to identify that weekend and night maintenance generally went faster and better.
- marcosdumay 5y agoWell, I claim that is a way too broad definition of "transformative". The people kept creating the same products, by almost the same procedure.
- Jtsummers 5y agoThe funnel experiment examples are interesting. The fourth case (T' = H) is a great example of normalization of deviance [0]. Without a clearly communicated target to return to, you reliably shift further and further from that initial target, not even oscillating around the target like in the third case. [0] https://en.wikipedia.org/wiki/Normalization_of_deviance https://en.wikipedia.org/wiki/Normalization_of_deviance
- hbarka 5y agoThe irony is that Deming’s philosophy was much better received in Japan that in the United States. American cars were considered crappy during the 70s and 80s while the Japanese were iterating their quality practices. Deming was not the only philosopher-guru of manufacturing quality. There was Peter Drucker, Walter Shewhart (Shewhart cycle), Joseph Juran, Shigeo Shingo, Philip Crosby (quality is free, rework is costly), Taguchi (Taguchi methods), Ishikawa (fishbone). JIT, Kanban, Kaizen funny enough are Japanese manufacturing terms that have made its way into our programming dialect.
- hef19898 5y agoAlso funny that the US defense industry was very close to some of these concepts during WW2. Only to have the Japanese adopt them really efficiently afterwards.