3 ms·

I think the conflation of software testing and Quality Assurance is a major shame of the tech industry. It's warped the understanding of the field of QA and pa

I think the conflation of software testing and Quality Assurance is a major shame of the tech industry. It's warped the understanding of the field of QA and pays little respect to Deming's ideas and goals.

mikey_p 3y ago
Check out the episode of Gene Kim's podcast with Elisabeth Hendrickson, it does such a great job exploring the myths and realities around software testing. https://itrevolution.com/podcast/the-idealcast-episode-3/ https://itrevolution.com/podcast/the-idealcast-episode-3/
- fuzzfactor 3y ago
  Thanks for this, also there is a reference to her excellent paper from 2000 "Better Testing - Worse Quality?" where she emphasizes the need for engineers to do their own testing for the most part. Still posted in a few places which was worth reading. Inherent quality depends completely on the factory floor. But I would say with code development bugs all you can expect the engineers to catch are the ones that can be found during their combined development/testing progress as it goes along. So they should honestly think it's perfect before turning it over to dedicated testers who will put their full effort into trying to find defects that were possible to be overlooked up until that point. Kind of like having a machinist making motor shafts who can easily see when they are coming out bent and fix it so that doesn't happen, plus manually gage his parts as he goes along to make sure the diameter is holding steady. He's doing all he can to produce what then look like perfect parts. Next the QC guy comes along and using automated measurements, tests the whole shaft along its entire length, and finds a few below-spec items that slipped through the cracks. Feedback to engineering can be used to make sure it doesn't happen again. The engineer is building value as intended performance is being accomplished, the tester is not building value from scratch or doing invoiceable work but they are essential to make the most of the value that is created.
projektfu 3y ago
It reflects a similar problem in industry. If you look at the trade magazines during the Renaissance in quality management, many paid lip service to the ideas of statistical quality control while trying to sell gadgets and data processing systems to enable 100% inspection. That's because leadership change can't be bought and requires work from leadership. QA, I've always felt, is the wrong approach, in that I don't think quality is something you can assure, especially if it's not designed into the product. Whether you're talking about software testing, ISO 9000, GMP, or whatever, these approaches are good at providing a lot of activity and reports and not necessarily coordinated with quality product. They may be an essential part of achieving quality, but they don't cause it. The US government, for a long time, was killing itself with QA that allowed purchasing from anywhere that could supply an "acceptable quality level", meaning on average 3% of any item was defective and an assembly could have a high likelihood of not working. Similarly to software process, quality management gets fetishized and cargo-culted, so you end up with six sigma, "zero defects", total quality management, etc. I read that Deming once saw a TQM placard and said something like, "I hope I'm not guilty by association."
- fuzzfactor 3y ago
  >software testing, ISO 9000, GMP Major difference is that ISO and GMP are intended to be mandated in a way that can only result in excess mindless behavior to some extent, but software testing can actually be made to avoid that. >may be an essential part of achieving quality, but they don't cause it. In the case of ISO 9000 with 20/20 hindsight it's more like it can be bureaucratically essential to gain acceptance among those orgs who don't know quality enough to recognize it on their own. This was primarily an anti-competitive standard to keep less-bureaucratic products (from less-developed counties as well as the US) from outside Europe away from the continent, issued during an unfavorable currency situation before the Euro or even the EU existed. If Deming were here today we'd be talking about the way ISO 9000 brought the continuous improvement process to its knees by requiring additional bureaucracy which diverts more resources than were once being more effectively deployed in true quality improvement efforts. The result has been a freezing in place of the quality that was reached when ISO 9000 accreditation is achieved. It's a huge milestone and the effort just to maintain that level of recognition is entirely focused on no further changes being made if at all possible, due to the extreme added difficulty of pushing a change up or down through the entire organization. Not every one is frozen back to 1987, but the pervasiveness for so long has had an exorbitantly unmeasurable cost. Especially in terms of reduced innovation. Plus many US companies that were in production before ISO were making higher quality products then than they do now, it really helps to see this if you were there at the time.

mikey_p 3y ago

Check out the episode of Gene Kim's podcast with Elisabeth Hendrickson, it does such a great job exploring the myths and realities around software testing. https://itrevolution.com/podcast/the-idealcast-episode-3/ https://itrevolution.com/podcast/the-idealcast-episode-3/

fuzzfactor 3y ago
Thanks for this, also there is a reference to her excellent paper from 2000 "Better Testing - Worse Quality?" where she emphasizes the need for engineers to do their own testing for the most part. Still posted in a few places which was worth reading. Inherent quality depends completely on the factory floor. But I would say with code development bugs all you can expect the engineers to catch are the ones that can be found during their combined development/testing progress as it goes along. So they should honestly think it's perfect before turning it over to dedicated testers who will put their full effort into trying to find defects that were possible to be overlooked up until that point. Kind of like having a machinist making motor shafts who can easily see when they are coming out bent and fix it so that doesn't happen, plus manually gage his parts as he goes along to make sure the diameter is holding steady. He's doing all he can to produce what then look like perfect parts. Next the QC guy comes along and using automated measurements, tests the whole shaft along its entire length, and finds a few below-spec items that slipped through the cracks. Feedback to engineering can be used to make sure it doesn't happen again. The engineer is building value as intended performance is being accomplished, the tester is not building value from scratch or doing invoiceable work but they are essential to make the most of the value that is created.

projektfu 3y ago

It reflects a similar problem in industry. If you look at the trade magazines during the Renaissance in quality management, many paid lip service to the ideas of statistical quality control while trying to sell gadgets and data processing systems to enable 100% inspection. That's because leadership change can't be bought and requires work from leadership. QA, I've always felt, is the wrong approach, in that I don't think quality is something you can assure, especially if it's not designed into the product. Whether you're talking about software testing, ISO 9000, GMP, or whatever, these approaches are good at providing a lot of activity and reports and not necessarily coordinated with quality product. They may be an essential part of achieving quality, but they don't cause it. The US government, for a long time, was killing itself with QA that allowed purchasing from anywhere that could supply an "acceptable quality level", meaning on average 3% of any item was defective and an assembly could have a high likelihood of not working. Similarly to software process, quality management gets fetishized and cargo-culted, so you end up with six sigma, "zero defects", total quality management, etc. I read that Deming once saw a TQM placard and said something like, "I hope I'm not guilty by association."

fuzzfactor 3y ago
>software testing, ISO 9000, GMP Major difference is that ISO and GMP are intended to be mandated in a way that can only result in excess mindless behavior to some extent, but software testing can actually be made to avoid that. >may be an essential part of achieving quality, but they don't cause it. In the case of ISO 9000 with 20/20 hindsight it's more like it can be bureaucratically essential to gain acceptance among those orgs who don't know quality enough to recognize it on their own. This was primarily an anti-competitive standard to keep less-bureaucratic products (from less-developed counties as well as the US) from outside Europe away from the continent, issued during an unfavorable currency situation before the Euro or even the EU existed. If Deming were here today we'd be talking about the way ISO 9000 brought the continuous improvement process to its knees by requiring additional bureaucracy which diverts more resources than were once being more effectively deployed in true quality improvement efforts. The result has been a freezing in place of the quality that was reached when ISO 9000 accreditation is achieved. It's a huge milestone and the effort just to maintain that level of recognition is entirely focused on no further changes being made if at all possible, due to the extreme added difficulty of pushing a change up or down through the entire organization. Not every one is frozen back to 1987, but the pervasiveness for so long has had an exorbitantly unmeasurable cost. Especially in terms of reduced innovation. Plus many US companies that were in production before ISO were making higher quality products then than they do now, it really helps to see this if you were there at the time.