3 ms·

> We have software QA departments that operate very much like manufacturing or aerospace QA. I don’t work in this industry, but this seems fairly ridiculous on

by amluto 3y ago

> We have software QA departments that operate very much like manufacturing or aerospace QA. I don’t work in this industry, but this seems fairly ridiculous on its face: software is not at all like manufacturing. In manufacturing, there’s a design and a manufacturing process, and a critical function of QA is ensuring that the manufactured produce is manufactured to spec. With software, the software is written, compiled, and then repeatedly copied. And something should verify that it’s copied correctly, but this is straightforward and boring. So software QA ought to be much more like the kind of validation that happens when designing hardware, not like the kind of testing and validation that happens as products are manufactured.

irrational 3y ago
Ideally there should be a solid spec written and then qa can test against the spec. Maybe there is somewhere that does write solid specs, including accounting for corner cases, but in my 25 years working professionally in the industry, I’ve never seen it.
- stickfigure 3y ago
  The only complete and precise specification of software is the code itself. If some other form of specification was complete, we would be able to auto-generate the code.
  - cogwheel 3y ago
    This is beside the point. The code specifies what the product _is_, not what it "should" be. If you ask for a word processor and I deliver a perfectly bug-free and feature-complete calculator would you really believe it lived up to spec?
    - _gabe_ 3y ago
      This is also beside the point. I think both of you are trying to warn against the dangers that lie on both sides of this coin: people can invest too heavily in a specification and waste an enormous amount of time, and people can immediately jump into coding and code something that does not do what it was intended to do. Like with all things in life, there’s a balance between these two extremes that’s correct. You need some level of specification so you know what you’re building, but you have to keep in mind that the final code defines what the behavior truly is. Sometimes, that behavior unintentionally becomes part of the specification because users begin to rely on it. I do like the fact that you both used hyperbole to succinctly illustrate the dangers of veering too far in either direction though :)
      - stickfigure 3y ago
        A (human language) specification is simply _enough_ information about a system that a human can figure out the intention of the author. The smarter and more context-rich the human, the simpler the specification can be. The dumber and less context-rich the human, the closer the specification needs to be to code. It's asymptotic. By the time you reach a human who is as dumb as an actual computer, the specification _is_ the code.
- _glass 3y ago
  I work in MedTech. We do this. A design has to be reviewed by QA, and is then tested, and the test is reviewed again. So just to counter the narrative, there are companies that do that, and it is working. In other jobs I also saw the cargo cult of QA. But in some industries it is just crucial, otherwise the pressure is too high to cut corners to implement something. It is a good mechanism to counter the need to move fast and break things.
- sgerenser 3y ago
  I work in software for CPU design/verification. Even here, where in theory there should be a rock-solid spec, there's not. There's a 12,000 page architectural specification, which is very helpful for specifying all the end-user visible state. But the microarchitectural specification is scattered all over different PDFs, visio docs, excel sheets, and sometimes the only spec is the RTL code itself.
legacynl 3y ago
> this seems fairly ridiculous on its face I think that's why people always tell each other to not take things at face value. Of course there is a big difference between sw and hw QA, in the thing that they test, and how they test them. But they are also very similar. Any QA department has to think about ways that things can go wrong, and what things to test for, how to test, which testing methods, which standards to handle, keeping certifications, etc. During testing you also need to keep reevaluating if you actually are catching each problem/bug and how to implement changes in your company that decreases the amount of problems or increase the amount that you catch. I think in that way there's a lot of overlap in thinking about business processes and how to identify problems with them. Of course once a specific binary gets tested and approved by QA it shouldn't matter if it gets copied or whatever as long as you make sure its the same binary (by a checksum for example). But still making sure that errors don't reach the customer, is vital in any QA. If errors does happen, QA is the department that can make sure that it doesn't happen again. And ofc be able to proof in court that you did your due diligence if something does happen.

by amluto 3y ago

irrational 3y ago

Ideally there should be a solid spec written and then qa can test against the spec. Maybe there is somewhere that does write solid specs, including accounting for corner cases, but in my 25 years working professionally in the industry, I’ve never seen it.

stickfigure 3y ago
The only complete and precise specification of software is the code itself. If some other form of specification was complete, we would be able to auto-generate the code.
- cogwheel 3y ago
  This is beside the point. The code specifies what the product _is_, not what it "should" be. If you ask for a word processor and I deliver a perfectly bug-free and feature-complete calculator would you really believe it lived up to spec?
  - _gabe_ 3y ago
    This is also beside the point. I think both of you are trying to warn against the dangers that lie on both sides of this coin: people can invest too heavily in a specification and waste an enormous amount of time, and people can immediately jump into coding and code something that does not do what it was intended to do. Like with all things in life, there’s a balance between these two extremes that’s correct. You need some level of specification so you know what you’re building, but you have to keep in mind that the final code defines what the behavior truly is. Sometimes, that behavior unintentionally becomes part of the specification because users begin to rely on it. I do like the fact that you both used hyperbole to succinctly illustrate the dangers of veering too far in either direction though :)
    - stickfigure 3y ago
      A (human language) specification is simply _enough_ information about a system that a human can figure out the intention of the author. The smarter and more context-rich the human, the simpler the specification can be. The dumber and less context-rich the human, the closer the specification needs to be to code. It's asymptotic. By the time you reach a human who is as dumb as an actual computer, the specification _is_ the code.
_glass 3y ago
I work in MedTech. We do this. A design has to be reviewed by QA, and is then tested, and the test is reviewed again. So just to counter the narrative, there are companies that do that, and it is working. In other jobs I also saw the cargo cult of QA. But in some industries it is just crucial, otherwise the pressure is too high to cut corners to implement something. It is a good mechanism to counter the need to move fast and break things.
sgerenser 3y ago
I work in software for CPU design/verification. Even here, where in theory there should be a rock-solid spec, there's not. There's a 12,000 page architectural specification, which is very helpful for specifying all the end-user visible state. But the microarchitectural specification is scattered all over different PDFs, visio docs, excel sheets, and sometimes the only spec is the RTL code itself.

legacynl 3y ago

> this seems fairly ridiculous on its face I think that's why people always tell each other to not take things at face value. Of course there is a big difference between sw and hw QA, in the thing that they test, and how they test them. But they are also very similar. Any QA department has to think about ways that things can go wrong, and what things to test for, how to test, which testing methods, which standards to handle, keeping certifications, etc. During testing you also need to keep reevaluating if you actually are catching each problem/bug and how to implement changes in your company that decreases the amount of problems or increase the amount that you catch. I think in that way there's a lot of overlap in thinking about business processes and how to identify problems with them. Of course once a specific binary gets tested and approved by QA it shouldn't matter if it gets copied or whatever as long as you make sure its the same binary (by a checksum for example). But still making sure that errors don't reach the customer, is vital in any QA. If errors does happen, QA is the department that can make sure that it doesn't happen again. And ofc be able to proof in court that you did your due diligence if something does happen.