3 ms·

Unit tests don't just fail to find bugs because they don't give complete coverage (all testing has that problem, and the author is mistaken to say that end-to-e

Unit tests don't just fail to find bugs because they don't give complete coverage (all testing has that problem, and the author is mistaken to say that end-to-end testing shows all bugs), they fail to find bugs resulting from faulty assumptions about how the units will interact. Boeing's faulty Starliner test demonstrated how that goes [1]. The true value in unit testing is in revealing the bugs it will find early in the process. One benefit of this is there will likely be fewer to be found in the end-to-end testing (or on deployment.) There is no value in debating which is better. The important issue is how to allocate your testing budget effectively, because there is never even close to enough time to exhaustively test. [1] https://www.engadget.com/2020-02-29-boeing-starliner-failed-first-flight-report.html https://www.engadget.com/2020-02-29-boeing-starliner-failed-...

ozim 6y ago
Pyramid approach, inverted pyramid, crab shaped does not make sense because it tries to be catch all guideline. Same with any 'maturity models' for software projects. People are forgetting there is something like 'risk based' approach to testing. This is the only way to allocate budget effectively, where stuff that can drop half of your database should be tested well and stuff that will make your UI look funny probably less.
- Jtsummers 6y ago
  I hate to say this, but I do think CMMI's model is a good idea at its core. The model itself is actually pretty solid. That is, everything it describes is something a good organization should do, and the requirement to document the processes + train people is a sound one. But the appraisals and how most businesses treat it are, well, moronic. The appraisals are easily gamed so that if you do everything on paper you're pretty much going to get your target level (3 is good enough for 99% of contracts if matters at all). And businesses try to treat the model as a process, when it's not. The former means that the appraisal results are useless. You cannot judge anyone by being level 3 or 5. That often just means they played the game better, not that they're actually better. The latter, though, is the worst part. CMMI is broken into a number of process areas (I'm happy to report I no longer recall the number of areas). It describes them in psuedo-process or procedural terms. That is, you could take what they describe and make a process, but it would be a very linear and simple process. What's intended is for you to say how your process maps to their model. But organizations don't do that, they do moronic things instead like scrap the above and write a whole new document that's 40 pages of nothing but plagiarizing the CMMI book saying how they do things, but it's almost entirely a lie because no one (except the sacrificial team sent to be appraised) actually does it that way.

ozim 6y ago

Pyramid approach, inverted pyramid, crab shaped does not make sense because it tries to be catch all guideline. Same with any 'maturity models' for software projects. People are forgetting there is something like 'risk based' approach to testing. This is the only way to allocate budget effectively, where stuff that can drop half of your database should be tested well and stuff that will make your UI look funny probably less.

Jtsummers 6y ago
I hate to say this, but I do think CMMI's model is a good idea at its core. The model itself is actually pretty solid. That is, everything it describes is something a good organization should do, and the requirement to document the processes + train people is a sound one. But the appraisals and how most businesses treat it are, well, moronic. The appraisals are easily gamed so that if you do everything on paper you're pretty much going to get your target level (3 is good enough for 99% of contracts if matters at all). And businesses try to treat the model as a process, when it's not. The former means that the appraisal results are useless. You cannot judge anyone by being level 3 or 5. That often just means they played the game better, not that they're actually better. The latter, though, is the worst part. CMMI is broken into a number of process areas (I'm happy to report I no longer recall the number of areas). It describes them in psuedo-process or procedural terms. That is, you could take what they describe and make a process, but it would be a very linear and simple process. What's intended is for you to say how your process maps to their model. But organizations don't do that, they do moronic things instead like scrap the above and write a whole new document that's 40 pages of nothing but plagiarizing the CMMI book saying how they do things, but it's almost entirely a lie because no one (except the sacrificial team sent to be appraised) actually does it that way.