3 ms·

We do something similar. Any bug that is submitted by a customer (rather than QA, or a trial team, or other developers) is analyzed. 50% of our bugs are due to

We do something similar. Any bug that is submitted by a customer (rather than QA, or a trial team, or other developers) is analyzed. 50% of our bugs are due to poor requirements. 25% are due to poor design. The other 25% are just coding problems. Most of the requirements problems derive from lack of understanding about the tradeoffs in the current system or lack of understanding what the customer actually wants. Most of the design problems are a result of excessive complexity. People often forget to simplify and refactor prior to adding a new feature. They end up having to keep a larger system in their brain when designing than they otherwise need to. Therefore they miss interactions that exist or forget to handle them in the new design. Most of the coding problems would have been caught had unit tests been clearer and closer to 100% branch/code coverage. It is amazing how many bugs lurk in the last 5% of testing. It is actually quite fun to tell a product manager that they are the cause for a bug :) Usually it is the developer that

chriswarbo 10y ago
> Most of the coding problems would have been caught had unit tests been clearer and closer to 100% branch/code coverage. It is amazing how many bugs lurk in the last 5% of testing. Presumably, coverage starts with the low-hanging fruit (simple, linear code; heavily-exercised code like startup routines and main loops; etc.) and grows to encompass ever more unweildy, smelly, edge-case parts of the codebase. Hence, by increasing code coverage, we're boiling off the solvent of run-of-the-mill code, such that what remains uncovered gets an ever stronger concentration of hackery, unintelligibility and black magic. By 95% coverage, the bits remaining are those parts people don't want to touch, so I'd expect the concentration of bugs to be highest there. :)
- hankerfaster 10y ago
  precisely! As programmers that is what we care about. But really, at that point the business often doesn't care. Not when a requirement bug could cause months of lost engineer time. Our experience is requirement bugs are much more costly than a missed unit test resulting in a bug.

chriswarbo 10y ago

> Most of the coding problems would have been caught had unit tests been clearer and closer to 100% branch/code coverage. It is amazing how many bugs lurk in the last 5% of testing. Presumably, coverage starts with the low-hanging fruit (simple, linear code; heavily-exercised code like startup routines and main loops; etc.) and grows to encompass ever more unweildy, smelly, edge-case parts of the codebase. Hence, by increasing code coverage, we're boiling off the solvent of run-of-the-mill code, such that what remains uncovered gets an ever stronger concentration of hackery, unintelligibility and black magic. By 95% coverage, the bits remaining are those parts people don't want to touch, so I'd expect the concentration of bugs to be highest there. :)

hankerfaster 10y ago
precisely! As programmers that is what we care about. But really, at that point the business often doesn't care. Not when a requirement bug could cause months of lost engineer time. Our experience is requirement bugs are much more costly than a missed unit test resulting in a bug.