3 ms·
To play devil's advocate here, what usually suffers the most in integration-only testing is testing error handling. In some codebases it's extremely important t
by apaprocki 4y ago
To play devil's advocate here, what usually suffers the most in integration-only testing is testing error handling. In some codebases it's extremely important to test all the error handling paths because they are not extremely exceptional events. Many times, the disproportionate amount of failures are from error handling not precisely doing the right thing even though it looks reasonable upon code review. It might not mean 100% coverage, but in these situations, ensuring comprehensive tests of all the error conditions (inducing OOMs and other resource limits) helps make sure the failure paths work just as well as the normal ones. Failure of a failure path can manifest at best in ways such as hiding the true source of an issue, telemetry problems, logging statements without salient values; at worst, crashing because of a chain of events triggered by the seemingly innocuous error handling that passed a code review. OOM modeling to test every code path that allocates in a function is the most tedious thing in a gtest, but often yields the most surprising, actionable fixes. Sadly, most programmers just ignore OOMs and hand-waive "if allocation fails, the system is broken and my software doesn't need to work correctly".
- bvrmn 4y agoThis. So many "do not cover" hints for exception handler blocks. So many lost root causes when handler tries to format a error.