2 ms·
I know that statement is true, but WHY is it true? I have never seen anyone look at a coverage report and take any useful action. We made code coverage a part
by stcredzero 3y ago
I know that statement is true, but WHY is it true? I have never seen anyone look at a coverage report and take any useful action.
We made code coverage a part of CI, and we've been using that to drive up code coverage. It's been working. Our tests have gone from ~25% to ~50% coverage. Breakages making it out to staging have gone down.
Also, some incentives are well aligned with this. If people see that things aren't easy to test, then they get rearranged to make them easy to test. Not all incentives are aligned, but not everything is perfect. This is why reviews are useful.
When I've looked at the reports I keep finding trivial code that would be hard to test for little gain.
If your trivial code is also hard to test, then this indicates architectural code debt. Why isn't the trivial code trivial to test?
- bluGill 3y agoHow do you test the code that calls a OS function? (think of something like shutdown - you don't want your test to turn off your CI system) The obvious answer is write a wrapper and a mock implementation of the wrapper. Now you have the wrapper that is untested, so you write a wrapper for that so you - I actually had someone go down this 3 levels before giving up. I work in embedded where we have to control real hardware.
- nickpsecurity 3y agoThe other thing you can do is composition of individually-verified parts. In your example, you make tests that really exercise the shutdown function. Also, note any preconditions these functions need to meet so the application calling it can be consistent with specs. From there, just assume shutdown works if used within the tested spec. This method also encourages reusing working specs, code, and tests. We see that in DO-178C with the RTOS’s, etc.