3 ms·
I think the problem lies with the definition of "unit" which you quote above. As with designs stacking on top of one another at various levels, one level's fun
by notacoward 10y ago
I think the problem lies with the definition of "unit" which you quote above. As with designs stacking on top of one another at various levels, one level's functional test is often the next level's unit test. Your app's unit test might mock the very same things that are covered by the functional test for an underlying library. That library's unit tests might mock the very same things that are covered by the functional tests for one of its own dependencies, including the OS. And so on, even down to the level of functional units within a chip. (Chip verifiers have a lot to teach us software types about this kind of thing BTW, and I know because I've worked with a few.)
The distinction I always try to make is whether you're testing a contract or an implementation. If you're testing a contract, ROI is likely to remain high even as you move to finder granularities. If you're testing an implementation, so that the tests fail even when the implementation is 100% correct, then ROI falls off a cliff.
That's exactly what happens when the "unit" gets too small. If certain private methods can't be called a certain way because of constraints imposed by the class's public methods, then testing those calls is testing things that can't happen. That takes time away from testing (and implementing fixes for) things that can happen. If the class's contract changes so that those internal methods can be called in new ways, then yes, a 100% code-coverage unit test might catch the error. So, in all likelihood, would the new functional test accompanying the public-method change. The delta between the two, times the likelihood of such a scenario occurring in the first place, is too small to justify the cost of writing those tests and the likelihood that they'll generate false negatives.