2 ms·
Are there really that many teams who have just too darned many otherwise-perfect automated tests? I freely concede it's theoretically true, any non-trivial cost
by jerf 15y ago
Are there really that many teams who have just too darned many otherwise-perfect automated tests? I freely concede it's theoretically true, any non-trivial cost/benefit analysis with an infinity or zero for one or the other is always wrong, but I'd be more concerned about people using this sort of thing as an internal excuse to ignore tests than hopeful this will actually save someone.
- lucisferre 15y agoAs I read it it wasn't that they were simply too many but that they were not otherwise-perfect. The implication is that we probably have a lot of automated testing cruft but keep it because it is a passing test. This is something I've seen frequently. Even when I strive to keep the tests as long as I can I end up deleting them sooner or later because they are just too brittle and hard to work with. Tests themselves are a form of code coupling so they almost always come with a cost. I do agree with you (and said as much in my comment on G+) that there is some danger to this discussion becoming and excuse for laziness and ignorance, but that in itself isn't a good reason to not have the discussion.
- jerf 15y agoI added the "otherwise perfect" clause because of his third paragraph, which to my mind basically meant that for the rest of his post he wasn't talking about tests that could be fixed to run faster (with reasonable effort), but the remaining tests that are left even after you've polished the setup to a reasonable degree. Certainly I've broken tests with new code, examined broken tests, and simply nuked the broken tests before. Usually it's because they're either better covered by some other test written since then, or covering use cases I thought I'd have that never emerged. Sometimes I fix them to be less brittle over time. It is a good discussion in theory, I just question whether there's that many teams for which this really applies. I may be suffering from excessive reliance on my own personal experience. As I said yesterday in another comment, I tend to not to worry about whether something is "unit" or "integration" or "acceptance" testing, and most of testing is actually at least one abstraction level above what would be properly considered "unit testing". I personally don't have much difficulty with test stability as a result. People who write lower level unit tests may encounter this far more often.