4 ms·

As I read it it wasn't that they were simply too many but that they were not otherwise-perfect. The implication is that we probably have a lot of automated test

As I read it it wasn't that they were simply too many but that they were not otherwise-perfect. The implication is that we probably have a lot of automated testing cruft but keep it because it is a passing test. This is something I've seen frequently. Even when I strive to keep the tests as long as I can I end up deleting them sooner or later because they are just too brittle and hard to work with. Tests themselves are a form of code coupling so they almost always come with a cost. I do agree with you (and said as much in my comment on G+) that there is some danger to this discussion becoming and excuse for laziness and ignorance, but that in itself isn't a good reason to not have the discussion.

jerf 15y ago
I added the "otherwise perfect" clause because of his third paragraph, which to my mind basically meant that for the rest of his post he wasn't talking about tests that could be fixed to run faster (with reasonable effort), but the remaining tests that are left even after you've polished the setup to a reasonable degree. Certainly I've broken tests with new code, examined broken tests, and simply nuked the broken tests before. Usually it's because they're either better covered by some other test written since then, or covering use cases I thought I'd have that never emerged. Sometimes I fix them to be less brittle over time. It is a good discussion in theory, I just question whether there's that many teams for which this really applies. I may be suffering from excessive reliance on my own personal experience. As I said yesterday in another comment, I tend to not to worry about whether something is "unit" or "integration" or "acceptance" testing, and most of testing is actually at least one abstraction level above what would be properly considered "unit testing". I personally don't have much difficulty with test stability as a result. People who write lower level unit tests may encounter this far more often.

jerf 15y ago

I added the "otherwise perfect" clause because of his third paragraph, which to my mind basically meant that for the rest of his post he wasn't talking about tests that could be fixed to run faster (with reasonable effort), but the remaining tests that are left even after you've polished the setup to a reasonable degree. Certainly I've broken tests with new code, examined broken tests, and simply nuked the broken tests before. Usually it's because they're either better covered by some other test written since then, or covering use cases I thought I'd have that never emerged. Sometimes I fix them to be less brittle over time. It is a good discussion in theory, I just question whether there's that many teams for which this really applies. I may be suffering from excessive reliance on my own personal experience. As I said yesterday in another comment, I tend to not to worry about whether something is "unit" or "integration" or "acceptance" testing, and most of testing is actually at least one abstraction level above what would be properly considered "unit testing". I personally don't have much difficulty with test stability as a result. People who write lower level unit tests may encounter this far more often.