4 ms·

Also, this kind of mentality frequently results in what I consider negative-value tests. A test suite is supposed to do two things: a) identify bugs by faili

Also, this kind of mentality frequently results in what I consider negative-value tests. A test suite is supposed to do two things: a) identify bugs by failing on incorrect behavior, and b) enable refactoring by continuing to pass when underlying code doesn't change behavior Focusing on 100% unit test coverage all the time—particularly in code that isn't doing simple, single responsibility stuff—often results (in my experience) in tests that overly mock other parts of the system. The net result is that you end up writing tests that simple check that "the code was written the way it is currently written". Tests like this fail on the first count because they fundamentally cannot detect incorrect behavior, and they fail on the second count because a non-trivial refactor inevitably changes the implementation even if the resulting behavior stays the same. Not that mocks aren't useful in some scenarios, but if you find yourself writing entire test suites based on them you're likely creating tests that are negative value. They won't catch undiscovered bugs, and they'll give you bad information when you're refactoring.

lichtenberger 5y ago
That's precisely what I did in the beginning of my career. To test every method and mock all the dependencies. This had the impact, because we usually don't use a dependency injection framework, that I had to add an interface for almost every class, adding factories with interfaces almost everywhere and then write tests which expect certain method calls and to basically debug the tests while writing them until all calls have been mocked. Thus, whenever I changed the production code I had to change the test as well. IMHO in my own projects I tend to write tests which test certain parts of a system as well as some dependencies. So usually I'd say they are integration tests instead of unit tests. But I can refactor stuff and once I break something it's usually detected by one of the tests. Maybe unit tests should test for certain cases border cases, which are hard to test with integration tests.
seanhunter 5y ago
I heartily agree with your general sentiment, but think you're not mentioning a very important function of a test suite which is: c) explicitly and accurately document expected behavior including in a variety of edge cases ...this is sort of implicit in your first 2 but I think worth breaking out. Going for high test coverage (in places where I've done TDD-style dev) has really forced me to think through those edge cases and what I want my code to do in all of them. I think negative-value testing comes from the mindset of "how do I write a test which exercises this branch?" but a positive version of that is "I put this branch to handle the case where foo is getting close to the boundary. Let's put in test cases with foo at and close to the boundary and make sure it does what it is supposed to." If the test focuses on behavior then you find the bugs and enable refactoring and you get high coverage because if you can't exercise all the paths probably some of them are dead and can just get deleted.

lichtenberger 5y ago

That's precisely what I did in the beginning of my career. To test every method and mock all the dependencies. This had the impact, because we usually don't use a dependency injection framework, that I had to add an interface for almost every class, adding factories with interfaces almost everywhere and then write tests which expect certain method calls and to basically debug the tests while writing them until all calls have been mocked. Thus, whenever I changed the production code I had to change the test as well. IMHO in my own projects I tend to write tests which test certain parts of a system as well as some dependencies. So usually I'd say they are integration tests instead of unit tests. But I can refactor stuff and once I break something it's usually detected by one of the tests. Maybe unit tests should test for certain cases border cases, which are hard to test with integration tests.

seanhunter 5y ago

I heartily agree with your general sentiment, but think you're not mentioning a very important function of a test suite which is: c) explicitly and accurately document expected behavior including in a variety of edge cases ...this is sort of implicit in your first 2 but I think worth breaking out. Going for high test coverage (in places where I've done TDD-style dev) has really forced me to think through those edge cases and what I want my code to do in all of them. I think negative-value testing comes from the mindset of "how do I write a test which exercises this branch?" but a positive version of that is "I put this branch to handle the case where foo is getting close to the boundary. Let's put in test cases with foo at and close to the boundary and make sure it does what it is supposed to." If the test focuses on behavior then you find the bugs and enable refactoring and you get high coverage because if you can't exercise all the paths probably some of them are dead and can just get deleted.