4 ms·

Unit tests can be hard to maintain for several reasons. When you refactor your code (renaming methods, changing parameters, splitting methods up, etc), you will

Unit tests can be hard to maintain for several reasons. When you refactor your code (renaming methods, changing parameters, splitting methods up, etc), you will need to update those tests. Also some types of unit testing can be more brittle then others. For example if you use mocks to invocation counts, refactorings can be even more costly.

tynpeddler 8y ago
This comes up because there's a secret disagreement about the definition of the term "unit test". Some people see units tests as per method tests. If you have 20 methods in a class, all 20 methods get unit tested. Others see units as the smallest bits of irreducible business or tech knowledge. So if you have 20 methods in a class, you test the 5 public ones, or the methods that compose a useful "unit". This is the definition used by the original test driven development book. Many of the unit test advocates I've read use the second definition because testing private functionality incurs all the problems the article points out. Using the second definition leads to the testing pyramid, which is a thing of beauty. Unit tests have a few mocks to the unit's partners. The unit tests test for the correctness of the unit. The integration tests are one level up in terms of code coverage in a single test, and they ensure that the mocks used in the unit tests are correct. The e2e tests are then used to test that the app starts correctly and is configured correctly. The obvious happy and sad path scenarios are also tested to ensure that dependencies are functioning correctly (in a complex application there can be hundred of "obvious" e2e tests). In my career, every single instance I've seen about someone complaining about writing unit tests, they're either using the first definition I mentioned above, or their code is not composed correctly. I'm sure exceptions exist somewhere, but I've yet to see them. Using integration tests like the article describes leads to a host of problems. For starters, it's extremely difficult to get complete code coverage without huge or redundant test suites. These large test suites are harder to read and take longer to write because more complex testing requires more complex mocking. Regression is also more likely during large refactors since you haven't clearly enumerated expected behavior.
- miceeatnicerice 8y ago
  Fun fact: in the glossary at the end of the original xp book, it describes unit tests as 'tests from the perspective of the programmer', and that's that - a woolly cop-out.
- bni 8y ago
  Thats a great point about about the 2 definitions. Many devs dont seem to get encapsulation at all and definition 1 might be related to that. If you expose everything from a module I guess both your calling code an your unit test are unclear of where the boundaries are
joshuamorton 8y ago
Those are all code smells. Unit tests should only change if you also change public apis (I have one exception to this rule, which is very complex private methods which could easily be their own unit). Then if you're changing the tests, you're also refactoring everything else. >For example if you use mocks to invocation counts, refactorings can be even more costly. These should only be costly if you're over-testing. You shouldn't be mocking and asserting calls on every call within a system under test, you should be checking the ones that matter. Correctly written unit tests are a refactoring aid.

tynpeddler 8y ago

This comes up because there's a secret disagreement about the definition of the term "unit test". Some people see units tests as per method tests. If you have 20 methods in a class, all 20 methods get unit tested. Others see units as the smallest bits of irreducible business or tech knowledge. So if you have 20 methods in a class, you test the 5 public ones, or the methods that compose a useful "unit". This is the definition used by the original test driven development book. Many of the unit test advocates I've read use the second definition because testing private functionality incurs all the problems the article points out. Using the second definition leads to the testing pyramid, which is a thing of beauty. Unit tests have a few mocks to the unit's partners. The unit tests test for the correctness of the unit. The integration tests are one level up in terms of code coverage in a single test, and they ensure that the mocks used in the unit tests are correct. The e2e tests are then used to test that the app starts correctly and is configured correctly. The obvious happy and sad path scenarios are also tested to ensure that dependencies are functioning correctly (in a complex application there can be hundred of "obvious" e2e tests). In my career, every single instance I've seen about someone complaining about writing unit tests, they're either using the first definition I mentioned above, or their code is not composed correctly. I'm sure exceptions exist somewhere, but I've yet to see them. Using integration tests like the article describes leads to a host of problems. For starters, it's extremely difficult to get complete code coverage without huge or redundant test suites. These large test suites are harder to read and take longer to write because more complex testing requires more complex mocking. Regression is also more likely during large refactors since you haven't clearly enumerated expected behavior.

miceeatnicerice 8y ago
Fun fact: in the glossary at the end of the original xp book, it describes unit tests as 'tests from the perspective of the programmer', and that's that - a woolly cop-out.
bni 8y ago
Thats a great point about about the 2 definitions. Many devs dont seem to get encapsulation at all and definition 1 might be related to that. If you expose everything from a module I guess both your calling code an your unit test are unclear of where the boundaries are

joshuamorton 8y ago

Those are all code smells. Unit tests should only change if you also change public apis (I have one exception to this rule, which is very complex private methods which could easily be their own unit). Then if you're changing the tests, you're also refactoring everything else. >For example if you use mocks to invocation counts, refactorings can be even more costly. These should only be costly if you're over-testing. You shouldn't be mocking and asserting calls on every call within a system under test, you should be checking the ones that matter. Correctly written unit tests are a refactoring aid.