3 ms·
After 15 years of writing unit tests, I haven't reached the same conclusion as the author. What I've found in my time is that unit testing can be good, but lik
by kstenerud 7y ago
After 15 years of writing unit tests, I haven't reached the same conclusion as the author.
What I've found in my time is that unit testing can be good, but like anything it's not a panacea. It requires discipline, and like normal code, it has code smells.
Black box unit tests are the most likely to be good tests, and white box unit tests are the most likely to be bad tests. The more you depend on the inner workings of a function in order to test it, the more likely it is that you are coupling your test to the implementation rather than the purpose of the unit being tested. Once you tie to the implementation, refactoring becomes a LOT harder, because changes will break the tests even if they don't break the functionality.
Mocks are also a major source of trouble, and more likely to be a code smell. If your tests are using a mock to test how many times your unit called it, either your tests are bad or your architecture is wrong.
There are three main kinds of code:
- Code that fetches data
- Code that stores data
- Code that transforms data
Mocks are necessary when you mix these. If you have a function that opens a DB connection, fetches data, transforms the data, and then stores the data, you now have an extra problem to deal with (the database), when all you wanted to do was test the transformation. Things would be far easier if you separated the transformation out, tested that in isolation, and then integrated that encapsulated functionality with fetching/storing code. This also improves separation of concerns and code duplication, since now your fetching/storing code can be generalized and also tested in isolation.
Actually I lie. There is a fourth kind of code: code that modifies state. This is the evilest, smelliest code around, and it's also something that unfortunately we can't get completely away from. But we can manage it, by isolating state, reducing the need for or scope of the state, and providing "configuration object" function entry points to make testing these monstrosities less nasty.
Code coverage is not just a measure of quality, but also of waste. If your code is not being called, then one of three things is happening:
1. It's error checking code for another API it's calling, which you normally shouldn't be writing tests for (unless that API is known to be buggy and you need to guard against it).
2. It's not contributing to the goals of the program, and can be taken out.
3. It does contribute to the goals of the program, in which case you need a test for it.
You can't reach 100% code coverage because of (1). But you absolutely should check WHICH code is covered in your tests because of (2) and (3). Anything higher than 80% coverage is pure luck, and tells you nothing about quality or wastage. In many cases, even 60-70% is sufficient.
- winrid 7y agoIf you don't use mocks then won't you just end up with integration tests?
- falsedan 7y agoWho cares what they’re called as long as they’re fast and don’t crash when running at scale/fill the disk or memory
- winrid 7y agoBecause if you're not using mocks the tests get unweildy. Suddenly changing components three layers down the dependency tree breaks "unit" tests at the top. Those are the tests you just end up @ignore'ing.
- msclrhd 7y agoMock things like database access, or better yet pass the database connection via an interface and create a test version of the database access so you can create a database-like thing for your tests (e.g. an in-memory database). If you are depending on another component of the application (a lexer, a JSON class, a maths function, etc.) don't mock that because if that class breaks you want tests to fail instead of silently passing because the broken class/function was mocked. If you are depending on thirdparty libraries, don't mock those unless you have to (i.e. if you cannot run the tests). This will help avoid unexpected bugs after upgrading libraries or if supporting different versions of a library. If you are writing code for a complex infrastructure (e.g. an plugin for an IDE), try to use test-specific versions of enough of the infrastructure to get the rest functioning, and use the real versions of as much as you can. This will help pick up issues in your code when new versions of that infrastructure make changes -- you want your tests to fail in this case, as the real code would fail. Understand why your tests are breaking and address that. Use @Ignore as a last resort. If APIs or behaviour has changed, update the code and tests to reflect that. If you are supporting different versions, create version-specific compatibility layers.