4 ms·

Early on in my career I was overly enthusiastic about writing unit tests for maximum coverage. I wouldn't say they were totally useless, but anecdotally I don't

Early on in my career I was overly enthusiastic about writing unit tests for maximum coverage. I wouldn't say they were totally useless, but anecdotally I don't think they caught many bugs/regressions and over time it was a lot of code to maintain. While it made refactoring feel 'safe' to do, it also slowed me down quite a bit. Now, my strategy is to instead focus on writing a few solid end-to-end/integration tests. These tests often find just as many bugs/regressions, are actually testing the entire system, and much easier to maintain. Most bugs happen due to bad interactions between systems. I still write unit tests for some tricky code, it just isn't my first choice.

john-tells-all 4y ago
After many years of experience, I agree. Tests are a business investment, of tech resources, to create business value. It's good to focus on the Testing Pyramid [1]. High level tests are slow and brittle, but connect the low-level code to business features. Unit tests are fast and detail oriented. In practice I write 1-2 high level tests (generally end to end, sometimes UI, or API/integration tests) to help focus development and have something that the business understands. Unit tests are helpful to "smooth the path forward" to ensure the code works as expected. Integration tests are great to iterate on, so that new code and tests actually work with real APIs correctly. Tests are not free. However they create a lot of value -- they create (business and tech) confidence that the system is working as expected. Like you mentioned they assist refactoring, which makes the code much cheaper and easier to work with. [1] https://martinfowler.com/articles/practical-test-pyramid.html https://martinfowler.com/articles/practical-test-pyramid.htm... (disclaimer: writing a book about tech feedback loops, e.g. tests)
nonethewiser 4y ago
Uncovered code tells you something useful. That there are no assertions made about some code. Covered code tells you nothing. It tells you this code may or may not have assertions made about it. Tracking coverage is good because it shows you what code isnt tested. But once its covered, you have no idea. So instead of increasing coverage, you should be evaluating the new assertions being made to uncovered code. But virtually everywhere I've been has just used coverage going up to mean your tests are sufficient.
roblh 4y ago
What do you use primarily for writing integration/end-to-end tests? Do you have any specific strategies for it? I’ve been using cypress but not having a ton of success with it, it just feels like everything is permanently broken, and generating/maintaining mock data is exhausting.
arealaccount 4y ago
Unit tests are more about writing good code and less about catching regressions. Code that you can easily write an automated test for generally is easier to maintain.
quietbritishjim 4y ago
> While it made refactoring feel 'safe' to do, it also slowed me down quite a bit. I find that unit tests actually make refactoring harder. The problem is, unit tests need to be rewritten, or at least substantially shuffled around, every time you refactor. If you're making a "big bang" change for a major new feature, and that requires refactoring, then it's possible to justify that cost. But that's not how it normally happens. Instead, a bunch of small changes over time usually make it apparent that the code would be a lot clearer and easier to maintain of the arrangement of responsibilities were different. It's already hard to ever justify the cost of that refactoring under any one small change. But add in the cost of updating all the unit tests and it becomes even more unlikely. Those updates also feels like demoralising busy work, whereas the actual change feels productive, so it also adds a human factor. Overall, the net effect is that code with a lot unit tests ends up with a stagnant and confusing design. As you say, a lot of the practical benefits can still be gained by end to end tests, while avoiding both the cost of writing so many tiny tests and the impact on long term design.

john-tells-all 4y ago

After many years of experience, I agree. Tests are a business investment, of tech resources, to create business value. It's good to focus on the Testing Pyramid [1]. High level tests are slow and brittle, but connect the low-level code to business features. Unit tests are fast and detail oriented. In practice I write 1-2 high level tests (generally end to end, sometimes UI, or API/integration tests) to help focus development and have something that the business understands. Unit tests are helpful to "smooth the path forward" to ensure the code works as expected. Integration tests are great to iterate on, so that new code and tests actually work with real APIs correctly. Tests are not free. However they create a lot of value -- they create (business and tech) confidence that the system is working as expected. Like you mentioned they assist refactoring, which makes the code much cheaper and easier to work with. [1] https://martinfowler.com/articles/practical-test-pyramid.html https://martinfowler.com/articles/practical-test-pyramid.htm... (disclaimer: writing a book about tech feedback loops, e.g. tests)

nonethewiser 4y ago

Uncovered code tells you something useful. That there are no assertions made about some code. Covered code tells you nothing. It tells you this code may or may not have assertions made about it. Tracking coverage is good because it shows you what code isnt tested. But once its covered, you have no idea. So instead of increasing coverage, you should be evaluating the new assertions being made to uncovered code. But virtually everywhere I've been has just used coverage going up to mean your tests are sufficient.

roblh 4y ago

What do you use primarily for writing integration/end-to-end tests? Do you have any specific strategies for it? I’ve been using cypress but not having a ton of success with it, it just feels like everything is permanently broken, and generating/maintaining mock data is exhausting.

arealaccount 4y ago

Unit tests are more about writing good code and less about catching regressions. Code that you can easily write an automated test for generally is easier to maintain.

quietbritishjim 4y ago

> While it made refactoring feel 'safe' to do, it also slowed me down quite a bit. I find that unit tests actually make refactoring harder. The problem is, unit tests need to be rewritten, or at least substantially shuffled around, every time you refactor. If you're making a "big bang" change for a major new feature, and that requires refactoring, then it's possible to justify that cost. But that's not how it normally happens. Instead, a bunch of small changes over time usually make it apparent that the code would be a lot clearer and easier to maintain of the arrangement of responsibilities were different. It's already hard to ever justify the cost of that refactoring under any one small change. But add in the cost of updating all the unit tests and it becomes even more unlikely. Those updates also feels like demoralising busy work, whereas the actual change feels productive, so it also adds a human factor. Overall, the net effect is that code with a lot unit tests ends up with a stagnant and confusing design. As you say, a lot of the practical benefits can still be gained by end to end tests, while avoiding both the cost of writing so many tiny tests and the impact on long term design.