3 ms·

Tests should mostly be integration, not unit. Unit tests tie you down to the specific implementation, which means changing the implementation becomes much more

Tests should mostly be integration, not unit. Unit tests tie you down to the specific implementation, which means changing the implementation becomes much more onerous. Integration tests have much more value, give you higher confidence that the code does what it claims, and is easier to code review. https://kentcdodds.com/blog/write-tests https://kentcdodds.com/blog/write-tests

makeitdouble 3y ago
I had that mindset for a while, just to fall into enough issues that I had to balance it again. Integration tests are important for features, unit tests stay important for code blocks. My realization was that a ton of what we wrote aren't features, but blocks. For instance, let's imagine an API that build an invoice. Where do you put the VAT calculation test ? Do you jam your 100+ VAT test cases in the integrations tests, or do you unit test the VAT calculation in isolation and only cover the main cases in the integration ones ? Same for name display, invoice number generation, Invoice items fetching etc. If you want a decent coverage and also document/properly test the edge cases, unit test will easily be half or more of your whole test volume. And of course you'll want to reuse there blocks, so make sure they're rock solid, so pay even more attention to tests.
imiric 3y ago
Eh, I disagree. I find the traditional test pyramid a suitable model still. Sure, unit tests require much more maintenance than higher level tests, but they give you confidence that the smallest parts of the codebase work as intended. They're the ones that test all sorts of failure scenarios and edge cases, which is typically not the purpose of integration tests. They also should be inexpensive to run, simple to write, and require minimal setup. Unit tests should also give you immediate feedback that something went wrong, by pinpointing the exact component that failed. In contrast, integration tests might happily pass, as they're not granular enough to cover all code paths. Integration tests focus on, well, that the integration between components is working as expected. So I still insist on having mostly unit tests, many integration tests, and some E2E tests. Doing it otherwise because it saves you maintenance efforts will haunt you in the long run.
- bluGill 3y ago
  Most people don't have a rather obvious interface on their units to tests. If you are implementing one of the CS3025 (your university probably used a different number scheme) algorithms, then the interface is obvious and worth a unit tests. Even if there is some doubt (rare), you will have so many users that is isn't possible to change anymore anyway. Most of us are writing glue code that isn't as clear what the api should be. Unit tests just get in the way of refactoring when requirements change.
  - imiric 3y ago
    On the one hand, I agree that writing unit tests before the design has settled and the interfaces are stable can be frustrating. Many people find TDD a chore because of this. But if you delay adding unit tests until after the interfaces are more or less stable, it gives you higher confidence to refactor the implementation, which is invaluable. On the other hand, the benefit of TDD is that it helps with the actual API design. You immediately experience the API as a user, and this process drives the implementation, and ensures you end up with a friendly and testable API. If you add tests after the design, then you might have a more difficult time refactoring the implementation to make it testable, and the DX could suffer. So I see both sides of the argument, but in either case, unit tests are very important, and I wouldn't avoid them just because they're a chore to maintain.
    - bluGill 3y ago
      TDD gives you a testable interface, but otherwise doesn't help think what the api should be. You can so TDD using only integration tests. This means tests are not added after the design. It only means that your individual functions don't have tests. I do this allowed the time, it makes refactoring easier. The most common objection: how do you know what failed? Turns out to be a non issues, since I make my tests fast I can run them often and thus the failure has an obvious cause: the last thing I changed.
- MoreQARespect 3y ago
  Unit tests help in a very specific circumstance - when you're testing complex calculations or logical code with simple inputs or outputs. Some projects have almost zero code like this. In all other situations they suck. When they fail it doesnt necessarily mean anything is wrong and vice versa. It just mirrors a portion of the code. It's a lot of expensive maintenance with no payoff. In contrast integration tests are usually much more meaningful - they will typically test a real system or user interaction.
teddyh 3y ago
> Tests should mostly be integration, not unit. Other people disagree: <https://blog.thecodewhisperer.com/series#beware-the-integrated-tests-scam-was-integrated-tests-are-a-scam https://blog.thecodewhisperer.com/series#beware-the-integrat...> In short (IIUC), integration tests do not give you any feedback on when your design needs to change, while unit tests do.
Silhouette 3y ago
Unit tests tie you down to the specific implementation, which means changing the implementation becomes much more onerous. If you test unit interfaces and not implementation details then that problem goes away.
- recursive 3y ago
  Unit tests tie you down to a specific problem decomposition.
  - Silhouette 3y ago
    Units give you a specific problem decomposition. Unit tests just help to make sure that the units you have right now are doing their jobs. Obviously if you change what units you have or how responsibilities are allocated among them then you will need corresponding changes in the unit tests.
    - bluGill 3y ago
      Which is good only if you have the comprehensive integration tests as well.
      - Silhouette 3y ago
        I don't see why. There are some type of code where I think unit testing is very beneficial and other types of code where I'd probably prefer a different style of testing. But for code where unit testing helps I find it useful independent of any of other types of testing that may or may not be happening as well.
dav 3y ago
You should really do some research on this. Good luck on your journey.

makeitdouble 3y ago

I had that mindset for a while, just to fall into enough issues that I had to balance it again. Integration tests are important for features, unit tests stay important for code blocks. My realization was that a ton of what we wrote aren't features, but blocks. For instance, let's imagine an API that build an invoice. Where do you put the VAT calculation test ? Do you jam your 100+ VAT test cases in the integrations tests, or do you unit test the VAT calculation in isolation and only cover the main cases in the integration ones ? Same for name display, invoice number generation, Invoice items fetching etc. If you want a decent coverage and also document/properly test the edge cases, unit test will easily be half or more of your whole test volume. And of course you'll want to reuse there blocks, so make sure they're rock solid, so pay even more attention to tests.

imiric 3y ago

Eh, I disagree. I find the traditional test pyramid a suitable model still. Sure, unit tests require much more maintenance than higher level tests, but they give you confidence that the smallest parts of the codebase work as intended. They're the ones that test all sorts of failure scenarios and edge cases, which is typically not the purpose of integration tests. They also should be inexpensive to run, simple to write, and require minimal setup. Unit tests should also give you immediate feedback that something went wrong, by pinpointing the exact component that failed. In contrast, integration tests might happily pass, as they're not granular enough to cover all code paths. Integration tests focus on, well, that the integration between components is working as expected. So I still insist on having mostly unit tests, many integration tests, and some E2E tests. Doing it otherwise because it saves you maintenance efforts will haunt you in the long run.

bluGill 3y ago
Most people don't have a rather obvious interface on their units to tests. If you are implementing one of the CS3025 (your university probably used a different number scheme) algorithms, then the interface is obvious and worth a unit tests. Even if there is some doubt (rare), you will have so many users that is isn't possible to change anymore anyway. Most of us are writing glue code that isn't as clear what the api should be. Unit tests just get in the way of refactoring when requirements change.
- imiric 3y ago
  On the one hand, I agree that writing unit tests before the design has settled and the interfaces are stable can be frustrating. Many people find TDD a chore because of this. But if you delay adding unit tests until after the interfaces are more or less stable, it gives you higher confidence to refactor the implementation, which is invaluable. On the other hand, the benefit of TDD is that it helps with the actual API design. You immediately experience the API as a user, and this process drives the implementation, and ensures you end up with a friendly and testable API. If you add tests after the design, then you might have a more difficult time refactoring the implementation to make it testable, and the DX could suffer. So I see both sides of the argument, but in either case, unit tests are very important, and I wouldn't avoid them just because they're a chore to maintain.
  - bluGill 3y ago
    TDD gives you a testable interface, but otherwise doesn't help think what the api should be. You can so TDD using only integration tests. This means tests are not added after the design. It only means that your individual functions don't have tests. I do this allowed the time, it makes refactoring easier. The most common objection: how do you know what failed? Turns out to be a non issues, since I make my tests fast I can run them often and thus the failure has an obvious cause: the last thing I changed.
MoreQARespect 3y ago
Unit tests help in a very specific circumstance - when you're testing complex calculations or logical code with simple inputs or outputs. Some projects have almost zero code like this. In all other situations they suck. When they fail it doesnt necessarily mean anything is wrong and vice versa. It just mirrors a portion of the code. It's a lot of expensive maintenance with no payoff. In contrast integration tests are usually much more meaningful - they will typically test a real system or user interaction.

teddyh 3y ago

> Tests should mostly be integration, not unit. Other people disagree: <https://blog.thecodewhisperer.com/series#beware-the-integrated-tests-scam-was-integrated-tests-are-a-scam https://blog.thecodewhisperer.com/series#beware-the-integrat...> In short (IIUC), integration tests do not give you any feedback on when your design needs to change, while unit tests do.

Silhouette 3y ago

Unit tests tie you down to the specific implementation, which means changing the implementation becomes much more onerous. If you test unit interfaces and not implementation details then that problem goes away.

recursive 3y ago
Unit tests tie you down to a specific problem decomposition.
- Silhouette 3y ago
  Units give you a specific problem decomposition. Unit tests just help to make sure that the units you have right now are doing their jobs. Obviously if you change what units you have or how responsibilities are allocated among them then you will need corresponding changes in the unit tests.
  - bluGill 3y ago
    Which is good only if you have the comprehensive integration tests as well.
    - Silhouette 3y ago
      I don't see why. There are some type of code where I think unit testing is very beneficial and other types of code where I'd probably prefer a different style of testing. But for code where unit testing helps I find it useful independent of any of other types of testing that may or may not be happening as well.

dav 3y ago

You should really do some research on this. Good luck on your journey.