4 ms·

"This is because an end-to-end test covers a much greater area of the code base." I disagree. A single unit test will cover less area than an end-to-end test,

"This is because an end-to-end test covers a much greater area of the code base." I disagree. A single unit test will cover less area than an end-to-end test, but unit tests in aggregate will cover a greater area.

marcosdumay 8y ago
> but unit tests in aggregate will cover a greater area. If integration tests can't cover some part of your code, it's because that part does not matter. Get rid of it.
- billsmithaustin 8y ago
  I agree that some scenarios are more important than others, but that seems to be orthogonal to the claim that I was objecting to. I also disagree that whether an integration test can cover some part of your code is the litmus test for whether the code is relevant. There are types of errors that are relevant but complicated to simulate in an integration test, or at least in an automated integration test. An example would be a network partition. It's involved to code a test to cause the partition, wait for the various timeouts to take place, then heal the partition, then wait some more for things to start talking and recovering. And depending on who owns the test environment and how your organization is managed, you may need to negotiate with a DevOps / Operations / Security person to get permission to run iptables in the appropriate machines. On the other hand, unit-testing what happens when a connection fails is probably pretty easy.
  - marcosdumay 8y ago
    > It's involved to code a test to cause the partition, wait for the various timeouts to take place, then heal the partition, then wait some more for things to start talking and recovering. Yet, on the layer that is charged with surviving a partition, that's exactly what you should be doing. Your unit test can verify that some machine sends the designed message down the wire, it can check if it responds to the expected message, but it can not even check if the software can detect a partition, even less that it can survive one. I would recommend you make those timeouts configurable from the user point of view.
- Jtsummers 8y ago
  It isn't always feasible to introduce certain error conditions with integration tests (depending on how much of your system you're integrating, whether faults can be injected at that level or not). You often need unit testing to ensure that your code handles those failure modes correctly. That code can be critical to a system, but hard to exercise directly.

marcosdumay 8y ago

> but unit tests in aggregate will cover a greater area. If integration tests can't cover some part of your code, it's because that part does not matter. Get rid of it.

billsmithaustin 8y ago
I agree that some scenarios are more important than others, but that seems to be orthogonal to the claim that I was objecting to. I also disagree that whether an integration test can cover some part of your code is the litmus test for whether the code is relevant. There are types of errors that are relevant but complicated to simulate in an integration test, or at least in an automated integration test. An example would be a network partition. It's involved to code a test to cause the partition, wait for the various timeouts to take place, then heal the partition, then wait some more for things to start talking and recovering. And depending on who owns the test environment and how your organization is managed, you may need to negotiate with a DevOps / Operations / Security person to get permission to run iptables in the appropriate machines. On the other hand, unit-testing what happens when a connection fails is probably pretty easy.
- marcosdumay 8y ago
  > It's involved to code a test to cause the partition, wait for the various timeouts to take place, then heal the partition, then wait some more for things to start talking and recovering. Yet, on the layer that is charged with surviving a partition, that's exactly what you should be doing. Your unit test can verify that some machine sends the designed message down the wire, it can check if it responds to the expected message, but it can not even check if the software can detect a partition, even less that it can survive one. I would recommend you make those timeouts configurable from the user point of view.
Jtsummers 8y ago
It isn't always feasible to introduce certain error conditions with integration tests (depending on how much of your system you're integrating, whether faults can be injected at that level or not). You often need unit testing to ensure that your code handles those failure modes correctly. That code can be critical to a system, but hard to exercise directly.