3 ms·
> Integration tests lie somewhere between unit tests and end-to-end tests so they provide the best balance. Therefore, they have the highest ROI. Being in betw
by IanCal 8y ago
> Integration tests lie somewhere between unit tests and end-to-end tests so they provide the best balance. Therefore, they have the highest ROI.
Being in between does not mean it gives the highest ROI or the best balance. They can easily share the worst of both.
> Lean Testing takes an economic point of view to reconsider the Return on Investment of unit tests.
But does not in any way actually try and work this out, which means that you can't make the conclusions that are in this.
> The Return on investment (ROI) of an end-to-end test is higher than that of a unit test. This is because an end-to-end test covers a much greater area of the code base. Even taking into account higher costs, it provides disproportionally more confidence.
And yet no figures or evidence.
Many e2e tests can cover the same path segments, meaning adding a new test may not increase your confidence by much at all. But they're still taking longer to run, and despite the articles insistence that integration and unit tests are brittle, I've found E2E tests are also brittle, just in different ways.
I shouldn't have to change my string normalisation tests just because the website changed moving the resulting output field to a new place on the page. They shouldn't break because the login flow changes.
I have definitely seen (and written) E2E tests that provide almost no real value, and unit tests that provide huge amounts.
> Plus, end-to-end tests test the critical paths that your users actually take. Whereas unit tests may test corner cases that are never or very seldomly encountered in practice
Unfortunately low proportions of errors still can account for extremely large actual numbers of problems. 1/1000 bugs will happen all the time if you've got a reasonable number of users. Those bugs also may happen to most of your users if they're somewhat random, or worse may heavily impact one group.
> For many products, it is acceptable to have the common cases work but not the exotic ones (‘Unit Test Fetish’). If you miss a corner case bug due to low code coverage that affects 0.1% of your users you might survive. If your time to market increases due to high code coverage demands you might not survive.
Or if roughly one in 1000 customers keep taking your production server down due to a bug you might not, and you may have been just fine waiting another week to go live.
I agree with the idea, you should consider what it is you're trying to achieve and what the best ways of doing that are. What you absolutely shouldn't do is use terms like ROI and economic without any analysis, purely to justify not doing something you don't really want to do.