4 ms·
Of course context matters but _generally_, end to end tests should be mostly avoided: - the execution time of end to end tests are orders of magnitude larger t
by eckesicle 3y ago
Of course context matters but _generally_, end to end tests should be mostly avoided:
- the execution time of end to end tests are orders of magnitude larger than unit tests. In a medium ish application your end to end tests will take several seconds to spin up, for example by spinning up fresh databases (even with rollback in gets slow) and other detachable services. A unit test is usually a malloc away and thousands can be run in parallel on your local box in a second. End to end tests easily take hours of compute time time to run in the real world. Good luck running the suite on your laptop. Nothing kills productivity quicker than a test suite that runs slowly
- unit tests are also independent of other code so it’s easy to get some confidence that your code does the right thing by only testing the touched files in your diff before running the entire suite on CI.
- unit tests encourage good code quality since it’s easier to test injectable code. E2E instead incentivises copy pasted test suites that are thousands of lines of code long.
- in the general case end to end tests are asynchronous and require orders of magnitude more effort to run. Asynchronous tests are flaky. Flaky builds slow down your velocity.
- E2E tests rot really fast. One change leads to thousands of broken E2E tests.
- E2E is not a replacement for monitoring your application. The class of bugs you will catch in your E2E suite is a strict subset of the class of bugs you will catch with monitoring and metrics. Build the latter instead and roll out your code incrementally to your users.
- E2E tests are complicated to set up and maintain in ops. Unit tests not so much. Somewhat unexpectedly your development velocity scales pretty much linearly with stack complexity.
In summary (and again context matters and there aren’t any absolutes, but …):
- Don’t do automated E2E tests.
- Do unit tests
- Do metrics and monitoring
You have a limited budget, spend it where you get a good bang for your buck.