3 ms·
> Often E2E tests have such sunk costs involved that they materially affect the project roadmap. I've seen this first hand. When the e2e tests take hours to ru
by shakezula 5y ago
> Often E2E tests have such sunk costs involved that they materially affect the project roadmap.
I've seen this first hand. When the e2e tests take hours to run, are flaky on a good day, and are only really understood by one or two people on the whole team, they can be a major roadblock to new features or even just moderate refactors.
- hinkley 5y agoThat definitely happens. The E2E tests tend to make assumptions about how the app works (encode not just the requirements but also the architecture) and some features change the design. In order to add this feature we have to fix dozens of other tests. I’ve seen people on multiple projects team up to fix these, take over a day working together, and still not be done. They always try to tweak the tests but the test assumptions fight them. Meanwhile if we add a feature that invalidates a unit test, you just delete the unit test and start over. Unit tests are cattle, E2E tests are pets.
- hinkley 5y agoI should add as well: after that day, day and a half working together on old tests, those engineers look beaten down. They are not having a good time. It’s miserable work. It must be some sort of Stockholm syndrome that people in this state still defend the tests. Even after they’ve invested more time and energy into fixing them than we ever would just manually testing that part of the code in perpetuity.