I work a lot on compilers and VMs and have written tens of thousands of tests at different scales over the years. Different kinds of tests serve different purpo

I work a lot on compilers and VMs and have written tens of thousands of tests at different scales over the years. Different kinds of tests serve different purposes. Unit tests help you pinpoint errors in the code. They can exhaustively test (only) small components to make sure they are fully compliant. They are a refactoring and development aid to the extent that they are focused (don't involve too many components), quick (run in seconds or less), not too tightly coupled to the code under test (i.e. can change code under test without changing tests), and explanatory (failure output is easy to understand and points exactly at the faulting component). Making good unit tests is an art form. Some people love their mocking frameworks. Personally, I hate them. Mocks are confusing and they try to check behavior rather than input/output. They make refactoring hard because they test behavior rather than results, and they are usually confusing. Integration tests are about putting one or more systems together to test their interactions. More than just a single unit, we can put services together and test their interface. They can be more exhaustive about testing a component's interface because the combinatorics haven't exploded yet. Because there is a lot more code under test, failures are less explanatory and thus there is more work to investigate these failures. Investments here that help are to make failure modes as helpful as possible. That, too, is an art form. End-to-end tests are inherently going to be slow. We put the whole system together and run some canned interactions on it. It might be flaky (because large scale, because networks, because OOM, timeouts, etc). End to end tests are generally a bitch to debug, because essentially anything could be at fault...well, anything except the things that are clearly passing their unit tests and integration tests. Which is why you need to have good unit and integration tests, so that you don't need many end-to-end tests. It sounds from the article like they reduced or eliminated their end-to-end tests and went for more integration tests. That does seem to have paid off. Sometimes tests are slow and bad, and other kinds of tests are better. I would say though, working now on a system with many, many, distributed moving parts, you do want to at least have some end-to-end tests that make sure everything comes up properly. Nothing like committing a change that passes all the small scale tests and then a component fails to come up because some stupid command-line flag is set wrong. You gotta have tests for anything you could absent-mindedly break. And all of that testing needs to be a one button push away. You can't have tests that developers don't run, or don't know exist. Personally I like having shell scripts that are checked in and all at least one that does the whole enchilada, even if it is just a wrapper around the build system's or CI's test targets.