3 ms·

> Do not isolate code when you test it. > Only isolate your code from truly external services That makes tests more trustworthy but also sometimes harder to m

by brumar 2y ago

> Do not isolate code when you test it. > Only isolate your code from truly external services That makes tests more trustworthy but also sometimes harder to maintain I think. I have seen cases where small changes on the code base created strong ripple effects with many tests to update. Arguably, the tests were not very well written or organized and with too many high level tests. Still, this and the very large execution time of the test collection made me realized that for medium to large projects, I will be much more careful in the future before going all in with the no-mock approach.

nobleach 2y ago
Or, write code that _can_ work in isolation. (Pure functions). You can bet when GM builds fuel injectors, they have a harness that they can place them in to make sure that JUST the injector gets tested... putting it in a car and driving around is the wrong approach. The same is often true for software. Ensuring that things integrate is vital. Ensuring that functions run in isolation also has value.
- battered8310 2y ago
  Absolutely, whenever there is hardware in the loop, the more of the system you test with the more expensive the test is and the harder it is to isolate issues. That’s not to say you can skip end to end testing, but the more testing pushed to the complete system level will drive up cost and schedule.
  - nobleach 2y ago
    To beat my example into the ground. If I were to put a fuel injector (system under test) into a vehicle, and drive it around a parking lot (E2E test), only to have the vehicle stall out. Is the fuel injector at fault? How would you prove that? No, you pull out the component and you "bench test" it. Believe me, after many MANY hours spend fighting bugs where the extensive E2E suite passed, we finally had to go back to adding in unit tests for the small units under the surface. Not all... but many places.
- t0mas88 2y ago
  This makes sense when the cost of running the integration tests is high due to real hardware being in the loop. But for most business software this isn't the case and e2e tests aren't actually costly. Looking at a previous project we ran full e2e tests from an end user perspective in three major browsers in less than 15 minutes at a cloud cost of only a few hundred dollars per month. Compared to the cost of developers on the project that was a negligible amount. Adding more unit tests would make the feedback cycle shorter on some changes, but also add development time to create and maintain those tests. So in line with the article we did so only sparingly on a few critical paths that were likely to cause issues. The rest was well covered by e2e tests.
  - sarchertech 2y ago
    e2e tests that run in 15 minutes in the cloud are useless for TDD though which is what the article is talking about.

by brumar 2y ago

nobleach 2y ago

Or, write code that _can_ work in isolation. (Pure functions). You can bet when GM builds fuel injectors, they have a harness that they can place them in to make sure that JUST the injector gets tested... putting it in a car and driving around is the wrong approach. The same is often true for software. Ensuring that things integrate is vital. Ensuring that functions run in isolation also has value.

battered8310 2y ago
Absolutely, whenever there is hardware in the loop, the more of the system you test with the more expensive the test is and the harder it is to isolate issues. That’s not to say you can skip end to end testing, but the more testing pushed to the complete system level will drive up cost and schedule.
- nobleach 2y ago
  To beat my example into the ground. If I were to put a fuel injector (system under test) into a vehicle, and drive it around a parking lot (E2E test), only to have the vehicle stall out. Is the fuel injector at fault? How would you prove that? No, you pull out the component and you "bench test" it. Believe me, after many MANY hours spend fighting bugs where the extensive E2E suite passed, we finally had to go back to adding in unit tests for the small units under the surface. Not all... but many places.
t0mas88 2y ago
This makes sense when the cost of running the integration tests is high due to real hardware being in the loop. But for most business software this isn't the case and e2e tests aren't actually costly. Looking at a previous project we ran full e2e tests from an end user perspective in three major browsers in less than 15 minutes at a cloud cost of only a few hundred dollars per month. Compared to the cost of developers on the project that was a negligible amount. Adding more unit tests would make the feedback cycle shorter on some changes, but also add development time to create and maintain those tests. So in line with the article we did so only sparingly on a few critical paths that were likely to cause issues. The rest was well covered by e2e tests.
- sarchertech 2y ago
  e2e tests that run in 15 minutes in the cloud are useless for TDD though which is what the article is talking about.