4 ms·
Testing is important, but unit tests are the least important of them. Powerful type systems and compilers that catch your mistakes before you can run anything
by Olreich 5y ago
Testing is important, but unit tests are the least important of them.
Powerful type systems and compilers that catch your mistakes before you can run anything is the best kind of testing. It’s fully automated and will inform you about most things early on. Practically, this kills dynamic languages for projects expected to become larger than proof of concept (pydantic, typescript, etc. mitigate, but must be in and enforced from the start).
After that manual testing is the most important. Specifically because it can be done by hand with isolated inputs and outputs. This won’t catch the refactoring breaking something disparate, but will ensure you can reproduce production issues with the code locally where you can expect it.
After that, integration tests are the most important. Give the whole component/system an input, and verify the output. This should be as fast as possible but should mock as little as possible. This should be able to be done from a local machine with a debugger running and still be fast. If an integration test takes half an hour, it’s never going to get run except on a Thursday at 6pm when you’re trying to get out a change before the Sprint review. Then it will fail and make life hell, because running it is so slow that running with a debugger is going to be hell.
After that, logging is the most important form of testing. Every input or start of a workflow should get an ID that follows it through the system with logging to watch it go. Any problem identified should include that Id so that you can grab logs from all the disparate systems. Specifically, that ID should be attached to the inputs or each component so you can replay it in a controlled environment later.
Then you finally get to unit tests, because once your software works as expected, they let you lock down the implementation and ensure that any changes will get flagged by the tests. If you lock down the unit tests before the software works, then you wind up having to go back and redo the unit tests for every issue found, as the implementation wasn’t correct so the unit tests can’t test it. This is the big lie of unit testing. They don’t prove correctness, they prove implementation details.
After that you want to get to end-to-end that workflow the whole user experience with a product or system. But if you don’t get there, that’s really okay.
Most problems are caught by the compiler, manual tests, and integration tests during development. Any problems that do come up can be identified quickly and repaired by the highly inspectable infrastructure that provides you IDs to dig out logs for any input, and while you’re fixing the issue, your unit tests will let you know when the implementation has changed in a way that requires more thought (though the compiler probably did first and the unit test complaints are just noise).
- deleted 5y ago[deleted]
- lbayes 5y agoI spent quite a few years living in dynamic languages and looking enviously over the wall at those lucky engineers who could use static type systems to catch all their bugs (truly, this is not sarcasm). Then I finally got an opportunity to work in a handful of those environments, of course the first was everyone's favorite whipping boy, Java. Yikes! Maybe I've just been unlucky or unsmart, but wow, I've seen (and even made) some really impressive messes with these languages (specifically: Java, ActionScript 3.0, Closure (.js), Typescript, Go, C++ and C). In my experience, especially in the context of UI development, the static type systems I've encountered generally make the task of unit testing more difficult and time consuming, while adding little to reliability. At the end of the day, I just don't bump into major type mismatches in dynamic languages that cause a whole lot of hearteache. All that said, Go was probably the best static type system I've worked with until I (recently) discovered Zig, which has a really interesting type system (esp. related to comptime) and has testing built into the language itself. While it might at first look like it, I'm not saying static type systems don't have a home in my heart. I'm just saying that in my experience (just like TDD and Unit Testing), they haven't led to the bug-free panacea that proponents tend to imply. I do find it endlessly fascinating that 2 people can walk through similar-ish challenges and arrive at entirely opposite conclusions about how to best tackle them. Thanks!