5 ms·
> Because most of the time, it reduces "test precision". Which means when such a test detects an issue, there's now a bigger area of the code where the issue co
by arkh 9y ago
> Because most of the time, it reduces "test precision". Which means when such a test detects an issue, there's now a bigger area of the code where the issue could be located.
Tests are not a debug tool. Tests are here to tell you when you broke something.
When this happen you can get your debugging toolbox out: stack traces, profilers and things like GDB. And then follow the steps your test script did.
- amarkov 9y agoAll the projects I've most enjoyed working on had clear, explicit test failures, which almost always gave me enough information to hunt down what I'd broken and where. Obviously that's not realistic for some kinds of projects. But even in a legacy C project, if I had to pull out stack traces and profilers and GDB every single time I wanted to understand why a test failed, I would say I'm working in a pretty terrible codebase.
- newfoundglory 9y agoI disagree and am horrified by the dystopic image of our entire test suite having two possible results: "everything passed" or "something is wrong".
- arkh 9y agoNot two results. But "when you click on this after that this happen while it shouldn't" is usually enough info to start debugging: you have reproducible steps. Which you can do with a debugger running so you see exactly were and how things break. And it is a lot less brittle than "well we tried to refactor some minor thing and now everything is red; but we don't know if the software behavior changed or just because our tests were just checking the implementation".
- newfoundglory 9y agowell, if you're determined to write stupid tests you can do it at any level. And that is not enough info to start debugging unless you had the machine in a known and reproducible state to begin with, which is where we get back to writing focused reproducible tests.