4 ms·
It doesn’t make sense to write no tests at all but I understand this sentiment based on problems with testing that I have seen before: - 1. Test infrastructure
by makecheck 10y ago
It doesn’t make sense to write no tests at all but I understand this sentiment based on problems with testing that I have seen before:
- 1. Test infrastructure is too complex. If I have to create a bunch of config files, obey a questionable directory structure, etc. before I can even write my test case, there is a problem. There should be very little magic between you and your test front-end.
- 2. Test infrastructure is too lacking. It is also a problem to have too little support. There should be at least enough consistency between tests that you can take a look at another test and emulate it. There should be clearly-identified tools for common operations such as pattern-excluding "diff", a "diff" that ignores small numerical differences, etc. depending on the purpose.
- 3. Existing tests should not be overly-brittle. Do NOT just "diff" a giant log file (or worse, several files), and call it a day; that means damn near any code change will cause the test to “fail” and create more work. Similarly, make absolutely certain when you develop the test that it can fail properly: temporarily force your failure condition so you know your error-detection logic is sound.
- 4. Tests should not be overly-large. Do not just take some entire product and throw it at your test, creating a half hour of run time and 40,000 possible failure points just because it happens to cover your function under test. It is vital to have a small, focused example.
If your test environment has problems like these, I fully understand the desire to balance time constraints against the hell of dealing with new or existing test cases, and wanting to avoid it completely.
And if you’re in charge of such an environment, you owe it to yourself to devote serious time to fixing the test infrastructure itself.