3 ms·

The SQLite project seems to have an alternative view: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html A lot of us would consider SQLit

by jasode 9y ago

The SQLite project seems to have an alternative view: https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html A lot of us would consider SQLite to be high quality and relatively bug-free. The extensive test suite they've built up that exercises each release is a huge reason for it. Of test code quantity, Coplien writes: >If your coders have more lines of unit tests than of code, it probably means one of several things. They may be paranoid about correctness; paranoia drives out the clear thinking and innovation that bode for high quality. > - Keep regression tests around for up to a year > - Throw away tests that haven’t failed in a year. SQLite appears to keep their tests. Somebody files a bug; SQLite writes a test that reproduces that bug; the test code remains long after the bug is fixed. This prevents the bug from reappearing. (Isn't that the main purpose of "regression" in the phrase "regression test"?) I also think it's better to keep relevant regression tests for years. E.g. Consider that there's piece of code that's been working correctly for 5 years with 5 years of passing regression tests . Imagine a new programmer wants to rewrite the code to optimize it for speed and reduced memory usage. I think we'd feel much more confidient if the new code passes those same regression tests that were accumulated over 5 years. As for test code size ratio... A lot of good comprehensive tests will have LOC outnumbering the actual code being tested. This is especially true for library code that's used in many places up the stack. I wrote string parsing routines and a reverse Boyer-Moore search routine where the test code (test edge cases, test nulls, test string sizes at 2^32 boundaries, etc) was 10 times larger than the actual code. Of testing's utility, Coplien writes: - Testing can’t replace good development - [...] Tests don’t improve quality: developers do ... which looks like a strawman and a false dichotomy. Can anyone cite a credible development philosophy that believes testing can replace bad developers or bad process? We could say that about <ANYTECHNOLOGY> such that <ANYTECHNOLOGY> can't replace quality developers. Garbage Collection doesn't improve quality, developers do. Array boundary checking doesn't improve quality, developers do. And so on. Or maybe there's a difference in terminology? I wonder if Coplien considers SQLite testing "system test" or a "unit test"? Does he consider SQLite "white box testing" or "black box testing"?

Mikushi 9y ago
> Can anyone cite a credible development philosophy that believes testing can replace bad developers or bad process? None. But that's hardly the issue, I too often have seen people championing heavy testing as a way to deliver quality software, because it's the agile way. The problem is not with the philosophies but the way people interpret them or misinterpret them. The sarcastic quote in the article sums my feeling on the behaviour I've seen in a lot of shops: "I find that weeks of coding and testing can save me hours of planning.". I've had the hardest time convincing people to spend a week thinking about our approach and solution, but adding months of development time to expend test coverage and suddenly everyone thinks it's worth it (And usually with nothing to back that up).
- abritinthebay 9y ago
  Of course, if done well, TDD does both.
softwarefounder 9y ago
Is there a reason why someone would throw away unit tests? I've never understood this. Time was taken to write it, test it, upheave the bug, and now we want to remove the safeguards that we spent time/money on? Leaving the possibility for the bug to infest again? Scrapping unit tests is scrapping time.
- vkou 9y ago
  If you are testing at the wrong layer of abstraction (Your test is tightly coupled to the implementation of your class, as opposed to its observable behavior), then refactoring your code will require refactoring the test. The correct solution to this is to not throw passing tests out, but to stop doing white-box testing. (Not to mention that a test that may pass in your CI environment may fail - frequently - in your local workspace.)

by jasode 9y ago

Mikushi 9y ago

> Can anyone cite a credible development philosophy that believes testing can replace bad developers or bad process? None. But that's hardly the issue, I too often have seen people championing heavy testing as a way to deliver quality software, because it's the agile way. The problem is not with the philosophies but the way people interpret them or misinterpret them. The sarcastic quote in the article sums my feeling on the behaviour I've seen in a lot of shops: "I find that weeks of coding and testing can save me hours of planning.". I've had the hardest time convincing people to spend a week thinking about our approach and solution, but adding months of development time to expend test coverage and suddenly everyone thinks it's worth it (And usually with nothing to back that up).

abritinthebay 9y ago
Of course, if done well, TDD does both.

softwarefounder 9y ago

Is there a reason why someone would throw away unit tests? I've never understood this. Time was taken to write it, test it, upheave the bug, and now we want to remove the safeguards that we spent time/money on? Leaving the possibility for the bug to infest again? Scrapping unit tests is scrapping time.

vkou 9y ago
If you are testing at the wrong layer of abstraction (Your test is tightly coupled to the implementation of your class, as opposed to its observable behavior), then refactoring your code will require refactoring the test. The correct solution to this is to not throw passing tests out, but to stop doing white-box testing. (Not to mention that a test that may pass in your CI environment may fail - frequently - in your local workspace.)