3 ms·
My point was that even bad unit tests are better than both good/bad word docs or no unit tests. So even bad tests written by bad programmers have a value. They
by alkonaut 9y ago
My point was that even bad unit tests are better than both good/bad word docs or no unit tests. So even bad tests written by bad programmers have a value. They have a large COST too (which is what he's arguing), my argument was merely that the value might not be less than the cost, which is his assertion.
> Software engineering research has shown that the most costeffective
> places to remove bugs are during the transition from
> analysis to design, in design itself, and in the disciplines of
> coding. It's much easier to avoid putting bugs in than to take
> them out.
Everyone knows that. It's just beating a dead horse. But companies demonstrably want to ship buggy and feature rich software fast, rather than ship well designed and lean software. And b) don't want to pay for developers of the kind that write good code from the beginning. So with that out of the way, the whole point of real world development methodology is to make sure that the feature-bloated code hastily written by average developers doesn't lack a specification, doesn't deteriorate over time into something that has to be abandoned, and doesn't have such a high risk of modification/refactoring that it can't be maintained.
Now there are some good points to 1.4 "code is better than tests" actually applies: make asserts in the code rather than in the tests, where possible. Fully agreee with that. Or even better I'd say: don't "assert" things, just make the invalid state impossible. This is what types are for. If you can't take a null into a method, don't even accept an object that could be null by accident. Accept an Option type. Yes even in Java. Even in bloody C. Anything that a compiler could have caught shouldn't be either a runtime assert nor a test.
> if bad programmers write bad code that needs unit tests, they're also going to write bad unit tests that don't test the code correctly. So what's the point?
Well firstly they are usually better than no tests. And second, they tend to fail too often, rather than too little. This is what causes the enormous cost incurred by these tests. But apart from that 90% unnecessary cost (test failures are failures of the code rather than the costs, so after changing the code you have to change the test) - they still do add a lot of confidence for refactoring. Because in my experience with a large but bad test suite, there are often false positives but false negatives are much rarer.
- mannykannot 9y agoThe logical conclusion of this line of argument, which is dependent on the proposition that even bad unit tests have value, is that we could just automatically generate copious numbers of unit test cases for a given code base, and we would have something of value. In reality, this approach would be useless even for functionally-invisible refactoring or rewriting, precisely because it would be at the unit level. For that, let alone any substantive changes, Coplien's claim that higher-level tests are better stands, so this article cannot be dismissed as "beating a dead horse."
- alkonaut 9y ago> The logical conclusion of this line of argument, which is dependent on the proposition that even bad unit tests have value, is that we could just automatically generate copious numbers of unit test cases for a given code base, and we would have something of value. I don't think that conclusion is any more valid than the proposition that if we just skipped the developer alltogether and generated both the business code AND the test, we'd have something of value. > Coplien's claim that higher-level tests are better stands, so this article cannot be dismissed as "beating a dead horse." i think higher level tests are good, but I don't think that makes lower level tests bad. I think there should usually be a health mix, with systems tested on several levels.
- mannykannot 9y ago> I don't think that conclusion is any more valid than the proposition that if we just skipped the developer alltogether and generated both the business code AND the test, we'd have something of value. That would have some value, as evidenced by all the useful spreadsheets produced by end users, but the the value of the conjunction ("the business code AND the test") lies entirely in the 'business code' part, which is why we do not automatically generate unit tests for spreadsheet macros, or any other code for that matter. >I think higher level tests are good, but I don't think that makes lower level tests bad. That's not my point; my point is that unit tests are not automatically useful. Once we accept that, Coplien's argument that there are better ways to spend our limited resources cannot be dismissed (and resources are always limited, because of the combinatorial explosion, which is most evident at the unit level).
- ekidd 9y ago> The logical conclusion of this line of argument, which is dependent on the proposition that even bad unit tests have value, is that we could just automatically generate copious numbers of unit test cases for a given code base, and we would have something of value. Actually, this is a surprisingly useful strategy! There are two common versions of this idea: 1. Invariant testing with random data. This usually goes by the name of "QuickCheck", and it allows writing tests that say things like, "If we reverse a random list twice, then it should equal the original list." In practice, this catches tons of corner case bugs. 2. Fuzz testing. In this case, the invariant is, "No matter how broken the input, my program should never crash (or corrupt the heap, or access uninitialized memory, or whatever)." Then you generate a billion random test cases and see what happens. This usually finds tons of bugs, and it's a staple of modern security testing. So yeah, even random test cases are very valuable. :-)
- hinkley 9y agoAny idiot can ship version 1.0. I get exasperated with people who try to act like they have some special wisdom to share about getting the first version out. Yes, there are some specific things you need to do that some of us lack, but you can find those in very nearly every single self-help or motivational book. It takes some proper skill, vision, and maturity to ship version 3.0. And you probably won’t get a big payday if you can’t.