5 ms·
I know that's just an example and that you probably don't actually write tests like that in your day job, but it's a terrible unit test. For example, it fails t
by the_af 12y ago
I know that's just an example and that you probably don't actually write tests like that in your day job, but it's a terrible unit test. For example, it fails to tell apart addition from the constant function 7.
Even if you add more data points, you're still not testing addition.
In a way, "testing for specific values" is very misleading. Consider that it'd be the wrong if you were actually trying to prove something. Now, tests aren't proofs, but we still should write them to be as strong as it's practical.
Strong static typing with property testing (a la quickcheck) whenever possible are preferrable in my opinion.
- zvrba 12y ago> For example, it fails to tell apart addition from the constant function 7. Right. But how would you test for the addition function without exhaustive O(N^2) search over the whole input space? (You need to test for commutativity; if you're going to test for associativity, it grows to O(N^3)).
- the_af 12y agoMy comment was in the context of the "unit tests vs type systems" debate. It was meant to illustrate my opinion that "more general" is better than "specific values" when testing, as much as is practically possible. I'm aware that addition is a toy example, but suppose we want to test our implementation: Except for very simple verification, to exclude obviously broken implementations, I'd rule out testing specific values such as 3+4=7. And, like you said, performing an exhaustive exploration of all values is out of the question. So I'd try property testing instead. Relevant properties in this case are associativity, commutativity, etc. As an example, I'd try writing properties such as: for all X, Y: add(X, Y) = add(Y, X) I'm not arguing that everything can be tested like this, or that the properties are always easy to formulate; but when they are, I think this is the superior approach. The tools that I'm aware of that can do this (such as QuickCheck or ScalaCheck) come from statically typed languages, though I don't see why they couldn't be used with dynamically typed languages. The point is that this starts to look a lot closer to static typing and "testing generalities", philosophically, than what proponents of dynamic typing + unit testing propose. Testing generalities is better than testing specifics, because it's at least a step closer to a proof of correctness.
- dragonwriter 12y ago> The tools that I'm aware of that can do this (such as QuickCheck or ScalaCheck) come from statically typed languages, though I don't see why they couldn't be used with dynamically typed languages. The tools use type information to determine the universe to draw test values from and the mechanism used to do it. You can actually do something very similar for dynamically typed languages, but if you don't have queryable type annotations for parameters, etc., you have to have more verbose test specifications that provide the scope of testing. E.g., in a statically-typed language (or a dynamically-typed one with optional type annotations), if your add function is defined as something equivalent to: double add(double x, double y) { ... } then your test system can use that information and a value generator function for doubles to generate the appropriate test data to validate the property. OTOH, in a dynamically typed langage where you just have something like def add(x, y) { ... } Without some additional specification, the test framework doesn't know how to generate the X and Y values for the test you propose.
- the_af 12y agoYou are right, without additional information property testing would be less useful. Which is yet another reason to favor static typing in my opinion.
- lerno 12y agoThere are degrees of static typing. Swift tries to get close to Haskell, but without the full set of tools to do so without losing things along the way. This is sort of the worst of both worlds. In the meantime Strongtalk already demonstrated that you can write a fairly rich optional type system to reap the compile time benefits of static typing while retaining the runtime power of message passing. I think that would have been more fruitful.
- josephlord 12y agoWhy do you say it is the worst of all worlds? To me it is a very good compromise and about as good a situation as you can get while maintaining C/Objective-C compatibility. Don't forget there are performance benefits to being less dynamic (not using runtime message passing) and you can opt in to that runtime mode if you want by making classes @objc.
- mpweiher 12y ago> it fails to tell apart addition from the constant function 7 It's not supposed to do that. Have you heard of TDD? In a TDD/XP setting, the constant function 7 would be the appropriate implementation for making that test pass, because it is the simplest thing that could possibly work. Then you add another test, let's say add(40,2) EXPECT(42). Now you could extend your add() function to do case analysis, and maybe in a first step you even do that. But then you refactor towards better code, and you replace the case analysis with actual addition. For addition the steps are, of course, a bit contrived, because it is "obvious" what is supposed to happen. For production code the technique works really well in keeping the solution as simple as possible but no simpler. You probably wouldn't believe all the "obviously needed" code I haven't written because of it. Another interesting benefit is that it splits coding into two distinct activities: (1) making the test pass as stupidly as you can and only then (2) making the code good by refactoring an existing solution. Doing a good job of (2) is much, much easier when you are transforming a solution known to work and safeguarded by tests. Also having just two test to induce addition may seem a bit sparse, and it is(!), but in my practical experience with TDD, I have been utterly surprised by how few concrete examples have been sufficient to safely cover potentially large spaces. Much fewer than I would have suspected or believed possible.
- the_af 12y agoI'm familiar with TDD, both its strenghts and limitations (http://ravimohan.blogspot.com.ar/2007/04/learning-from-sudoku-solvers.html http://ravimohan.blogspot.com.ar/2007/04/learning-from-sudok..., http://beust.com/weblog/2014/05/11/the-pitfalls-of-test-driven-development/ http://beust.com/weblog/2014/05/11/the-pitfalls-of-test-driv..., etc). I understand that this kind of testing often works (though I'm less enthused with TDD as a design technique, which is unfortunately what TDD proponents emphasize, and what you seem to be describing here). I'm saying it is too limited. "But software is written and tested this way", you can say. But buggy software is written every day, even with testing, which is why we should strive to improve our testing tools & processes. To be clear I'm not saying "abandon TDD" (ok, maybe I'm not rooting for TDD in particular, but I'm definitely not saying "abandon unit testing"). What I am saying is "complement it with static typing and more advanced testing techniques, such as property testing." Finally: >> it fails to tell apart addition from the constant function 7 > It's not supposed to do that. Well, in a way it is. Your tests must attempt to detect flawed implementations, even though they cannot prove correctness. I'm sure you can think of cases where an algorithm sometimes seems to return the correct result, but fails in some cases.