6 ms·

There are actually many poor claims you made in your posts about "good tests". > I'm aware that addition is a toy example, but suppose we want to test our impl

by owensd 12y ago

There are actually many poor claims you made in your posts about "good tests". > I'm aware that addition is a toy example, but suppose we want to test our implementation: > Except for very simple verification, to exclude obviously broken implementations, I'd rule out testing specific values such as 3+4=7. And, like you said, performing an exhaustive exploration of all values is out of the question. > So I'd try property testing instead. Relevant properties in this case are associativity, commutativity, etc. > As an example, I'd try writing properties such as: > for all X, Y: add(X, Y) = add(Y, X) These properties can also be satisfied by implementations of add() that: - return a constant value - return the smallest number of (x, y) - return the largest number of (x, y) The cases you threw out as an "obviously broken implementation" are required to actually validate that functionality of the method. The functionality of the method is also one of the properties of the method. You can write it in a more generic way than simply: assert(7, add(3, 4)). However, those tests are _also_ required. Without them, you never actually test that the `add()` function does what it's supposed to: add two numbers together. Regardless of type system, you also have to worry about underflows and overflows - another property of the functionality of the method. > You are right, without additional information property testing would be less useful. Which is yet another reason to favor static typing in my opinion. Static typing doesn't help you constrain sets of inputs; it may not be valid that your method accepts all ranges of integers. You could have a method `addbase2(int x, int y)` that is to be used only when x and y are powers of two because of an optimization you perform in that method. Static typing doesn't help you generate the correct input set for x and y. The only thing that static typing provides, in regards to test cases, is this: def add(x, y) assert x is int assert y in int return x + y // test cases assertIsThrown(add("foo", "bar")) That was the test case you had. Regardless, the point of the article was not about static typing being bad. There is value it. However, there is also value in not being so rigid in your type system that things don't work well. add((short)0, (long)1) // compiler error if you have an extremely rigid type system Generic systems typically swing the pendulum far to the right requiring an extremely rigid type system. That always causes pain. The question you have to ask, is the ROI worth it. For some, it is. For others, it's not.

the_af 12y ago
Note I never claimed unit testing should be disregarded (I practice it and recognize its benefits), or that static types catch all errors, or that add(x,y) was anything but a toy example. Please note I didn't throw out add(3,4)==7, but instead pointed out it's terribly inadequate as a test. Additional testing tools must be employed; unit testing alone of this kind is not enough. With property testing you're still not proving correctness. Tests cannot prove correctness. But it's a step in the right direction. Sure, maybe you have a function "add" that is associative, commutative, and has a neutral element, and it's still not integer addition. I'd argue your confidence in such a function will be a lot higher than if you had simply unit tested a few border cases. You can still do that in addition to property testing, anyway. > The only thing that static typing provides, in regards to test cases, is this [example] This assertion is wrong. Static typing done well provides a lot of things "for free", such as restricting incorrect behavior. For example, if you write generic methods you can rule out entire classes of misbehavior. It's not that you "type check" that you are not using something that is not an int (as in your example), but that you simply forbid entire groups of operations at compile time! Here's another toy example to illustrate the point: what values can a function with the following signature return? f :: [a] -> a (For the purposes of this question, you can read that as "a function that takes a list of type a and returns a value of type a). Now, repeat the exercise with a dynamically typed language. What values can the following function return? (If you want, for the purposes of this question, assume it returns an atomic value and not a collection). dynamic_f(a_list) This has an obvious implication on the effort you must make when testing either function.
- owensd 12y ago
  > Please note I didn't throw out add(3,4)==7, but instead pointed out it's terribly inadequate as a test. Additional testing tools must be employed; unit testing alone of this kind is not enough. I don't follow what you are saying here at all... It's an API test with no external integration points, what other kind of tests besides unit tests would you have? Your `add(x, y) = add(y, x)` are still unit tests. Also, you said that you would "rule out testing specific values such as 3+4=7". You have to test specific values; the contract of the function is: f(x, y) = z Where z is the mathematical sum of x and y. Specific value testing is the only way you can verify that claim for a given set of inputs. >> The only thing that static typing provides, in regards to test cases, is this [example] > This assertion is wrong. Static typing done well provides a lot of things "for free", such as restricting incorrect behavior. This was taken out of context; this was in reference your to "generator" of test values for X and Y. The type signature alone is inadequate to generate test values. Regardless, f :: [a] -> a Is no harder to test in a dynamic language. let r = f([...]) assert(r, correct_value) assert(r.type, correct_type) It's up to the contract of the function to determine what, if any, validation needs to be done on the input. This is true regardless of type system. The only question is this: do you also check the type. Again, type is only _one_ of the constraints that get applied to parameters. In the add function example, the other constraints are: 1. x <= TYPE_MAX (for free in a statically typed system) 2. x <= TYPE_MAX - y 3. y <= TYPE_MAX (for free in a statically typed system) 4. y <= TYPE_MAX - x Plus the similar for TYPE_MIN. In the `addbase2` example, additional constraints are: 1. x power of 2 2. y power of 2 In this example, we still need to add verification for two-thirds of the constraints. On the flip-side, with generics, especially with the type of generics we see in Swift (using the `f :: [a] -> a` example), you'll probably need to model constraints of the collection, the element type of the collection, and the type of indexer that is being used if you wish to make your function actually work. And then your implementation only works for those that rigidly adhere to the type conformance, where as the dynamic one can work for any type that conforms to the protocol, whether loosely or explicitly. This flexibility is very powerful, is not hard to code safely around, and requires significant less code gymnastics before you can even get your code compiling.

by owensd 12y ago

the_af 12y ago

Note I never claimed unit testing should be disregarded (I practice it and recognize its benefits), or that static types catch all errors, or that add(x,y) was anything but a toy example. Please note I didn't throw out add(3,4)==7, but instead pointed out it's terribly inadequate as a test. Additional testing tools must be employed; unit testing alone of this kind is not enough. With property testing you're still not proving correctness. Tests cannot prove correctness. But it's a step in the right direction. Sure, maybe you have a function "add" that is associative, commutative, and has a neutral element, and it's still not integer addition. I'd argue your confidence in such a function will be a lot higher than if you had simply unit tested a few border cases. You can still do that in addition to property testing, anyway. > The only thing that static typing provides, in regards to test cases, is this [example] This assertion is wrong. Static typing done well provides a lot of things "for free", such as restricting incorrect behavior. For example, if you write generic methods you can rule out entire classes of misbehavior. It's not that you "type check" that you are not using something that is not an int (as in your example), but that you simply forbid entire groups of operations at compile time! Here's another toy example to illustrate the point: what values can a function with the following signature return? f :: [a] -> a (For the purposes of this question, you can read that as "a function that takes a list of type a and returns a value of type a). Now, repeat the exercise with a dynamically typed language. What values can the following function return? (If you want, for the purposes of this question, assume it returns an atomic value and not a collection). dynamic_f(a_list) This has an obvious implication on the effort you must make when testing either function.

owensd 12y ago
> Please note I didn't throw out add(3,4)==7, but instead pointed out it's terribly inadequate as a test. Additional testing tools must be employed; unit testing alone of this kind is not enough. I don't follow what you are saying here at all... It's an API test with no external integration points, what other kind of tests besides unit tests would you have? Your `add(x, y) = add(y, x)` are still unit tests. Also, you said that you would "rule out testing specific values such as 3+4=7". You have to test specific values; the contract of the function is: f(x, y) = z Where z is the mathematical sum of x and y. Specific value testing is the only way you can verify that claim for a given set of inputs. >> The only thing that static typing provides, in regards to test cases, is this [example] > This assertion is wrong. Static typing done well provides a lot of things "for free", such as restricting incorrect behavior. This was taken out of context; this was in reference your to "generator" of test values for X and Y. The type signature alone is inadequate to generate test values. Regardless, f :: [a] -> a Is no harder to test in a dynamic language. let r = f([...]) assert(r, correct_value) assert(r.type, correct_type) It's up to the contract of the function to determine what, if any, validation needs to be done on the input. This is true regardless of type system. The only question is this: do you also check the type. Again, type is only _one_ of the constraints that get applied to parameters. In the add function example, the other constraints are: 1. x <= TYPE_MAX (for free in a statically typed system) 2. x <= TYPE_MAX - y 3. y <= TYPE_MAX (for free in a statically typed system) 4. y <= TYPE_MAX - x Plus the similar for TYPE_MIN. In the `addbase2` example, additional constraints are: 1. x power of 2 2. y power of 2 In this example, we still need to add verification for two-thirds of the constraints. On the flip-side, with generics, especially with the type of generics we see in Swift (using the `f :: [a] -> a` example), you'll probably need to model constraints of the collection, the element type of the collection, and the type of indexer that is being used if you wish to make your function actually work. And then your implementation only works for those that rigidly adhere to the type conformance, where as the dynamic one can work for any type that conforms to the protocol, whether loosely or explicitly. This flexibility is very powerful, is not hard to code safely around, and requires significant less code gymnastics before you can even get your code compiling.