5 ms·

> For example, it fails to tell apart addition from the constant function 7. Right. But how would you test for the addition function without exhaustive O(N^2)

by zvrba 12y ago

> For example, it fails to tell apart addition from the constant function 7. Right. But how would you test for the addition function without exhaustive O(N^2) search over the whole input space? (You need to test for commutativity; if you're going to test for associativity, it grows to O(N^3)).

the_af 12y ago
My comment was in the context of the "unit tests vs type systems" debate. It was meant to illustrate my opinion that "more general" is better than "specific values" when testing, as much as is practically possible. I'm aware that addition is a toy example, but suppose we want to test our implementation: Except for very simple verification, to exclude obviously broken implementations, I'd rule out testing specific values such as 3+4=7. And, like you said, performing an exhaustive exploration of all values is out of the question. So I'd try property testing instead. Relevant properties in this case are associativity, commutativity, etc. As an example, I'd try writing properties such as: for all X, Y: add(X, Y) = add(Y, X) I'm not arguing that everything can be tested like this, or that the properties are always easy to formulate; but when they are, I think this is the superior approach. The tools that I'm aware of that can do this (such as QuickCheck or ScalaCheck) come from statically typed languages, though I don't see why they couldn't be used with dynamically typed languages. The point is that this starts to look a lot closer to static typing and "testing generalities", philosophically, than what proponents of dynamic typing + unit testing propose. Testing generalities is better than testing specifics, because it's at least a step closer to a proof of correctness.
- dragonwriter 12y ago
  > The tools that I'm aware of that can do this (such as QuickCheck or ScalaCheck) come from statically typed languages, though I don't see why they couldn't be used with dynamically typed languages. The tools use type information to determine the universe to draw test values from and the mechanism used to do it. You can actually do something very similar for dynamically typed languages, but if you don't have queryable type annotations for parameters, etc., you have to have more verbose test specifications that provide the scope of testing. E.g., in a statically-typed language (or a dynamically-typed one with optional type annotations), if your add function is defined as something equivalent to: double add(double x, double y) { ... } then your test system can use that information and a value generator function for doubles to generate the appropriate test data to validate the property. OTOH, in a dynamically typed langage where you just have something like def add(x, y) { ... } Without some additional specification, the test framework doesn't know how to generate the X and Y values for the test you propose.
  - the_af 12y ago
    You are right, without additional information property testing would be less useful. Which is yet another reason to favor static typing in my opinion.
    - lerno 12y ago
      There are degrees of static typing. Swift tries to get close to Haskell, but without the full set of tools to do so without losing things along the way. This is sort of the worst of both worlds. In the meantime Strongtalk already demonstrated that you can write a fairly rich optional type system to reap the compile time benefits of static typing while retaining the runtime power of message passing. I think that would have been more fruitful.
      - josephlord 12y ago
        Why do you say it is the worst of all worlds? To me it is a very good compromise and about as good a situation as you can get while maintaining C/Objective-C compatibility. Don't forget there are performance benefits to being less dynamic (not using runtime message passing) and you can opt in to that runtime mode if you want by making classes @objc.
        lerno 12y ago
        Well, in practice the broken generics stop large amounts of reasonable uses of generics, the lack of co/contravariance force you into hideous workarounds everywhere. As for the promised runtime performance improvements they have yet to surface and worse: the language is still plagued by extremely uneven performance, even for optimized builds. I could go on at great length citing issues that will be hard to fix within the next year or so. Unfortunately. This is what my fairly hard won (15k loc of Swift code) experience with the language has revealed so far. I can honestly say I really wish you were right.
        josephlord 12y ago
- the_af 12y ago
  Downvoter: out of curiosity, which part of my post did you disagree with?

by zvrba 12y ago

the_af 12y ago

My comment was in the context of the "unit tests vs type systems" debate. It was meant to illustrate my opinion that "more general" is better than "specific values" when testing, as much as is practically possible. I'm aware that addition is a toy example, but suppose we want to test our implementation: Except for very simple verification, to exclude obviously broken implementations, I'd rule out testing specific values such as 3+4=7. And, like you said, performing an exhaustive exploration of all values is out of the question. So I'd try property testing instead. Relevant properties in this case are associativity, commutativity, etc. As an example, I'd try writing properties such as: for all X, Y: add(X, Y) = add(Y, X) I'm not arguing that everything can be tested like this, or that the properties are always easy to formulate; but when they are, I think this is the superior approach. The tools that I'm aware of that can do this (such as QuickCheck or ScalaCheck) come from statically typed languages, though I don't see why they couldn't be used with dynamically typed languages. The point is that this starts to look a lot closer to static typing and "testing generalities", philosophically, than what proponents of dynamic typing + unit testing propose. Testing generalities is better than testing specifics, because it's at least a step closer to a proof of correctness.

dragonwriter 12y ago
> The tools that I'm aware of that can do this (such as QuickCheck or ScalaCheck) come from statically typed languages, though I don't see why they couldn't be used with dynamically typed languages. The tools use type information to determine the universe to draw test values from and the mechanism used to do it. You can actually do something very similar for dynamically typed languages, but if you don't have queryable type annotations for parameters, etc., you have to have more verbose test specifications that provide the scope of testing. E.g., in a statically-typed language (or a dynamically-typed one with optional type annotations), if your add function is defined as something equivalent to: double add(double x, double y) { ... } then your test system can use that information and a value generator function for doubles to generate the appropriate test data to validate the property. OTOH, in a dynamically typed langage where you just have something like def add(x, y) { ... } Without some additional specification, the test framework doesn't know how to generate the X and Y values for the test you propose.
- the_af 12y ago
  You are right, without additional information property testing would be less useful. Which is yet another reason to favor static typing in my opinion.
  - lerno 12y ago
    There are degrees of static typing. Swift tries to get close to Haskell, but without the full set of tools to do so without losing things along the way. This is sort of the worst of both worlds. In the meantime Strongtalk already demonstrated that you can write a fairly rich optional type system to reap the compile time benefits of static typing while retaining the runtime power of message passing. I think that would have been more fruitful.
    - josephlord 12y ago
      Why do you say it is the worst of all worlds? To me it is a very good compromise and about as good a situation as you can get while maintaining C/Objective-C compatibility. Don't forget there are performance benefits to being less dynamic (not using runtime message passing) and you can opt in to that runtime mode if you want by making classes @objc.
      - lerno 12y ago
        Well, in practice the broken generics stop large amounts of reasonable uses of generics, the lack of co/contravariance force you into hideous workarounds everywhere. As for the promised runtime performance improvements they have yet to surface and worse: the language is still plagued by extremely uneven performance, even for optimized builds. I could go on at great length citing issues that will be hard to fix within the next year or so. Unfortunately. This is what my fairly hard won (15k loc of Swift code) experience with the language has revealed so far. I can honestly say I really wish you were right.
        josephlord 12y ago
the_af 12y ago
Downvoter: out of curiosity, which part of my post did you disagree with?