4 ms·

The best way to define a spec in the most unambiguous way is actually writing tests.

by knocte 9y ago

deleted 9y ago
[deleted]
eesmith 9y ago
I disagree. Consider this counter-example, "This function takes a float64 'x' where 1.0d <= f <= 100000.0d and returns a float64 'y' which is the square root of 'x' to within 10ulp." Tests can verify that the implementation with certain inputs is correct. If done incorrectly they can even over-specify the spec, like asserting that sqrt(16.0d) exactly equals 4.0d when it should also allow 4.000000000000001 But I don't see how they are less ambiguous or why they are better. (As a real-world example, look to the Pentium FDIV bug.)
- Ace17 9y ago
  Tests are formal. Which means there's a lot less room for ambiguity, and that you can check an implementation for nearly zero-cost. I'm sure you can guess what the following function does by only looking at the following (incomplete) test suite: assertEquals([], f([], [])); assertEquals([1,2,3,4], f([1,2,3,4], [])); assertEquals([1,2,3,4,5,6,7,8], f([1,2,3,4], [5,6,7,8])); assertEquals([1,2,3,4,5,6,7,8], f([1,3,5,7], [2,4,6,8])); assertEquals([1,2,3,4,5], f([1,2,3,5], [4])); ... now, imagine the same test suite, but with the name 'f' replaced with 'mergeSortedLists'. By the way, I don't get how your example actually proves your point: you're basically saying that tests "done incorrectly" could be harmful. So what?
  - eesmith 9y ago
    It appears that your f is: def f(x, y): return sorted(x+y) However, it could also be the more memory efficient in-place version: def f(x, y): x.extend(y) x.sort() return x It could even be: def f(x, y): return list(set(x).union(y)) ... and it turns out I was wrong. It's: import heapq def f(x, y): return list(heapq.merge(x, y)) I only figured that out when I read the name "mergeSortedLists". It's the text description which clued me in to what it was supposed to be, not the test cases. I'm saying that test cases only verify specific data points. They don't define what happens across the entire range of inputs. I'm sure you can't guess what the following tests verify: assertEquals(f(1), 1) assertEquals(f(2), 2) assertEquals(f(3), 3) assertEquals(f(4), 4) assertEquals(f(5), 5) assertEquals(f(6), 6) assertEquals(f(7), 7) assertEquals(f(8), 8) assertEquals(f(9), 9) assertEquals(f(10), 10) assertEquals(f(11), 11) assertEquals(f(12), 12) assertEquals(f(13), 14) Now replace this exemplar-based specification with "f(i) returns the i^th 11-smooth number: numbers whose prime divisors are all <= 11; i>= 1". What test suite gives a better definition of what f is supposed to do than the one-line text-based specification? (The OEIS says those tests could also be the divisors of 27720, the "Paradigm Shift Sequence for a (-4,5) production scheme with replacement", "Numbers all of whose prime factors are palindromes", or even the positive "Numbers in decimal representation, such that in German their digits are in alphabetic order", and more.)

The best way to define a spec in the most unambiguous way is actually writing tests.

by knocte 9y ago

The best way to define a spec in the most unambiguous way is actually writing tests.

deleted 9y ago

[deleted]

eesmith 9y ago

I disagree. Consider this counter-example, "This function takes a float64 'x' where 1.0d <= f <= 100000.0d and returns a float64 'y' which is the square root of 'x' to within 10ulp." Tests can verify that the implementation with certain inputs is correct. If done incorrectly they can even over-specify the spec, like asserting that sqrt(16.0d) exactly equals 4.0d when it should also allow 4.000000000000001 But I don't see how they are less ambiguous or why they are better. (As a real-world example, look to the Pentium FDIV bug.)

Ace17 9y ago
Tests are formal. Which means there's a lot less room for ambiguity, and that you can check an implementation for nearly zero-cost. I'm sure you can guess what the following function does by only looking at the following (incomplete) test suite: assertEquals([], f([], [])); assertEquals([1,2,3,4], f([1,2,3,4], [])); assertEquals([1,2,3,4,5,6,7,8], f([1,2,3,4], [5,6,7,8])); assertEquals([1,2,3,4,5,6,7,8], f([1,3,5,7], [2,4,6,8])); assertEquals([1,2,3,4,5], f([1,2,3,5], [4])); ... now, imagine the same test suite, but with the name 'f' replaced with 'mergeSortedLists'. By the way, I don't get how your example actually proves your point: you're basically saying that tests "done incorrectly" could be harmful. So what?
- eesmith 9y ago
  It appears that your f is: def f(x, y): return sorted(x+y) However, it could also be the more memory efficient in-place version: def f(x, y): x.extend(y) x.sort() return x It could even be: def f(x, y): return list(set(x).union(y)) ... and it turns out I was wrong. It's: import heapq def f(x, y): return list(heapq.merge(x, y)) I only figured that out when I read the name "mergeSortedLists". It's the text description which clued me in to what it was supposed to be, not the test cases. I'm saying that test cases only verify specific data points. They don't define what happens across the entire range of inputs. I'm sure you can't guess what the following tests verify: assertEquals(f(1), 1) assertEquals(f(2), 2) assertEquals(f(3), 3) assertEquals(f(4), 4) assertEquals(f(5), 5) assertEquals(f(6), 6) assertEquals(f(7), 7) assertEquals(f(8), 8) assertEquals(f(9), 9) assertEquals(f(10), 10) assertEquals(f(11), 11) assertEquals(f(12), 12) assertEquals(f(13), 14) Now replace this exemplar-based specification with "f(i) returns the i^th 11-smooth number: numbers whose prime divisors are all <= 11; i>= 1". What test suite gives a better definition of what f is supposed to do than the one-line text-based specification? (The OEIS says those tests could also be the divisors of 27720, the "Paradigm Shift Sequence for a (-4,5) production scheme with replacement", "Numbers all of whose prime factors are palindromes", or even the positive "Numbers in decimal representation, such that in German their digits are in alphabetic order", and more.)