2 ms·
There's a semantic issue. Random number test are less often that it does X and more often that it doesn't do !X. For example, the description for the final test
by dalke 11y ago
There's a semantic issue. Random number test are less often that it does X and more often that it doesn't do !X. For example, the description for the final test #6 is that it "generates random number between 1 and 100 inclusive". However, the test doesn't do that. It only tests that the method doesn't generate a value outside the range 1 to 100.
For example, if the implementation were that from step #2, which always returns 50, then it will pass the test, even though the underlying code isn't random. A better description of the final test is "generates a value between 1 and 100 inclusive", that is, remove the word 'random' and highlight that it only does one test.
The test suite really should fail if #2 were substituted for the final result. This also means this is not an example of red-green TDD as no test actually failed, to drive the code change.
"[Step 5] The tests will pass"
I don't think the author tested that code. It shouldn't pass, if I understand correctly what it's doing. It generates 100 random numbers in the range 1 to 100, inclusive, then tests that each of the values 1 to 100 are in the result. This is very unlikely. The odds of this happening are 100! / (100^100) or about 1E-42.
It worked in the original code because 100 draws of 6 values has a (5/6)^100 ~ 1E-8 chance of not failing.
One of the common failure modes in code like this is to omit one of the end points. The test should run long enough to see at least one 1 and one 100, and no 0 and no 101. Assuming a failure rate of 1 in a million is acceptable, that's log(1/1000000) / log(99/100) or about 1400 tests.
To answer the questions posed:
"Why generate 100 random numbers?" Because for 6 numbers it gives a high likelihood of checking that only the number 1-6 are generated. But it isn't a good test for numbers in the range 1-100.
"What is the minimum number of random numbers you need to write the test?" About 1500.
"How can we specify that the generated random number is within the expected range of numbers?" That's the easiest part of the test, and doesn't give much confidence that the test is valid.
"What if we can eliminate the loop altogether so that we don't generate 100 random numbers for each test run?" Then it's a low-quality test, which adds little that wouldn't be found by the high-quality tests which are still needed for this code.
Consider if the implementation were "Random.new.rand(0..100)". How often would the test from step #6 pass even though the underlying code is wrong? About 50 times on average.