3 ms·
You could approach this a number of ways. First, if you can ensure the seed of the generator, then that could be used on a case by case basis. But this tricky,
by henrikeh 6y ago
You could approach this a number of ways.
First, if you can ensure the seed of the generator, then that could be used on a case by case basis. But this tricky, since it introduces a subtle dependency on the implementation of biasedcoin.
Another option is to split the function/process into a stochastic and deterministic part. Test the deterministic part thoroughly. biasedcoin is too trivial for this, but it works nicely in more complicated setups.
For biasedcoin, I would probably go with a statistical approach. Given a certain bias, the expected value and distribution is known. I would simply test that several trials of the function lie within some bound.
- tcollier 6y agoThe statistical approach definitely gives a developer more confidence in the correctness of a solution and guards against regressions. Though any test that checks the outcome of a random process against statistical measures is expected to fail occasionally. For example, if the test flips the coin 1,000 times, there is better than a 99% chance that the outcome would result in 450 to 550 heads. So if you write the test using `450 <= heads <= 550`, you know that it will fail ~1% of the time. And if you expand the range to reduce the rate of false negatives, you have reduced the confidence the test is validating correctness. Having said that, I still find statistical tests to be very helpful when building out code that uses randomness. However, these tests typically do not make it into the CI/CD pipeline.
- smallnamespace 6y agoIn practice there is no such tradeoff, the tails of a Gaussian fall off extremely rapidly so if you simply 10x the number of iterations you’ll have failure become astronomically unlikely.
- daveFNbuck 6y agoYou can safely use this type of test in your pipeline if you set a fixed random seed.