3 ms·
The odds that this test will fail are on the order of a million to one. I could better those odds if I thought it necessary. That seems... pretty bad? Am I mis
by iainmerrick 7y ago
The odds that this test will fail are on the order of a million to one. I could better those odds if I thought it necessary.
That seems... pretty bad? Am I missing something?
If you have 1000 tests like that in a large codebase with a continuous build, you’re going to get an irritating number of random failures.
It’s not really a concern until you have a really large codebase. But if you leave it until you have a really large codebase, it’s a lot more effort to go back and fix the flaky tests.
- breck 7y agoAgreed. Flakey tests are horrible! The solution is a seed.
- hinkley 7y ago50 tests that fail 1% of the time will cause almost half of your builds to fail. And the odds of two failing in a row are about one in four. As the number of builds per day increases, the probability of a given day having multiple failed builds in a row (causing you to look closely at the build and commit logs) approaches 100%
- Chyzwar 7y agoUnit tests should not be flaky. Integration tests, acceptance test can be flaky by nature of networks but not unit tests.