4 ms·
Well, the question is what you do when a test starts failing non-deterministically. If your team respond to a random test failure by finding the root cause and
by michaelt 5y ago
Well, the question is what you do when a test starts failing non-deterministically.
If your team respond to a random test failure by finding the root cause and fixing it, uncertain test inputs could be right for you.
On the other hand, if your team responds by hitting the 'rerun tests' button until it passes? In that case random failures are just slowing you down.
- goto11 5y agoThe correct response is to "freeze" the parameters of the failing test into a regular test which fails consistently.
- someotherperson 5y agoI'm not entirely following here. What is the point of even having tests if there are teams out there who disregard their outputs? Freezing a set of parameters that always pass, in case you don't run into the failing edge case, defeats the entire point of testing in the first place. If it's one test that is failing because the test itself is defective, and if that test is not for a crucial part of the product, then skip the test in the interim and come back to it later. But giving it a happy path for success sounds really strange to me.