3 ms·
It depends. I recall one project with flaky tests, and shortly after joining I decided to investigate. It turned out to be a problem with floating-point values
by pjungwir 11y ago
It depends. I recall one project with flaky tests, and shortly after joining I decided to investigate. It turned out to be a problem with floating-point values in the shopping cart logic. Tests were using random item prices, so sometimes it appeared and sometimes not. I'm glad I found that one. :-)
On the other hand I've seen something like @flaky used in the Ruby world to run Capybara tests with a lot of Ajax, where you get failures because of timing issues deep down inside Selenium. It's not a problem with production or your app, but your testing tools.
In the second case I still am not totally comfortable, since it makes it easier to overlook flaky tests that really do need attention, but I can understand it there.
- draw_down 11y agoYeah, I'm with you. Of course things can never be perfect but we should be striving to make them as perfect as we can. I've felt Selenium pain too. But, and this is crucial-- notice how you understand why the tests were flappy. As opposed to just grumbling like "oh that dumb old thing again. I wish it would shut up, the thing totally works."