4 ms·
But exactly these tests tend to be the useless ones that just pass and don't find bugs. On 2 separate occasions I had an app with extensive unit tests that see
by fpig 9y ago
But exactly these tests tend to be the useless ones that just pass and don't find bugs.
On 2 separate occasions I had an app with extensive unit tests that seemed to work fine but also seemed to have strange rare bugs.
Both times I wrote a single additional "unit" test that fired up the environment (with mocks, same as the other unit tests) but then acted like a consumer of the API and spammed the environment with random (but not nonsense) calls for several minutes. These tests were quite complex so basically the exact opposite of what you're suggesting.
Not only did I immediately find the bug, but in both cases I found like 10 bugs before I got the test to even pass for the first time.
At the same time all the small unit tests were happily passing. Because they didn't hit edge cases (both in data and in timing) that nobody had thought of.
- taneq 9y agoYour API-spammer interface reminds me of the Macintosh "Monkey": https://www.folklore.org/StoryView.py?project=Macintosh&story=Monkey_Lives.txt https://www.folklore.org/StoryView.py?project=Macintosh&stor...
- fpig 9y agoYeah it's pretty similar except mine was not interesting to watch as it works :D
- josteink 9y agoA test can still be simple while providing garbage edge-case data. In fact that’s usually one of my default tests to ensure that I have a well-defined behavior even in these cases.
- fpig 9y agoYeah but the cases I had in mind, I probably coded correctly. The cases I forgot about are more likely to cause problems (weird cases in both data and especially timing / concurrency handling).