Indeed. The whole point of CI and simulation is to reduce significantly more expensive manual testing. A couple months ago, folk were grumbling about a non-det

Indeed. The whole point of CI and simulation is to reduce significantly more expensive manual testing. A couple months ago, folk were grumbling about a non-deterministic failure in the main build CI plan, for the N'th time without attempting to do anything about it. So, I proceeded to queue up dozens of jobs at a time, every 15 minutes for several hours, until I had several failures worth of interesting logs. I also had the interest of several other software teams, wondering why the entire CI pipeline was clogged up. It turned out someone years ago had written a cron job that would asynchronously delete every file on the build server older than a week, every hour on the hour. At some point some build artifacts got added that came from a cached copy on a remote server rather than locally built, and their modification timestamps were preserved. Any jobs that happened to be running while on the hour would have those files mysteriously disappear without a trace. Oops!