In a different life I did some freelancing, and worked on an app that had a really bad outage at midnight local time on a seemingly arbitrary Thursday. The firs

In a different life I did some freelancing, and worked on an app that had a really bad outage at midnight local time on a seemingly arbitrary Thursday. The first HTTP request that arrived to the company front page after midnight crashed. Every subsequent HTTP request did the same. A friend took the call, and started tracing logs. A strange problem came into focus - requests to the company front page were crashing in a subroutine for scheduling the weekly bulk FTP upload of materials to a paper printing shop - thousands of pamphlets and books the company would order to be printed each Thursday. It turned out that the way the scheduling worked was: Each HTTP request to the company's website would check "is it thursday?" and "if yes, are there any files in the print shops FTP server?". If you were the unlucky browser that happened to answer yes to both, your latency to load the website was very high, as it also included the very large FTP upload. Anyway. The routine that checked if todays date was a thursday worked by comparing the current date to a hand-written list of thursdays. The prior week had been the last thursday the hacker that wrote the FTP upload job had added to the list.