3 ms·
About 15 years ago was very happy with his source code backups, the code was on an IRIX server and a cronjob tarred up the backups and he kept multiple copies o
by iso1210 6y ago
About 15 years ago was very happy with his source code backups, the code was on an IRIX server and a cronjob tarred up the backups and he kept multiple copies of the tars offsite, and even offline. The tarballs even extracted fine and at first glance the directory structure and file lengths were ok.
Trouble is a bug in the file system meant that somehow the files had been overwritten with nulls.
The program continued to run, as it sat in memory. Until one day the box was rebooted (power test or something). It didn't come back. The files were great, containing multiple files all of the right looking length. Everything was null. "main.c" or whatever it was was 8 kilobytes of null characters.
This had been happening for some months so even the oldest source backups were wiped, and the actual binaries were out too. The system was called ELVIS, which made for a great status report.
As it was on an ancient irix machine and he was rewriting in PHP at the time he just accelerated that and we were back up in a few days (the program was a front end to a database and file stores on other machines, so it was just the web part)