Did this at insurance company back in 99. They'd been running on desktop pcs with tops off in a closet (server room) with carpet floors. Built a cluster in two

Did this at insurance company back in 99. They'd been running on desktop pcs with tops off in a closet (server room) with carpet floors. Built a cluster in two full racks and Friday after COV started migration. Failed all though Saturday and Sunday to restore data. Has tested backups (DAT drives) after backup which passed but something corrupted. Found another backup tape, and had it up at 0745am on Monday. Zero sleep all weekend had to call in some other admins to get it done. Was both stressful and exhilarating. Became extremely cautious with backup testing since then.