Mine is fairly simple: an UPDATE without a WHERE clause on a table tracking Other People's Money. Except that was how we discovered that backups weren't working

Mine is fairly simple: an UPDATE without a WHERE clause on a table tracking Other People's Money. Except that was how we discovered that backups weren't working... Really posting to share one that happened to me, though: At a previous job, my PostgreSQL clusters were on metal (blades; not my choice, but I wasn't given one). We were in the process of replacing the SSDs in the blades, both for capacity and performance. We'd, if I remember correctly, replaced one set (the replica, I think, so we could fail over to it and then upgrade the master). The lead sysadmin decided this represented a golden opportunity to get a side-by-side performance analysis of the new and old drives. (I'll leave aside for the moment the fact that he never said anything about doing this to me, which, you know...) So he ran fio. In read-write mode. Against the block device, not the filesystem. And he did that on the primary, replica, and performance test (so, identical to prod) machines at the same time. About 20 minutes later, one of the developers, who was investigating some other issue, reported seeing "strange errors" in his psql session. I looked at the logs, and everything got very, very still for a moment... We ended up having to rebuild the machines and restore from the previous night's backups (taken 14h prior), troll the Rails logs to find the affected orders, and refund them. I did finally get the large box for WAL archiving I'd long been lobbying for out of the deal, too, so that was nice.