I was in a successful company that nearly died in 2017, where our entire production system corrupted itself due to a sneaking scale bug we had ported the system

I was in a successful company that nearly died in 2017, where our entire production system corrupted itself due to a sneaking scale bug we had ported the system into. The problem was, the system with data had been running migrated for 3 months with the bug in it, so it was no longer possible to revert to the earlier working design. We were down for a week where no clients could run, and we spent the next 12 months purely digging ourselves out of that hole, with all new development paused, and all hands on limping the ship along. I would say, that bug was very close to ending us. Luckily, we never disappointed in a similar way since.