The same as anything really. You try and write a system, you realise that logically it was harder than you first assumed, you note there are failure scenarios

The same as anything really. You try and write a system, you realise that logically it was harder than you first assumed, you note there are failure scenarios that you're aware of. You mitigate and ship. Later as the bug reports come tumbling in you realise there were failure scenarios that you weren't aware of, usually because you were relying on something that you cant actually rely on. The fact that this is formalised is great because it means people who don't talk about when their systems don't work aren't giving you the whole picture.