3 ms·
From that link: At Google, our philosophy is that “rollbacks are normal.” When an error is found or reasonably suspected in a new release, the releasing team ro
by ams6110 10y ago
From that link: At Google, our philosophy is that “rollbacks are normal.” When an error is found or reasonably suspected in a new release, the releasing team rolls back first and investigates the problem second.
I like that -- reminds me of aviation, where a go-around is normal. If your approach to landing isn't stabilized, you're too high, too low, too fast, too slow, etc. don't try to save it. Go around and try again.
- londons_explore 10y agoAt our place, we rollback every few weeks just to test the system, even if nothing appears abnormal. Next we plan to automate the process - 1 in 10 rollouts will actually be a rollout, a rollback, and another rollout, checking system health at each step.
- hectormalot 10y agoI just wanted to suggest this. Similar to 'chaos monkey' killing my processes once every few days, developers would also look differently at rollback procedures if I guaranteed that one in 10 would be rolled back randomly.