3 ms·
Rollback is a reliable strategy when the rollback process is well understood. If a rollback process is not well known and well experienced, then it is a risk in
by liampulles 10mo ago
Rollback is a reliable strategy when the rollback process is well understood. If a rollback process is not well known and well experienced, then it is a risk in itself.
I'm not sure of the nature of the rollback process in this case, but leaning on ill-founded assumptions is a bad practice. I do agree that a global rollout is a problem.
- newsoftheday 10mo agoRollback carries with it the contextual understanding of complete atomicity; otherwise it's slightly better than a yeet. It's similar to backups that are untested.
- marcosdumay 10mo agoComplete atomicity carries with it the idea that the world is frozen, and any data only needs to change when you allow it to. That's to say, it's an incredibly good idea when you can physically implement it. It's not something that everybody can do.
- newsoftheday 10mo agoNo, complete atomicity doesn't require a frozen state, it requires common sense and fail-proof, fool-proof guarantees derived from assurances gained from testing. There is another name for rolling forward, it's called tripping up.
- programd 10mo agoGlobal rollout of security code on a timeframe of seconds is part of Cloudflare's value proposition. In this case they got unlucky with an incident before they finished work on planned changes from the last incident.
- flaminHotSpeedo 10mo agoThat's entirely incorrect. For starters, they didn't get unlucky. They made a choice to use the same system they knew was sketchy (which they almost certainly knew was sketchy even before 11/18) And on top of that, Cloudflare's value proposition is "we're smart enough to know that instantaneous global deployments are a bad idea, so trust us to manage services for you so you don't have to rely on in house folks who might not know better"