3 ms·
You are making a whole bunch of assumptions here without ANY info about how Microsoft rolls out updates to its Azure services.
by adflux 6y ago
You are making a whole bunch of assumptions here without ANY info about how Microsoft rolls out updates to its Azure services.
- king_magic 6y agoSure, but those assumptions are largely backed up by a single update to Azure AD knocking it out worldwide. End of the day, whatever they pushed was not nearly well tested enough, and they paid for it with a massive global outage that borked pretty much everything they sell.
- regecks 6y agoTo add another datapoint, we recently faced an issue where Microsoft rolled out a change to their OIDC endpoints, which broke our client's tenant, but not other tenants. (It was crashing with an HTTP 500 if the Accept request header was a certain value: the default value in OpenJDK 8). Some days later and after our complaints, Microsoft rolled back the change. I would call this evidence that they do practice gradual deployments of changes. I would guess that this outage was caused by something more complex than MS not slow-rolling their deployment.
- extrapickles 6y agoFor something as critical as auth, I would expect most changes to be rolled out over a few days just to make sure something like this cannot happen to everyone. Ideally a change would be shadowed for a few % of traffic before it was allowed to start rolling out.