4 ms·

I would stay away from any company that thinks "we need to go multicloud" as a response to this outage. This affected a single az in a single region. If it caus

I would stay away from any company that thinks "we need to go multicloud" as a response to this outage. This affected a single az in a single region. If it caused you downtime or a partial outage, it means you are not fully resilient to single az failures. The correct thing is to fix your application to handle that. If you can't handle a single az failure there is no way you are going to handle failing over across different cloud providers correctly.

CubsFan1060 5y ago
There will, however, be a lot of executives _talking_ about going multi cloud.
luhn 5y ago
To be fair though, from what I heard AZ outage caused an EC2 API brownout, so people couldn't launch new instances in the other AZs. That put a wrench in a lot of multi-AZ architectures. Not advocating for multi-cloud though...
dragonwriter 5y ago
> If it caused you downtime or a partial outage, it means you are not fully resilient to single az failures. Given the number of AWS global services that have dependencies on infra in US-EAST-1 (and, from the impacts of this and other past outages, seen vulnerable to single-AZ failures in US-EAST-1) that's...less avoidable for certain regions/AZs than one might naively expect. Most clouds seem to have at least some degree of this kind of vulnerability.
nonane 5y ago
> This affected a single az in a single region. If it caused you downtime or a partial outage, it means you are not fully resilient to single az failures. This is not true. Amazon is not being upfront about what happened here. It was simply not a single AZ failure. Our us-east-1 ELB load balancers were hosed and were unable to direct traffic to other AZs - they simply stopped working an were dropping traffic. We tried creating load balancers in different AZs and that didn't work either. How can you be resilient to single AZ failures if load balancers stop working region wide during a single AZ outage?
- acdha 5y ago
  Did your TAM go into any details on that? Over a couple hundred load-balancers, the only issue we had was taking longer to register new instances and that affected only a couple of them. Running services weren't interrupted, latency remained, etc. which is what I'd expect for a single AZ failure.

CubsFan1060 5y ago

There will, however, be a lot of executives _talking_ about going multi cloud.

luhn 5y ago

To be fair though, from what I heard AZ outage caused an EC2 API brownout, so people couldn't launch new instances in the other AZs. That put a wrench in a lot of multi-AZ architectures. Not advocating for multi-cloud though...

dragonwriter 5y ago

> If it caused you downtime or a partial outage, it means you are not fully resilient to single az failures. Given the number of AWS global services that have dependencies on infra in US-EAST-1 (and, from the impacts of this and other past outages, seen vulnerable to single-AZ failures in US-EAST-1) that's...less avoidable for certain regions/AZs than one might naively expect. Most clouds seem to have at least some degree of this kind of vulnerability.

nonane 5y ago

> This affected a single az in a single region. If it caused you downtime or a partial outage, it means you are not fully resilient to single az failures. This is not true. Amazon is not being upfront about what happened here. It was simply not a single AZ failure. Our us-east-1 ELB load balancers were hosed and were unable to direct traffic to other AZs - they simply stopped working an were dropping traffic. We tried creating load balancers in different AZs and that didn't work either. How can you be resilient to single AZ failures if load balancers stop working region wide during a single AZ outage?

acdha 5y ago
Did your TAM go into any details on that? Over a couple hundred load-balancers, the only issue we had was taking longer to register new instances and that affected only a couple of them. Running services weren't interrupted, latency remained, etc. which is what I'd expect for a single AZ failure.