4 ms·

Naive question: don't people who care about resiliency have their services in more than one datacenter? or datacenter failure is considered such a rare event th

Naive question: don't people who care about resiliency have their services in more than one datacenter? or datacenter failure is considered such a rare event that's it's not worth the cost/trouble of using more?

vhgyu75e6u 4y ago
AWS works with multiple availability zones (AZ) per region, some products by default deploy in several ones at the same time, while others leave it up to you.
vineyardmike 4y ago
Some people care about it but not enough to justify the added downsides - multi-data center is expensive (you pay per data center) and it’s complex (data sharding/duplication/sync). If you’re Amazon where every second is millions of $ in transactions you care more than StartUp that has 1 request per minute. Even if you accept the risk, you still care when your DC goes down. Also, a large chunk of AWS is managed from a single data center so if that one goes down you may still have issues with your service in another data center.
jedberg 4y ago
AWS makes it pretty easy to operate in multiple AZs within a region (each AZ is considered a separate datacenter but in real life each AZ is multiple datacenters that are really close to each other). That being said, there is still an added cost and complexity to operate in multiple AZs, because you have to synchronize data across the AZs. Also you have to have enough reserved instances to move into when you lose an AZ, because if you're running lean and each zone is serving 33% of your traffic, suddenly the two that are left need to serve 50% each. The bigger companies with overhead reservations will get all the instances before you can launch any on demand during an AZ failure.
- Tehnix 4y ago
  > each AZ is considered a separate datacenter but in real life each AZ is multiple datacenters that are really close to each other For AWS specifically, I’m fairly certain they maintain a minimum distance and are much more strict on requirements to be on different grids etc than other Cloud providers. A few years ago they were calling out Azure and Google Cloud on exactly what you describe (having data centers essentially on the same street almost).
  - dodobirdlord 4y ago
    I think you may have slightly misread. I think what’s being said is that a single logical AZ may actually be multiple physical datacenters in close proximity.
  - jedberg 4y ago
    A single AZ may have neighboring datacenters, but they are very strict on having datacenters for different AZs be at least 100km apart and on different flood plains and power grids.
    - throwbigdata 4y ago
      100km? Oh really?
      - throwbigdata 4y ago
        https://docs.aws.amazon.com/sap/latest/general/arch-guide-architecture-guidelines-and-decisions.html https://docs.aws.amazon.com/sap/latest/general/arch-guide-ar... Each Availability Zone can be multiple data centers.At full scale, it can contain hundreds of thousands of servers. They are fully isolated partitions of the AWS global infrastructure. With its own powerful infrastructure, an Availability Zone is physically separated from any other zones. There is a distance of several kilometers, although all are within 100 km (60 miles of each other).
        eurleif 4y ago
        So at most 100km, not at least 100km.
    - laluser 4y ago
      This should be at most 100km. Range is in 60km-100km range typically.
- mediascreen 4y ago
  At least in eu-north-1 the three AZs are located in different towns, about 50 km apart (Västerås, Eskilstuna and Katrineholm).
deleted 4y ago
[deleted]
jugg1es 4y ago
AWS makes it trivially easy to distribute across more than one datacenter... The only time that outages make the news is when they all fail in a region.
jamesfinlayson 4y ago
I'd consider using it, but the biggest roadblock for me is that I work in a regulated industry in Australia, and until AWS finishes their Melbourne region (next year maybe?) I'm stuck in one region because all private data needs to stay in Australia. Also, I think a lot, but not all of the services I use work okay with multiple regions. On top of that, I was looking at the documentation for KMS keys yesterday, and a KMS key can be multiregion, but if you don't create it as multiregion from the start, you can't update the multiregion attribute. So you need to create a new KMS key and update everything to use the new multiregion key.

vhgyu75e6u 4y ago

AWS works with multiple availability zones (AZ) per region, some products by default deploy in several ones at the same time, while others leave it up to you.

vineyardmike 4y ago

Some people care about it but not enough to justify the added downsides - multi-data center is expensive (you pay per data center) and it’s complex (data sharding/duplication/sync). If you’re Amazon where every second is millions of $ in transactions you care more than StartUp that has 1 request per minute. Even if you accept the risk, you still care when your DC goes down. Also, a large chunk of AWS is managed from a single data center so if that one goes down you may still have issues with your service in another data center.

jedberg 4y ago

AWS makes it pretty easy to operate in multiple AZs within a region (each AZ is considered a separate datacenter but in real life each AZ is multiple datacenters that are really close to each other). That being said, there is still an added cost and complexity to operate in multiple AZs, because you have to synchronize data across the AZs. Also you have to have enough reserved instances to move into when you lose an AZ, because if you're running lean and each zone is serving 33% of your traffic, suddenly the two that are left need to serve 50% each. The bigger companies with overhead reservations will get all the instances before you can launch any on demand during an AZ failure.

Tehnix 4y ago
> each AZ is considered a separate datacenter but in real life each AZ is multiple datacenters that are really close to each other For AWS specifically, I’m fairly certain they maintain a minimum distance and are much more strict on requirements to be on different grids etc than other Cloud providers. A few years ago they were calling out Azure and Google Cloud on exactly what you describe (having data centers essentially on the same street almost).
- dodobirdlord 4y ago
  I think you may have slightly misread. I think what’s being said is that a single logical AZ may actually be multiple physical datacenters in close proximity.
- jedberg 4y ago
  A single AZ may have neighboring datacenters, but they are very strict on having datacenters for different AZs be at least 100km apart and on different flood plains and power grids.
  - throwbigdata 4y ago
    100km? Oh really?
    - throwbigdata 4y ago
      https://docs.aws.amazon.com/sap/latest/general/arch-guide-architecture-guidelines-and-decisions.html https://docs.aws.amazon.com/sap/latest/general/arch-guide-ar... Each Availability Zone can be multiple data centers.At full scale, it can contain hundreds of thousands of servers. They are fully isolated partitions of the AWS global infrastructure. With its own powerful infrastructure, an Availability Zone is physically separated from any other zones. There is a distance of several kilometers, although all are within 100 km (60 miles of each other).
      - eurleif 4y ago
        So at most 100km, not at least 100km.
  - laluser 4y ago
    This should be at most 100km. Range is in 60km-100km range typically.
mediascreen 4y ago
At least in eu-north-1 the three AZs are located in different towns, about 50 km apart (Västerås, Eskilstuna and Katrineholm).

deleted 4y ago

[deleted]

jugg1es 4y ago

AWS makes it trivially easy to distribute across more than one datacenter... The only time that outages make the news is when they all fail in a region.

jamesfinlayson 4y ago

I'd consider using it, but the biggest roadblock for me is that I work in a regulated industry in Australia, and until AWS finishes their Melbourne region (next year maybe?) I'm stuck in one region because all private data needs to stay in Australia. Also, I think a lot, but not all of the services I use work okay with multiple regions. On top of that, I was looking at the documentation for KMS keys yesterday, and a KMS key can be multiregion, but if you don't create it as multiregion from the start, you can't update the multiregion attribute. So you need to create a new KMS key and update everything to use the new multiregion key.