3 ms·

Some AWS services can't span across multiple availability zones like EMR.

• skywhopper 7y ago
  Yes, and also for many managed services (like Kinesis or Lambda) you don't have any control over which AZs are in use, and so you have no choice but to wait for the AWS teams to address any problems caused on their own backends. Regardless, it's important to be aware of the risks of whatever tool you're using. It's unrealistic to expect any provider to be able to avoid failure entirely. You have to be aware of possible failure scenarios and have your own plan to address them.
• eropple 7y ago
  EMR seems kind of legacy from AWS's perspective these days. Probably very profitable, but they're seeming to try to push you towards things like Glue and Athena.
• bshacklett 7y ago
  Even in these cases, however, there are usually architectural methods for mitigating this kind of failure. In the case of EMR, using transient clusters with state stored in durable file stores such as EMRFS and vanilla S3 is one such option. It is inevitable that systems will fail. The best the industry can do is work to reduce the number of failures and understand failure modes well so that they can be planned for. AWS does a very good job of this in my experience. Regardless of whether applications are hosted in the cloud, on premises, co-located or in some hybrid configuration, it's important to design for that inevitable failure and keep business decision makers in the loop while doing so. Understanding requirements around RPO and RTO are extremely important in developing an architecture which meets the needs of the business, yet is still cost effective.