3 ms·

Don't use AWS, or at least don't put all your eggs into the AWS basket. AWS has gotten to the service and infrastructure footprint it has today by death marchi

Don't use AWS, or at least don't put all your eggs into the AWS basket. AWS has gotten to the service and infrastructure footprint it has today by death marching mediocre talent to ship untested and brittle things. This is true all the way down into the physical infrastructure, which often breaks and causes regional havoc. Great examples from the last 6 months include the 40 minute outage in US-WEST-2 in September, or the several hour event in US-EAST-1 during the East Coast wind storms. Bubble gum and baling wire. They're trying to stabilize things, but leadership and sales are often committing to large spend customers and financing new features with technical debt (when current technical debt is not being serviced). Those at the top are not really concerned with running a solid or innovative platform as much as just pure growth and revenue.

pavel_lishin 9y ago
> Great examples from the last 6 months include the 40 minute outage in US-WEST-2 in September, or the several hour event in US-EAST-1 during the East Coast wind storms. Bubble gum and baling wire. Were other non-Amazon companies impacted directly by the storms? I have no idea where Microsoft and Google cloud server farms live, but it's a bit of a stretch to say that a major weather event causing downtime is due to "bubble gum and bailing wire". You could be right - I don't know anything about their setup - but "throwaway985325" isn't exactly a trustworthy source.
- throwaway985325 9y ago
  I work on an internal service, and my weekly metrics reflect the health of a lot of services. It's more often than not that those metrics look like a printout from a seismograph sitting on an active fault. I don't know if other cloud providers were impacted, but I'm willing to bet they were not impacted intermittently for 6 hours.
hacknat 9y ago
Umm, GCE literally had a total global failure in 2016, AWS has never suffered such an event.
- toomuchtodo 9y ago
  AWS has suffered region wide outages of services that go unreported on their service status page for hours.
  - hacknat 9y ago
    Region wide is not the same thing as all regions at once.
ex-aws-now-goog 9y ago
I worked at AWS before moving to Google. AWS is not perfect (their oncall story being a good example) but they have _a lot_ of very talented engineers and are _really_ good at doing ops (and cloud systems are necessarily sprawling, complex beasts that require a lot of operational support). Perhaps your corner of AWS isn't as nice as the one I worked in but to generalize that to AWS management "death marching mediocre talent to ship untested and brittle things" is a very inaccurate assessment of AWS and its customers' perceptions of their products.

pavel_lishin 9y ago

> Great examples from the last 6 months include the 40 minute outage in US-WEST-2 in September, or the several hour event in US-EAST-1 during the East Coast wind storms. Bubble gum and baling wire. Were other non-Amazon companies impacted directly by the storms? I have no idea where Microsoft and Google cloud server farms live, but it's a bit of a stretch to say that a major weather event causing downtime is due to "bubble gum and bailing wire". You could be right - I don't know anything about their setup - but "throwaway985325" isn't exactly a trustworthy source.

throwaway985325 9y ago
I work on an internal service, and my weekly metrics reflect the health of a lot of services. It's more often than not that those metrics look like a printout from a seismograph sitting on an active fault. I don't know if other cloud providers were impacted, but I'm willing to bet they were not impacted intermittently for 6 hours.

hacknat 9y ago

Umm, GCE literally had a total global failure in 2016, AWS has never suffered such an event.

toomuchtodo 9y ago
AWS has suffered region wide outages of services that go unreported on their service status page for hours.
- hacknat 9y ago
  Region wide is not the same thing as all regions at once.

ex-aws-now-goog 9y ago

I worked at AWS before moving to Google. AWS is not perfect (their oncall story being a good example) but they have _a lot_ of very talented engineers and are _really_ good at doing ops (and cloud systems are necessarily sprawling, complex beasts that require a lot of operational support). Perhaps your corner of AWS isn't as nice as the one I worked in but to generalize that to AWS management "death marching mediocre talent to ship untested and brittle things" is a very inaccurate assessment of AWS and its customers' perceptions of their products.