5 ms·

There was definitely an issue. Around 25% of our servers in one availability zone of us-east-1 fell off the network for 15 minutes or so, starting around 13:47

There was definitely an issue. Around 25% of our servers in one availability zone of us-east-1 fell off the network for 15 minutes or so, starting around 13:47 GMT. They're back now. During this time period, we were also unable to access the console (500 errors).

edgesrazor 9y ago
Which AZ?
- snewman 9y ago
  "C", but that's meaningless because AWS scrambles the zone names for each account. (Presumably to prevent everyone from putting all their servers in "A".)
  - edgesrazor 9y ago
    Interesting - I had no idea they did that. Well, I guess checking my instances in C won't be any help to you. Sorry!
    - mewm 9y ago
      FYI, it's elaborated here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-re...
  - 1001101 9y ago
    Haha - I didn't know that. Makes sense. I've got a dropdown in one of my Cloudformation scripts for AZs, and every time I get to it, I spend way more time thinking about it than I should. You've saved me some time.
- deleted 9y ago
  [deleted]
Bartweiss 9y ago
Word from Amazon is "elevated packet loss", but I saw pretty much the same. Elevated to 100%, I guess.
tapoxi 9y ago
Why does it always seem to be us-east?
- sjg 9y ago
  I'd guess it's the oldest data centre so more prone to failures, but that's pure speculation.
- avenoir 9y ago
  Isn't it the cheapest region to use? Probably sees more use because of it.
  - Johnny555 9y ago
    us-west-2 (Oregon) and us-east-2 (Ohio) are the same price as us-east-1 (Virginia). At least that's true for most resources, I didn't check the full price list. I don't know about Ohio since I don't use it, but we've had far fewer problems in us-west-2 than in us-east-1
    - endersshadow 9y ago
      If I have a single-region service, I always put it in us-west-2. It's super reliable, and gets updates after us-east-1 and us-west-1, which means all the kinks are out before they hit us-west-2. On days like today, I without fail get a message from my friend who works at a shop where everything is in us-east-1 (multi-AZs) about how much he hates me for avoiding east like the plague.
- unquietcode 9y ago
  East 1 is indeed the oldest and has the most non-standard configuration of the bunch (besides China, of course). I definitely would recommend east 2 or west 2 for any new deployments.

edgesrazor 9y ago

Which AZ?

snewman 9y ago
"C", but that's meaningless because AWS scrambles the zone names for each account. (Presumably to prevent everyone from putting all their servers in "A".)
- edgesrazor 9y ago
  Interesting - I had no idea they did that. Well, I guess checking my instances in C won't be any help to you. Sorry!
  - mewm 9y ago
    FYI, it's elaborated here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-re...
- 1001101 9y ago
  Haha - I didn't know that. Makes sense. I've got a dropdown in one of my Cloudformation scripts for AZs, and every time I get to it, I spend way more time thinking about it than I should. You've saved me some time.
deleted 9y ago
[deleted]

Bartweiss 9y ago

Word from Amazon is "elevated packet loss", but I saw pretty much the same. Elevated to 100%, I guess.

tapoxi 9y ago

Why does it always seem to be us-east?

sjg 9y ago
I'd guess it's the oldest data centre so more prone to failures, but that's pure speculation.
avenoir 9y ago
Isn't it the cheapest region to use? Probably sees more use because of it.
- Johnny555 9y ago
  us-west-2 (Oregon) and us-east-2 (Ohio) are the same price as us-east-1 (Virginia). At least that's true for most resources, I didn't check the full price list. I don't know about Ohio since I don't use it, but we've had far fewer problems in us-west-2 than in us-east-1
  - endersshadow 9y ago
    If I have a single-region service, I always put it in us-west-2. It's super reliable, and gets updates after us-east-1 and us-west-1, which means all the kinks are out before they hit us-west-2. On days like today, I without fail get a message from my friend who works at a shop where everything is in us-east-1 (multi-AZs) about how much he hates me for avoiding east like the plague.
unquietcode 9y ago
East 1 is indeed the oldest and has the most non-standard configuration of the bunch (besides China, of course). I definitely would recommend east 2 or west 2 for any new deployments.