4 ms·

One of our EC2 instances in us-east-1c is unavailable and stuck in "stopping" state after a force stop. Interestingly enough, EC2 instances in us-east-1b don't

One of our EC2 instances in us-east-1c is unavailable and stuck in "stopping" state after a force stop. Interestingly enough, EC2 instances in us-east-1b don't seem to be affected. The console is throwing errors from time to time. As usual no information on AWS status page.

chrishynes 5y ago
I had the same issue with unavailable, but on an instance in us-east-1b. Finally just got the force stop to go through a minute ago and it's now running and available again.
- mike-cardwell 5y ago
  Your us-east-1b may be the parents us-east-1c. The letters are randomised per AWS account so that instances are spread evenly and biases to certain letters don't lead to biases to certain zones.
  - chrishynes 5y ago
    Huh, that's interesting. Didn't know that, but makes sense.
    - thrtythreeforty 5y ago
      It's pretty cool. If I recall, they call it "shuffle sharding."
    - ciceryadam 5y ago
      You can check which availability zone is with: aws ec2 describe-availability-zones --region us-east-1
- throwaway984393 5y ago
  I'm not sure if we should say "AWS is down" if only us-east-1 is down. That region is more unstable than Marjorie Taylor Greene on a one-legged stool.
  - fivea 5y ago
    > I'm not sure if we should say "AWS is down" if only us-east-1 is down. The thing is, us-east-1 represents the whole AWS for the majority of us.
    - flatiron 5y ago
      Can you expand on that? What feature do you use in east 1 that isn’t everywhere else that it’s your whole implementation?
      - fivea 5y ago
        > Can you expand on that? What feature do you use in east 1 that isn’t everywhere else that it’s your whole implementation? Your question reads as a strawman. It matters nothing if EC2 is also available in Mumbai or Hong Kong if by default the whole world deploys everything and anything to us-east-1, and us-east-1 alone. https://www.reddit.com/r/aws/comments/nztxa5/why_useast1_region/ https://www.reddit.com/r/aws/comments/nztxa5/why_useast1_reg...
        throwaway984393 5y ago
        It's not a strawman. There's a huge difference between "AWS is down" and "customers don't know how to use AWS". For the people who use AWS correctly, they only had some degraded service, not downtime.
        fivea 5y ago
        > It's not a strawman. There's a huge difference between "AWS is down" and "customers don't know how to use AWS". Deploying a service to a single region is not, nor has it ever been, "customers don't know how to use AWS". If anything, cargo culting this belief in global deployments being necessary, specially with services that have at most a regional demand, is a telltale sign a customer has no idea about what he is doing and is just mindlessly wasting money and engineering effort in something no one needs. This blend of bad cargo cult advice sounds like a variant of microservices everywhere.
300bps 5y ago
The 1c part is meaningless. Those letters are randomized per customer to prevent letter biases from leading to more people in 1a for instance.
JshWright 5y ago
Instances stuck in the "stopping" state is pretty common, in my experience.
crescentfresh 5y ago
Was stuck on stopping in us-east-1b. Cannot start now.
crescentfresh 5y ago
The affected zone is use1-az4. Whatever that maps to (1a, 1b, 1c) is different per customer.
- benedikt 5y ago
  you can find out which zone is mapped to use1-az4 for your account with awscli: aws ec2 describe-availability-zones | jq -r '.AvailabilityZones[] | select(.ZoneId == "use1-az4") | .ZoneName'
  - mnordhoff 5y ago
    Or if you open the EC2 console (it's up this time!) and scroll down to the bottom. https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Home https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#...: (Edit: I hope I didn't sound sarcastic. I don't open random console pages and scroll all the way down to check for new features. Some people will have noticed, some won't.)
    - deleted 5y ago
      [deleted]

chrishynes 5y ago

I had the same issue with unavailable, but on an instance in us-east-1b. Finally just got the force stop to go through a minute ago and it's now running and available again.

mike-cardwell 5y ago
Your us-east-1b may be the parents us-east-1c. The letters are randomised per AWS account so that instances are spread evenly and biases to certain letters don't lead to biases to certain zones.
- chrishynes 5y ago
  Huh, that's interesting. Didn't know that, but makes sense.
  - thrtythreeforty 5y ago
    It's pretty cool. If I recall, they call it "shuffle sharding."
  - ciceryadam 5y ago
    You can check which availability zone is with: aws ec2 describe-availability-zones --region us-east-1
throwaway984393 5y ago
I'm not sure if we should say "AWS is down" if only us-east-1 is down. That region is more unstable than Marjorie Taylor Greene on a one-legged stool.
- fivea 5y ago
  > I'm not sure if we should say "AWS is down" if only us-east-1 is down. The thing is, us-east-1 represents the whole AWS for the majority of us.
  - flatiron 5y ago
    Can you expand on that? What feature do you use in east 1 that isn’t everywhere else that it’s your whole implementation?
    - fivea 5y ago
      > Can you expand on that? What feature do you use in east 1 that isn’t everywhere else that it’s your whole implementation? Your question reads as a strawman. It matters nothing if EC2 is also available in Mumbai or Hong Kong if by default the whole world deploys everything and anything to us-east-1, and us-east-1 alone. https://www.reddit.com/r/aws/comments/nztxa5/why_useast1_region/ https://www.reddit.com/r/aws/comments/nztxa5/why_useast1_reg...
      - throwaway984393 5y ago
        It's not a strawman. There's a huge difference between "AWS is down" and "customers don't know how to use AWS". For the people who use AWS correctly, they only had some degraded service, not downtime.
        fivea 5y ago
        > It's not a strawman. There's a huge difference between "AWS is down" and "customers don't know how to use AWS". Deploying a service to a single region is not, nor has it ever been, "customers don't know how to use AWS". If anything, cargo culting this belief in global deployments being necessary, specially with services that have at most a regional demand, is a telltale sign a customer has no idea about what he is doing and is just mindlessly wasting money and engineering effort in something no one needs. This blend of bad cargo cult advice sounds like a variant of microservices everywhere.

300bps 5y ago

The 1c part is meaningless. Those letters are randomized per customer to prevent letter biases from leading to more people in 1a for instance.

JshWright 5y ago

Instances stuck in the "stopping" state is pretty common, in my experience.

crescentfresh 5y ago

Was stuck on stopping in us-east-1b. Cannot start now.

crescentfresh 5y ago

The affected zone is use1-az4. Whatever that maps to (1a, 1b, 1c) is different per customer.

benedikt 5y ago
you can find out which zone is mapped to use1-az4 for your account with awscli: aws ec2 describe-availability-zones | jq -r '.AvailabilityZones[] | select(.ZoneId == "use1-az4") | .ZoneName'
- mnordhoff 5y ago
  Or if you open the EC2 console (it's up this time!) and scroll down to the bottom. https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#Home https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#...: (Edit: I hope I didn't sound sarcastic. I don't open random console pages and scroll all the way down to check for new features. Some people will have noticed, some won't.)
  - deleted 5y ago
    [deleted]