4 ms·

Amazon is down ... implications for AWS?

67 points by redorb 18y ago

redorb 18y ago
I'm getting a HTTP Status Code: HTTP/1.1 503 Service Unavailable from here http://web-sniffer.net/?url=www.amazon.com&submit=Submit&http=1.1&gzip=yes&type=GET&uak=0 http://web-sniffer.net/?url=www.amazon.com&submit=Submit... and of course from my own PC
- dougp 18y ago
  Something was going on yesterday I was getting error screens all over the site.
byrneseyeview 18y ago
It's back up.
- mynameishere 18y ago
  It's back down.
  - davidw 18y ago
    Weird: > You have been denied access to this feature because we believe you violated the terms, conditions, rules, guidelines or policies of our site in the past. If you believe we have taken this action in error, you may contact us at ad-help-us@amazon.com.
    - me2i81 18y ago
      I got that too.
markbao 18y ago
http://downforeveryoneorjustme.com/amazon.com http://downforeveryoneorjustme.com/amazon.com Still down. https://amazon.com https://amazon.com still works.
- a-priori 18y ago
  http://amazon.ca http://amazon.ca also works. It's only the .com that's down.
- jcl 18y ago
  Odd... Amazon.com works fine for me, despite the downforeveryone site saying it doesn't.
nickb 18y ago
AWS services are all up. Just checked.
- toffer 18y ago
  AWS status updates here: http://status.aws.amazon.com/ http://status.aws.amazon.com/
  - redorb 18y ago
    on that page it says "we are having issues with aws.amazon.com" so the domain itself (including subdomains) are in trouble... - Im predicting the https: will break soon (Currently working) as people find out that it is working ... TC just said its been down for over an hour ...huge, huge money...
- prakash 18y ago
  .com is back up now.
astine 18y ago
S3 is still working. I don't know what's up with Amazon, but it seems to be just their storefront.
garyrichardson 18y ago
My EC2 instances are all up. It's specific to Amazon's main site.
lpgauth 18y ago
Http/1.1 Service Unavailable
deleted 18y ago
[deleted]
asenchi 18y ago
Amazing how long it's been down for, I would figure this to be something that they'd work to resolve nearly immediately.
- Thorsten 18y ago
  yeah, its being worked on as we speak.
  - redorb 18y ago
    "he strongest day of holiday sales for Amazon was December 10th, on which customers ordered 62.5 items pers second for a total of 5.4 million items. On its peak shipping day, the retailer shipped products internationally to more than 200 countries, moving more than 3.9 million shipments." from: http://www.gamasutra.com/php-bin/news_index.php?story=16762 http://www.gamasutra.com/php-bin/news_index.php?story=16762 - although its from the same spam article that said they sold 17 wiis a second (when they were in stock) in 2007 ... I would say every 10 minutes cost ... 78mm for a quarter in 07, (4 months = 120 day) 78mm/120 = 650k/day = 27k/hour = $450/sec ... they are losing big would love to hear the story behind it.
- icey 18y ago
  If there is ever a post-mortem for this outage, I really hope it gets linked here. I'm interested to see what it could have been.
deleted 18y ago
[deleted]
prakash 18y ago
I went to a recent AWS talk by one of their evangelists, where they mentioned that AWS is separate from amazon.com, in terms of the hardware used.
- ckinnan 18y ago
  It's time to put Amazon on AWS then?!?
comforteagle 18y ago
My ec2 instances are up.
prakash 18y ago
It would be interesting to know what kind of failover mechanisms they follow and the reasons for the outage, and how much money they lost due to this?
- asenchi 18y ago
  Well according to returns last year, about $503/revenue per second. $15.9 billion last year.
  - deleted 18y ago
    [deleted]
  - dangoldin 18y ago
    I'm not it's accurate to give a $ revenue for second. For me personally, if I wanted to buy something and the site didn't work. I would just come back in an hour; not sure how others would change their behavior.
    - asenchi 18y ago
      Oh certainly, that was just a figure. I suspect people will just come back. Still a fascinating number (btw, pulled from Reddit.com, forgot to credit originally).
      - mynameishere 18y ago
        I wouldn't trust that. http://www.reddit.com/info/6mbec/comments/c049fo7 http://www.reddit.com/info/6mbec/comments/c049fo7
    - DenisM 18y ago
      Sometimes I come to buy.com and it's closed, so I turn away and go to amazon. Eventually, I stopped going to buy.com altogether.
    - jfornear 18y ago
      Unrelated, but that is exactly why those "National Don't Buy Gas Days" on Facebook don't do a thing to affect gas prices.
mhartl 18y ago
I get "Http/1.1 Service Unavailable", but if I refresh the page in my browser it comes right up. Click on a link? "Http/1.1 Service Unavailable". Weird.
gibsonf1 18y ago
Our AWS web app (EC2 + S3) has been running fine without interruption or delay.
icey 18y ago
To me this smells of a rollout gone terribly, terribly wrong.
breck 18y ago
$14.8 billion in revenue ~= $28,000 per minute So an hour of downtime costs ~ $1.69 million Very rough estimates, but could be close.
- maximilian 18y ago
  Assuming you average their revenue. But i'll betcha they make a lot of their money in spikes, so going down on a random friday evening, although costly, isn't as costly as 1.69 million. Also, what percentage of people who wanna buy something at the moment its down, won't come back later?
brk 18y ago
Works for me now. I wonder if the outage was related to adding the 3G iphone to their catalog?
- rbanffy 18y ago
  LOL. That and the addition of the bazillion servers to cope with the sudden traffic surge the 3G iPhone will cause.
auston 18y ago
My question is, will a new order that I place today be properly logged...
maximilian 18y ago
Can you imagine being the guy or team that f'ed this up? Think how terribly nervous/stressed you'd be for the hour or so until you fixed it!? An hour would blow by in minutes as they frantically hammered on the keyboard in a terminal to see what got f'ed, just thinking that they are loosing your salary in revenue every minute or something awful. I can't imagine being an admin in that sorta situation.

67 points by redorb 18y ago

I'm getting a HTTP Status Code: HTTP/1.1 503 Service Unavailable from here http://web-sniffer.net/?url=www.amazon.com&submit=Submit&http=1.1&gzip=yes&type=GET&uak=0 http://web-sniffer.net/?url=www.amazon.com&submit=Submit... and of course from my own PC

dougp 18y ago
Something was going on yesterday I was getting error screens all over the site.

byrneseyeview 18y ago

It's back up.

mynameishere 18y ago
It's back down.
- davidw 18y ago
  Weird: > You have been denied access to this feature because we believe you violated the terms, conditions, rules, guidelines or policies of our site in the past. If you believe we have taken this action in error, you may contact us at ad-help-us@amazon.com.
  - me2i81 18y ago
    I got that too.

markbao 18y ago

http://downforeveryoneorjustme.com/amazon.com http://downforeveryoneorjustme.com/amazon.com Still down. https://amazon.com https://amazon.com still works.

a-priori 18y ago
http://amazon.ca http://amazon.ca also works. It's only the .com that's down.
jcl 18y ago
Odd... Amazon.com works fine for me, despite the downforeveryone site saying it doesn't.

nickb 18y ago

AWS services are all up. Just checked.

toffer 18y ago
AWS status updates here: http://status.aws.amazon.com/ http://status.aws.amazon.com/
- redorb 18y ago
  on that page it says "we are having issues with aws.amazon.com" so the domain itself (including subdomains) are in trouble... - Im predicting the https: will break soon (Currently working) as people find out that it is working ... TC just said its been down for over an hour ...huge, huge money...
prakash 18y ago
.com is back up now.

astine 18y ago

S3 is still working. I don't know what's up with Amazon, but it seems to be just their storefront.

garyrichardson 18y ago

My EC2 instances are all up. It's specific to Amazon's main site.

lpgauth 18y ago

Http/1.1 Service Unavailable

deleted 18y ago

[deleted]

asenchi 18y ago

Amazing how long it's been down for, I would figure this to be something that they'd work to resolve nearly immediately.

Thorsten 18y ago
yeah, its being worked on as we speak.
- redorb 18y ago
  "he strongest day of holiday sales for Amazon was December 10th, on which customers ordered 62.5 items pers second for a total of 5.4 million items. On its peak shipping day, the retailer shipped products internationally to more than 200 countries, moving more than 3.9 million shipments." from: http://www.gamasutra.com/php-bin/news_index.php?story=16762 http://www.gamasutra.com/php-bin/news_index.php?story=16762 - although its from the same spam article that said they sold 17 wiis a second (when they were in stock) in 2007 ... I would say every 10 minutes cost ... 78mm for a quarter in 07, (4 months = 120 day) 78mm/120 = 650k/day = 27k/hour = $450/sec ... they are losing big would love to hear the story behind it.
icey 18y ago
If there is ever a post-mortem for this outage, I really hope it gets linked here. I'm interested to see what it could have been.

deleted 18y ago

[deleted]

prakash 18y ago

I went to a recent AWS talk by one of their evangelists, where they mentioned that AWS is separate from amazon.com, in terms of the hardware used.

ckinnan 18y ago
It's time to put Amazon on AWS then?!?

comforteagle 18y ago

My ec2 instances are up.

prakash 18y ago

It would be interesting to know what kind of failover mechanisms they follow and the reasons for the outage, and how much money they lost due to this?

asenchi 18y ago
Well according to returns last year, about $503/revenue per second. $15.9 billion last year.
- deleted 18y ago
  [deleted]
- dangoldin 18y ago
  I'm not it's accurate to give a $ revenue for second. For me personally, if I wanted to buy something and the site didn't work. I would just come back in an hour; not sure how others would change their behavior.
  - asenchi 18y ago
    Oh certainly, that was just a figure. I suspect people will just come back. Still a fascinating number (btw, pulled from Reddit.com, forgot to credit originally).
    - mynameishere 18y ago
      I wouldn't trust that. http://www.reddit.com/info/6mbec/comments/c049fo7 http://www.reddit.com/info/6mbec/comments/c049fo7
  - DenisM 18y ago
    Sometimes I come to buy.com and it's closed, so I turn away and go to amazon. Eventually, I stopped going to buy.com altogether.
  - jfornear 18y ago
    Unrelated, but that is exactly why those "National Don't Buy Gas Days" on Facebook don't do a thing to affect gas prices.

mhartl 18y ago

I get "Http/1.1 Service Unavailable", but if I refresh the page in my browser it comes right up. Click on a link? "Http/1.1 Service Unavailable". Weird.

gibsonf1 18y ago

Our AWS web app (EC2 + S3) has been running fine without interruption or delay.

icey 18y ago

To me this smells of a rollout gone terribly, terribly wrong.

breck 18y ago

$14.8 billion in revenue ~= $28,000 per minute So an hour of downtime costs ~ $1.69 million Very rough estimates, but could be close.

maximilian 18y ago
Assuming you average their revenue. But i'll betcha they make a lot of their money in spikes, so going down on a random friday evening, although costly, isn't as costly as 1.69 million. Also, what percentage of people who wanna buy something at the moment its down, won't come back later?

brk 18y ago

Works for me now. I wonder if the outage was related to adding the 3G iphone to their catalog?

rbanffy 18y ago
LOL. That and the addition of the bazillion servers to cope with the sudden traffic surge the 3G iPhone will cause.

auston 18y ago

My question is, will a new order that I place today be properly logged...

maximilian 18y ago

Can you imagine being the guy or team that f'ed this up? Think how terribly nervous/stressed you'd be for the hour or so until you fixed it!? An hour would blow by in minutes as they frantically hammered on the keyboard in a terminal to see what got f'ed, just thinking that they are loosing your salary in revenue every minute or something awful. I can't imagine being an admin in that sorta situation.