4 ms·

I'm curious as to why their fix was to host the Service Health Dashboard on more AWS regions. It seems like the responsible thing to do is to host it entirely o

I'm curious as to why their fix was to host the Service Health Dashboard on more AWS regions. It seems like the responsible thing to do is to host it entirely on a competitor's service. That way, it's very simple to know that the status page will work no matter what happens to you.

deleted 10y ago
[deleted]
ghayes 10y ago
Or have it pull from two sources, one local (S3) and one remote (GCE or whatever), and make a hard positive from either source signal "down." Otherwise the page would be down if just the remote source were down.
- ckozlowski 10y ago
  Precisely. "we have changed the SHD administration console to run across multiple AWS regions."
djsumdog 10y ago
You have to eat your own dogfood, if you're trying to sell a service product. That being said, a better solution would be to stand up a separate infrastructure just for status pages.
- ceejayoz 10y ago
  "Except for status pages" is a perfectly reasonable exception to "eat your own dogfood".
pmoriarty 10y ago
If they did host their status page on a competitor's service, then they'd be reliant on that service, which might backfire if the competitor's service goes down while Amazon's own systems stay up. What they really need is failover capability, which can fire up the status page on a competitor's service (or maybe on a completely separate disaster recovery site site owned by Amazon) in case Amazon's own services go down. I'm sure Amazon's architects and engineers are more than capable of designing and implementing such a robust system and recognizing its importance. So it puzzles me as to why it wasn't done.
- angry-hacker 10y ago
  You can use several hosts, have two subdomains so if one is not red responding, engineers and managers know there are two status pages. Heck, have two different domains for them as well if there are dns issues. Amzstatus1.com and 2. Not dependent on Amazon domain anymore either.
chiph 10y ago
Or to host it on a static .html page that gets rewritten every 60 seconds or so by an external process, running on physical servers. Minimal stack, so minimal attack surface.
- dsl 10y ago
  That is effectively what happens, they just stupidly backed that file from S3. http://status.aws.amazon.com/data.json http://status.aws.amazon.com/data.json

deleted 10y ago

[deleted]

ghayes 10y ago

Or have it pull from two sources, one local (S3) and one remote (GCE or whatever), and make a hard positive from either source signal "down." Otherwise the page would be down if just the remote source were down.

ckozlowski 10y ago
Precisely. "we have changed the SHD administration console to run across multiple AWS regions."

djsumdog 10y ago

You have to eat your own dogfood, if you're trying to sell a service product. That being said, a better solution would be to stand up a separate infrastructure just for status pages.

ceejayoz 10y ago
"Except for status pages" is a perfectly reasonable exception to "eat your own dogfood".

pmoriarty 10y ago

If they did host their status page on a competitor's service, then they'd be reliant on that service, which might backfire if the competitor's service goes down while Amazon's own systems stay up. What they really need is failover capability, which can fire up the status page on a competitor's service (or maybe on a completely separate disaster recovery site site owned by Amazon) in case Amazon's own services go down. I'm sure Amazon's architects and engineers are more than capable of designing and implementing such a robust system and recognizing its importance. So it puzzles me as to why it wasn't done.

angry-hacker 10y ago
You can use several hosts, have two subdomains so if one is not red responding, engineers and managers know there are two status pages. Heck, have two different domains for them as well if there are dns issues. Amzstatus1.com and 2. Not dependent on Amazon domain anymore either.

chiph 10y ago

Or to host it on a static .html page that gets rewritten every 60 seconds or so by an external process, running on physical servers. Minimal stack, so minimal attack surface.

dsl 10y ago
That is effectively what happens, they just stupidly backed that file from S3. http://status.aws.amazon.com/data.json http://status.aws.amazon.com/data.json