8 ms·

It's not just AWS - check the down reports:https://downdetector.com/ https://downdetector.com/ Cloudflare having some significant issues as well on certain dom

It's not just AWS - check the down reports:https://downdetector.com/ https://downdetector.com/ Cloudflare having some significant issues as well on certain domains.

the_pwner224 5y ago
HN was also (briefly) down around that same time (roughly 1 hour ago from now).
chasd00 5y ago
can confirm i have multiple salesforce instances down.
nerdjon 5y ago
The list of affected services is a bit all over the place, especially since I highly doubt Xbox Live or Halo is running on AWS.
- iamricks 5y ago
  lol imagine if azure was just AWS in the backend
  - nerdjon 5y ago
    Is it bad that I can almost see that being a quick and dirty MVP to get out the door while you built your own cloud solution? Raises serious migration and cost issues, but... would be interesting.
    - vidarh 5y ago
      I think for some targeted things there might well be "value added" services you could offer to transparently wrap AWS. E.g. a "write-through" S3 wrapper was something I was actually looking at because some clients when I was contracting were very reluctant to trust anything but AWS for durability but at the same time AWS bandwidth costs were so extortionate that renting our own servers from somewhere like Hetzner and then proxying writes both to a local disk and to S3 and serve up from local disk with a fallback to pull a fresh copy from S3 if missing broke even at a quite small number of terabytes transferred each month. The nice part about something like that is that properly wrapped you can change your durable storage as needed, and can easily even selectively pick "cheaper but less trusted" options for less critical data. It also allows you to leverage AWS features to ride closer to the wire. E.g. to take another example than storage, I've used this to cut the cost of managed hosting by being to spill over onto EC2 instances in the past, allowing you to run at much higher utilisation rate than what you can safely on managed / colo / on-prem servers alone - as a result, ironically the ability to spill over onto EC2 makes EC2 far less competitive in terms of cost to actually run stuff on most of the time.
    - mr_toad 5y ago
      > a quick and dirty MVP to get out the door while you built your own cloud solution? Seemed to work for Dropbox.
- jon-wood 5y ago
  Down Detector doesn't really detect anything other than people saying "Is [service X] down?" on Twitter, which does mean that Xbox Live appears to be permanently offline if you believe them because the typical user for Xbox Live will declare anything from tripping over their ethernet cable to a tornado levelling their house preventing a connection to mean Xbox Live is down.
  - subandi 5y ago
    If that were true, the line should be flat-ish, but it and playstation's show the same extreme spike at the same time as aws etc.
  - Uehreka 5y ago
    It’s still useful if you remove units from the graph and treat it as a sparkline. If there are reliably ~100 Xbox Live complaints on Twitter per hour, then suddenly there are 3000, that’s an outage.
- s_fischer 5y ago
  For the core services? Definitely. But do we really know that some 3rd party API which doesn't fail gracefully isn't causing this?
yabones 5y ago
It's possible people are reporting the issue as CloudFlare because that's whose error page they see when a box on EC2 is unreachable.
- deleted 5y ago
  [deleted]
ren_engineer 5y ago
some sort of widescale attack would be the only explanation right?
PragmaticPulp 5y ago
DownDetector is showing everything down during that period, including Google. I suspect DownDetector itself suffered some outages during this period, which it shows as outages of every service it monitors.
- zaltekk 5y ago
  That's not how DownDetector works. It just relies on reports from users. The real failure case is users not understanding why they can't access whatever end service. Maybe they blame that service, maybe they blame their ISP, maybe they blame something else.
forgotpwd16 5y ago
This looks weird. At the same time all those services had a spike in outage reports.
jgrahamc 5y ago
No, we are not. But customers who use AWS are having trouble.
- rexreed 5y ago
  Thanks for clarifying! Things seem to have settled down.
buryat 5y ago
downdetector.com uses users complaints so it’s unreliable as people can blame anything

It's not just AWS - check the down reports:https://downdetector.com/ https://downdetector.com/ Cloudflare having some significant issues as well on certain domains.

the_pwner224 5y ago

HN was also (briefly) down around that same time (roughly 1 hour ago from now).

chasd00 5y ago

can confirm i have multiple salesforce instances down.

nerdjon 5y ago

The list of affected services is a bit all over the place, especially since I highly doubt Xbox Live or Halo is running on AWS.

iamricks 5y ago
lol imagine if azure was just AWS in the backend
- nerdjon 5y ago
  Is it bad that I can almost see that being a quick and dirty MVP to get out the door while you built your own cloud solution? Raises serious migration and cost issues, but... would be interesting.
  - vidarh 5y ago
    I think for some targeted things there might well be "value added" services you could offer to transparently wrap AWS. E.g. a "write-through" S3 wrapper was something I was actually looking at because some clients when I was contracting were very reluctant to trust anything but AWS for durability but at the same time AWS bandwidth costs were so extortionate that renting our own servers from somewhere like Hetzner and then proxying writes both to a local disk and to S3 and serve up from local disk with a fallback to pull a fresh copy from S3 if missing broke even at a quite small number of terabytes transferred each month. The nice part about something like that is that properly wrapped you can change your durable storage as needed, and can easily even selectively pick "cheaper but less trusted" options for less critical data. It also allows you to leverage AWS features to ride closer to the wire. E.g. to take another example than storage, I've used this to cut the cost of managed hosting by being to spill over onto EC2 instances in the past, allowing you to run at much higher utilisation rate than what you can safely on managed / colo / on-prem servers alone - as a result, ironically the ability to spill over onto EC2 makes EC2 far less competitive in terms of cost to actually run stuff on most of the time.
  - mr_toad 5y ago
    > a quick and dirty MVP to get out the door while you built your own cloud solution? Seemed to work for Dropbox.
jon-wood 5y ago
Down Detector doesn't really detect anything other than people saying "Is [service X] down?" on Twitter, which does mean that Xbox Live appears to be permanently offline if you believe them because the typical user for Xbox Live will declare anything from tripping over their ethernet cable to a tornado levelling their house preventing a connection to mean Xbox Live is down.
- subandi 5y ago
  If that were true, the line should be flat-ish, but it and playstation's show the same extreme spike at the same time as aws etc.
- Uehreka 5y ago
  It’s still useful if you remove units from the graph and treat it as a sparkline. If there are reliably ~100 Xbox Live complaints on Twitter per hour, then suddenly there are 3000, that’s an outage.
s_fischer 5y ago
For the core services? Definitely. But do we really know that some 3rd party API which doesn't fail gracefully isn't causing this?

yabones 5y ago

It's possible people are reporting the issue as CloudFlare because that's whose error page they see when a box on EC2 is unreachable.

deleted 5y ago
[deleted]

ren_engineer 5y ago

some sort of widescale attack would be the only explanation right?

PragmaticPulp 5y ago

DownDetector is showing everything down during that period, including Google. I suspect DownDetector itself suffered some outages during this period, which it shows as outages of every service it monitors.

zaltekk 5y ago
That's not how DownDetector works. It just relies on reports from users. The real failure case is users not understanding why they can't access whatever end service. Maybe they blame that service, maybe they blame their ISP, maybe they blame something else.

forgotpwd16 5y ago

This looks weird. At the same time all those services had a spike in outage reports.

jgrahamc 5y ago

No, we are not. But customers who use AWS are having trouble.

rexreed 5y ago
Thanks for clarifying! Things seem to have settled down.

buryat 5y ago

downdetector.com uses users complaints so it’s unreliable as people can blame anything