7 ms·

This outage is affecting Discord. https://status.discordapp.com/ https://status.discordapp.com/

by teovall 7y ago

bpye 7y ago
Discord seems to be preventing new connections.
soup10 7y ago
...
- RugnirViking 7y ago
  And why would that be? Are they a particular problem customer for google?
  - soup10 7y ago
    ...
    - kzrdude 7y ago
      and discord's 3rd party client policy is not doing them any favours
    - GhettoMaestro 7y ago
      Annoyed at being on call: How dare they do their expected responsibilities in exchange for pay. The nerve of them.
      - p1necone 7y ago
        It's very possible they're not "officially" on call and don't get any extra pay for having to work in the weekend. Just because that's the current expectation of a lot of salaried workers doesn't mean it's right.
    - npo9 7y ago
      Discord has millions of users. Less than 1000 people are probably bothered at Google to fix the problem. My sympathies lie with Discord users. I’m sure it’s ruining a lot more weekend plans with them. I’ve both a discord user and someone that has spent years in an on-call rotation that included nights and weekends.
      - deleted 7y ago
        [deleted]
- ecf 7y ago
  <> [Removed reply since the comment I was replying to was removed.]
  - npo9 7y ago
    This type of outage happens with all of the cloud providers from time to time.
    - scarface74 7y ago
      Yes. Technically it happens to all companies but politically it’s a lot different. If you tell your customers that you have an outage because AWS is down, they will shrug and say, things happen. You are in the same boat with a lot of other people, you will no more get blamed for AWS being down than if the whole city lost power. No one ever got fired for choosing AWS. If you’re on Azure, you might get a funny look from people who only know about AWS, but since it is Microsoft Azure you will get a pass. But people will judge you for choosing GCP. Any little thing that goes wrong, they will be questioning why you didn’t choose AWS.
      - xerox13ster 7y ago
        Not gonna lie, one of the first things I said to my friends is why is discord using Google, it should be AWS.
        Fellshard 7y ago
        The categories of errors - and Google's ability to recognize and respond to those errors in rapid form - has always seemed extraordinarily slow and reticent compared to AWS. It's one of the concerns I'd have when recommending it.
VolatileVoid 7y ago
Is it just me or are Discord's status updates legendarily bad to the point of being entirely useless? As a user of the discord app, don't tell me the "API is having issues." Just tell me, "Discord is currently down. We are working on it and apologise for the disruption." Everything else is unnecessary line noise.
- p1necone 7y ago
  Mentioning the cause of the issue seems better than just "shits broke yo" to me.
  - VolatileVoid 7y ago
    Really? What in their status update read to you as "the problem is on our side, not yours"? Understand that I'm a programmer and couldn't figure that out.
    - Operyl 7y ago
      Because if it was a problem on your end, it wouldn't be listed on their status page....?
    - aquova 7y ago
      "Google's Engineering team continues to work on mitigating the issue" I think that's pretty clear where the issue lies
- Fabricio20 7y ago
  The Discordapp status page is written to by developers, and as a developer I really like the insights as to what is going on. As a user, I expect you to just look at it and see it's "Having Issues, We are working on it".
scottmf 7y ago
We use Discord as an interface for moderating our app and monitoring spam/reported content so this kinda sucks. Not super important though.
seriesf 7y ago
Being taken down by slow IO in a single cloud zone seems to implicated their overall architecture.
- jrockway 7y ago
  Looking at their job postings, they use Cassandra and Elasticsearch, so I am somewhat surprised that they didn't survive this outage. (I have not ever run Cassandra at scale, but their website says in the first paragraph that it's designed to handle regional failures.) If I were on GCP and had paying customers, I would use Cloud Spanner. It's expensive, but it's a good piece of technology.
  - sieabahlpark 7y ago
    Most startup companies worry about building out features than building out reliable infrastructure. Almost always an after thought
  - seriesf 7y ago
    The gift of Spanner is it’s always slow, so if you start there you will naturally gain experience in either hiding latency or relaxing consistency requirements, which are both good skills.
    - vl 7y ago
      Spanner is pretty fast, but as with any database you have to use it correctly to get the speed. Since it’s a non-standard database, it requires non-standard tricks and skills.
      - jrockway 7y ago
        No, that's about right. You get one write per second per entity or something like that and that's as fast as it goes. The key is to choose your entity wiseley. I wrote an app inside Google that did on the order of 10,000 writes per second and it was no problem; each entity group only showed up once every minute or so.

This outage is affecting Discord. https://status.discordapp.com/ https://status.discordapp.com/

bpye 7y ago

Discord seems to be preventing new connections.

soup10 7y ago

...

RugnirViking 7y ago
And why would that be? Are they a particular problem customer for google?
- soup10 7y ago
  ...
  - kzrdude 7y ago
    and discord's 3rd party client policy is not doing them any favours
  - GhettoMaestro 7y ago
    Annoyed at being on call: How dare they do their expected responsibilities in exchange for pay. The nerve of them.
    - p1necone 7y ago
      It's very possible they're not "officially" on call and don't get any extra pay for having to work in the weekend. Just because that's the current expectation of a lot of salaried workers doesn't mean it's right.
  - npo9 7y ago
    Discord has millions of users. Less than 1000 people are probably bothered at Google to fix the problem. My sympathies lie with Discord users. I’m sure it’s ruining a lot more weekend plans with them. I’ve both a discord user and someone that has spent years in an on-call rotation that included nights and weekends.
    - deleted 7y ago
      [deleted]
ecf 7y ago
<> [Removed reply since the comment I was replying to was removed.]
- npo9 7y ago
  This type of outage happens with all of the cloud providers from time to time.
  - scarface74 7y ago
    Yes. Technically it happens to all companies but politically it’s a lot different. If you tell your customers that you have an outage because AWS is down, they will shrug and say, things happen. You are in the same boat with a lot of other people, you will no more get blamed for AWS being down than if the whole city lost power. No one ever got fired for choosing AWS. If you’re on Azure, you might get a funny look from people who only know about AWS, but since it is Microsoft Azure you will get a pass. But people will judge you for choosing GCP. Any little thing that goes wrong, they will be questioning why you didn’t choose AWS.
    - xerox13ster 7y ago
      Not gonna lie, one of the first things I said to my friends is why is discord using Google, it should be AWS.
      - Fellshard 7y ago
        The categories of errors - and Google's ability to recognize and respond to those errors in rapid form - has always seemed extraordinarily slow and reticent compared to AWS. It's one of the concerns I'd have when recommending it.

VolatileVoid 7y ago

Is it just me or are Discord's status updates legendarily bad to the point of being entirely useless? As a user of the discord app, don't tell me the "API is having issues." Just tell me, "Discord is currently down. We are working on it and apologise for the disruption." Everything else is unnecessary line noise.

p1necone 7y ago
Mentioning the cause of the issue seems better than just "shits broke yo" to me.
- VolatileVoid 7y ago
  Really? What in their status update read to you as "the problem is on our side, not yours"? Understand that I'm a programmer and couldn't figure that out.
  - Operyl 7y ago
    Because if it was a problem on your end, it wouldn't be listed on their status page....?
  - aquova 7y ago
    "Google's Engineering team continues to work on mitigating the issue" I think that's pretty clear where the issue lies
Fabricio20 7y ago
The Discordapp status page is written to by developers, and as a developer I really like the insights as to what is going on. As a user, I expect you to just look at it and see it's "Having Issues, We are working on it".

scottmf 7y ago

We use Discord as an interface for moderating our app and monitoring spam/reported content so this kinda sucks. Not super important though.

seriesf 7y ago

Being taken down by slow IO in a single cloud zone seems to implicated their overall architecture.

jrockway 7y ago
Looking at their job postings, they use Cassandra and Elasticsearch, so I am somewhat surprised that they didn't survive this outage. (I have not ever run Cassandra at scale, but their website says in the first paragraph that it's designed to handle regional failures.) If I were on GCP and had paying customers, I would use Cloud Spanner. It's expensive, but it's a good piece of technology.
- sieabahlpark 7y ago
  Most startup companies worry about building out features than building out reliable infrastructure. Almost always an after thought
- seriesf 7y ago
  The gift of Spanner is it’s always slow, so if you start there you will naturally gain experience in either hiding latency or relaxing consistency requirements, which are both good skills.
  - vl 7y ago
    Spanner is pretty fast, but as with any database you have to use it correctly to get the speed. Since it’s a non-standard database, it requires non-standard tricks and skills.
    - jrockway 7y ago
      No, that's about right. You get one write per second per entity or something like that and that's as fast as it goes. The key is to choose your entity wiseley. I wrote an app inside Google that did on the order of 10,000 writes per second and it was no problem; each entity group only showed up once every minute or so.