5 ms·

Discord Incident – Resolved

89 points by moelf 5mo ago

rweichler 5mo ago
Times like this make me miss the IRC days, I was just able to reproduce a bug in an semi-open-source project, and Discord went down right in the middle of me sending my findings. Now there's nothing I can do about it. I can only wait.
- aroman 5mo ago
  I mean, when freenode would go down it was more or less the same thing, no?
  - Brian_K_White 5mo ago
    Not remotely. IRC is distributed and federated. Not only are there countless networks, each network has countless servers, and each group of servers that are up and can see each other can operate on their own, all the way down to a single server, or up to any subset up to all. When a peering connection goes down and the network splits, maybe some people in the group disappear, or maybe from your point of view everyone else disappears. Maybe the remaining subset of other users is already good enough because it's enough to continue what you were tallking about and who you were talking with, or if not, you have the option to just try some other servers until you find where everyone alse is. Were "server" is an actual seperate instance of the server software operated by an independant person, hosted on whatever kind of hardware or vm they set up, connected to whatever network they are on, not what Discord calls a "server". Even if the entire group of say freenode servers goes down somehow (even though that's not really possible) there is still undernet and 400 other nets. Even without prior coordination it would be essentially trivial for the users to all just go looking for, or create on the spot, the same channel on some other net, and basically everyone finds each other again almost effortlessly. And that's if something unbelievable actually happens, let alone the normal minor breaks that actually happen once in a while. This is entirely different from being wholly at the mercy of the single entity Discord.
    - aroman 5mo ago
      You're arguing against a claim I did not make. Freenode had full-network outages periodically. ddos attacks, infrastructure failures etc. and when those happened, the practical experience was the same... people waited it out. Nobody coordinated a mass migration to undernet or stood up alternative servers for a few hours. (It took much bigger issues - social/organizational/political, not technical - to catalyze the mass migration.) You're making an argument about the virtues of decentralization - and I agree, decentralization is great! Just in practical reality, freenode (not IRC itself) had exactly the same failure mode as we just saw today.
      - nubinetwork 5mo ago
        > Nobody coordinated a mass migration to undernet or stood up alternative servers for a few hours. There was always oftc...
      - Brian_K_White 5mo ago
        I'm not really arguing the superiority except as a side effect of answering the question "isn't it the same?" No, it's not the same at all. When irc servers go down or networks split, the users just hop on other servers if they actually want to keep talking with the same specific people they were. The earlier comment posed the scenario of being high and dry when they needed to do basically support work with users. If they don't care then they don't bother, but that doesn't change the fact that if the communication mattered to you, then you are not stuck the way the earlier comment said. You are only as stuck is you feel like. And not just because of the hyperbolic technicality that you can stand up your own new server. You could, but you never need to. There are countless servers already. And there is no coordinated "mass migration" operation. The channel breaks, you just go browse a few other servers, or if it's exceptionally bad, maybe another network. It's effortless for each user. It takes 2 seconds. You're not trying 1000 other networks blindly either. You already know a few more popular/likely suspects to try first. And so does everyone else, and even when you guess "wrong" and go to dalnet and everyone else is on undernet, there will still be a channel on dalnet with a topic or a user telling you where to go. It's all just not a big deal. It took way more effort to write this comment explaining it than to actually deal with a net split and get back in communication with more or less everyone that was in whatever channel broke.
  - rweichler 5mo ago
    freenode was run by imbeciles
- Analemma_ 5mo ago
  How would that have been different in the IRC days?
  - omoikane 5mo ago
    There would have been a server split, and half of the people would be chatting among themselves wondering why the some people suddenly got disconnected, while being unaware that there might be a server problem because they can still continue to chat. The other half of the people would think the same.
    - StableAlkyne 5mo ago
      Don't forget everyone flooding both halves with "z0mg net split!" :P
      - echelon 5mo ago
        > "z0mg net split!" :P The 00's were an interesting time in internet culture. Internet slang like this disappeared almost completely once the whole world got access and platforms rooted out all the weird and niche communities.
      - skerit 5mo ago
        Netsplits were fun. Especially if you were on the splitting part. You could get to know new people you got stuck with.
  - uproarchat 5mo ago
    In the IRC days you could have been running your own server for you and friends. It would've taken something much worse, like your VPS dying or something upstream failing.
  - rweichler 5mo ago
    The IRC servers I idled in simply never went down. Sounds impossible, I know. You can choose to believe me or not.
- Aurornis 5mo ago
  I feel like I have different memories about the instability of Internet services in the past than some people do. Common IRC servers were not without problems. I think it was just more common to shrug it off and do something else until the problems went away.
  - cogman10 5mo ago
    The difference was it wasn't one global server for everyone. I think that's why the past feels like it was more stable. Now, aws or cloudflare gets a hickup and half the internet is nuked. The old internet was far more federated so doing something else meant to me "Welp, anandtech is down, let's go to pcper, digg, tomshardware, slashdot, etc" Sure stuff would go down, but it would be just that small community rather than most of chat for the internet.
    - filoleg 5mo ago
      Yeah, but (as a user) I would rather have one global server crash for 1-2hrs two-three times per year, as opposed to having each individual server randomly crash once a month for at least one each time. The more I sit down and try to remember how it actually was to use internet in late 00s, the only thing that always comes up is "there is no way people today would tolerate it nearly as well as we did back then".
  - bayindirh 5mo ago
    I never seen long-running problems in big server-federations like DALNet. Our local "big" IRC servers were generally down for 10 minutes at most. They were not empty either. Simple services recover faster. Federated infrastructure is much more resilient. We had slower computers, more considerate coders, and simpler software; so everything was snappier, even with 56K modems. For example, navigate to https://git.sr.ht/~bayindirh/ https://git.sr.ht/~bayindirh/. No scripts, pure HTML. running on a single server. Served instantly. This is possible. We, as in the world, just ignore it for shinier stones. Now, a small VPS in an AWS server lapses for 5 seconds, and half of internet is toast. Centralization for the PWN!
  - piva00 5mo ago
- jstummbillig 5mo ago
  I don't exactly know what you are comparing. No popular IRC network came anywhere near what we would find acceptable in terms of reliability today. It was an absolutely (wonderful) wildfire.
  - anyfoo 5mo ago
    Netsplits, where the entire IRC network would "split" into two (or more) effectively independent networks because some a link between two servers went down, were extremely common. I don't know if daily, or weekly, but common enough to be perceived as normal and expected in any case. In the earlier days of IRC, netsplits were sometimes used for channel takeover. If someone was on a split off part of the net where there were so few people in the channel that they could obtain op status, they could kill and ban the "legitimate" ops when the nets joined back together.
    - BoredPositron 5mo ago
      It was so much fun placing some eggdrops on servers that usually split to takeover channels.
- wetpaws 5mo ago
  [dead]
113 5mo ago
It's not a novel opinion but I'm tired of things being fucked all the time.
- grim_io 5mo ago
  Can't wait for Cloudflare to go the way of GitHub :)
- this_user 5mo ago
  It's gonna get a lot worse with all the AI slopcode that is about to be pushed directly to production.
  - majorchord 5mo ago
    You mean all the AI "slop" that's finding and writing new kernel exploits every day? And submitting hundreds of previously-unknown security bugs in critical software?
    - lpcvoid 5mo ago
      [dead]
    - suprjami 5mo ago
      What are you saying here? An LLM helped humans do something right once therefore it's perfect to use in every other situation too?
      - deleted 5mo ago
        [deleted]
    - dakolli 5mo ago
      llms aren't doing any of that. Some smart people are using llms find those vulns, there's a huge difference.
      - 5mo ago
uproarchat 5mo ago
I keep second guessing whether or not to mention Uproar here out of fear we'll get roasted, because it's still got its blemishes, but here goes: https://uproar.chat https://uproar.chat We're hoping to do better than discord, hopefully you get some use from it! Backend is written in go, frontend is vanilla html/js/css, TOS and PP are readable in one breath each. edit: looks like nobody can see this unless it's vouched. I guess because of the link and VPN.
- dang 5mo ago
  (It doesn't look like you've been spamming HN so I restored the comment)
  - uproarchat 5mo ago
    Thanks Dang!
- stryan 5mo ago
  From a normie perspective: - No screenshots on front page, I have no idea what it looks like - no video chat, no screen sharing - No downloadable version isn't a feature. What's a PWA? - "Live audio space" doesn't explain whether it's drop-in voice channels like discord/slack huddles or scheduled audio calls - The name makes it sound like a Discord clone From a technical perspective: - Not FOSS, can't self-host or federate. What makes this less likely to rug pull than Discord/any of the other alternatives - No information on who is making this - No information on how messages are encrypted - Webpage looks vaguely AI generated - Bot API is A) hidden at the bottom of the very long tutorial, B) seems to be limited to normal user actions (I could be wrong!), and C) desperately needs an index or sidebar - Unclear whether anonymous channels are truly anon or just anon on the client side Some stuff seems neat: I am intrigued by anonymous channels and from your feature table it hits more table-stakes features than most Discord alternatives. But I would give it a few touch ups if you want it to stand out.
  - uproarchat 5mo ago
    Thanks for checking it out and taking the time to give feedback. You made some good points. After staring at the site for so long, you go blind to these types of things, so you're helping a lot here. >no video chat, no screen sharing Screen sharing and video chat were added just a few days ago, and I wanted to get that tested more before mentioning it so prominently. >No downloadable version isn't a feature. What's a PWA? At the moment, we're a small team, and not all engineers. None of us know Swift or Kotlin, but aren't against learning. Usage is something we want to see before investing that much time into writing these, so in the meantime, a PWA is the best middle ground. One key differentiator from Discord is that we're not against third party clients, so if in the meantime someone comes out with one we're not going to make any efforts to stop it. >The name makes it sound like a Discord clone I see what you mean, if you had a magic wand, what would you call it? >"Live audio space" doesn't explain whether it's drop-in voice channels like discord/slack huddles or scheduled audio calls We have video now, but to answer directly, it's similar to a Space on X. Thank you for the feedback, I will be working this weekend to make these things better articulated on the website!
hohithere 5mo ago
[flagged]
coreylane 5mo ago
im surprised the aws outage hasn't been bigger news today https://www.cnbc.com/2026/05/08/aws-outage-data-center-fanduel-coinbase.html https://www.cnbc.com/2026/05/08/aws-outage-data-center-fandu...
- jeffwask 5mo ago
  It was one AZ. Kinda surprised those guys are built in a way where a single AZ failure takes them down.
  - dietr1ch 5mo ago
    AWS makes it annoying to be resilient as AZs aren't transparent to their users, so I'm more surprised some were prepared for it. It seems to me these day people are OK with AWS going down and just blaming it on AWS rather than on themselves for not being prepared for big outages. "Oh, nothing we can do because AWS/Cloudflare is down"
    - baronvonsp 5mo ago
      > AWS makes it annoying to be resilient as AZs aren't transparent to their users What does transparent mean here? AWS is super clear what resources are zonal and provides tons of guidance around making things multi-AZ. AZ outages aren't exactly frequent but they're reasonably likely. Being susceptible to AZ (or region) outages is very much an architectural decision. Or a bug that needs to be fixed (I'm sure Coinbase didn't YOLO single-AZ, they've undoubtedly learned about some edge case that needs to be fixed). Sure it may not be worth the cost/complexity for some systems but resiliency is like job one for anything in the cloud that costs money when it's down.
      - dietr1ch 5mo ago
        > What does transparent mean here? Transparent as a system/box can be, meaning that you can't see / know about it. (Yeah, I guess you can read that not transparent as obscure in disclosure of how their system works, but it shouldn't make much sense) > AWS is super clear what resources are zonal and provides tons of guidance around making things multi-AZ. Yeah, they allow people cheaping out for zonal resources and then going down with their zone.
huxflux 5mo ago
IRC unite!
n80sire 5mo ago
Likely due to aws incident.
ChrisArchitect 5mo ago
Can update to the direct link: https://discordstatus.com/incidents/4hpm4454hxtx https://discordstatus.com/incidents/4hpm4454hxtx
throwawayk7h 5mo ago
matrix.org alive and well

89 points by moelf 5mo ago

Times like this make me miss the IRC days, I was just able to reproduce a bug in an semi-open-source project, and Discord went down right in the middle of me sending my findings. Now there's nothing I can do about it. I can only wait.

aroman 5mo ago
I mean, when freenode would go down it was more or less the same thing, no?
- Brian_K_White 5mo ago
  Not remotely. IRC is distributed and federated. Not only are there countless networks, each network has countless servers, and each group of servers that are up and can see each other can operate on their own, all the way down to a single server, or up to any subset up to all. When a peering connection goes down and the network splits, maybe some people in the group disappear, or maybe from your point of view everyone else disappears. Maybe the remaining subset of other users is already good enough because it's enough to continue what you were tallking about and who you were talking with, or if not, you have the option to just try some other servers until you find where everyone alse is. Were "server" is an actual seperate instance of the server software operated by an independant person, hosted on whatever kind of hardware or vm they set up, connected to whatever network they are on, not what Discord calls a "server". Even if the entire group of say freenode servers goes down somehow (even though that's not really possible) there is still undernet and 400 other nets. Even without prior coordination it would be essentially trivial for the users to all just go looking for, or create on the spot, the same channel on some other net, and basically everyone finds each other again almost effortlessly. And that's if something unbelievable actually happens, let alone the normal minor breaks that actually happen once in a while. This is entirely different from being wholly at the mercy of the single entity Discord.
  - aroman 5mo ago
    You're arguing against a claim I did not make. Freenode had full-network outages periodically. ddos attacks, infrastructure failures etc. and when those happened, the practical experience was the same... people waited it out. Nobody coordinated a mass migration to undernet or stood up alternative servers for a few hours. (It took much bigger issues - social/organizational/political, not technical - to catalyze the mass migration.) You're making an argument about the virtues of decentralization - and I agree, decentralization is great! Just in practical reality, freenode (not IRC itself) had exactly the same failure mode as we just saw today.
    - nubinetwork 5mo ago
      > Nobody coordinated a mass migration to undernet or stood up alternative servers for a few hours. There was always oftc...
    - Brian_K_White 5mo ago
      I'm not really arguing the superiority except as a side effect of answering the question "isn't it the same?" No, it's not the same at all. When irc servers go down or networks split, the users just hop on other servers if they actually want to keep talking with the same specific people they were. The earlier comment posed the scenario of being high and dry when they needed to do basically support work with users. If they don't care then they don't bother, but that doesn't change the fact that if the communication mattered to you, then you are not stuck the way the earlier comment said. You are only as stuck is you feel like. And not just because of the hyperbolic technicality that you can stand up your own new server. You could, but you never need to. There are countless servers already. And there is no coordinated "mass migration" operation. The channel breaks, you just go browse a few other servers, or if it's exceptionally bad, maybe another network. It's effortless for each user. It takes 2 seconds. You're not trying 1000 other networks blindly either. You already know a few more popular/likely suspects to try first. And so does everyone else, and even when you guess "wrong" and go to dalnet and everyone else is on undernet, there will still be a channel on dalnet with a topic or a user telling you where to go. It's all just not a big deal. It took way more effort to write this comment explaining it than to actually deal with a net split and get back in communication with more or less everyone that was in whatever channel broke.
- rweichler 5mo ago
  freenode was run by imbeciles
Analemma_ 5mo ago
How would that have been different in the IRC days?
- omoikane 5mo ago
  There would have been a server split, and half of the people would be chatting among themselves wondering why the some people suddenly got disconnected, while being unaware that there might be a server problem because they can still continue to chat. The other half of the people would think the same.
  - StableAlkyne 5mo ago
    Don't forget everyone flooding both halves with "z0mg net split!" :P
    - echelon 5mo ago
      > "z0mg net split!" :P The 00's were an interesting time in internet culture. Internet slang like this disappeared almost completely once the whole world got access and platforms rooted out all the weird and niche communities.
    - skerit 5mo ago
      Netsplits were fun. Especially if you were on the splitting part. You could get to know new people you got stuck with.
- uproarchat 5mo ago
  In the IRC days you could have been running your own server for you and friends. It would've taken something much worse, like your VPS dying or something upstream failing.
- rweichler 5mo ago
  The IRC servers I idled in simply never went down. Sounds impossible, I know. You can choose to believe me or not.
Aurornis 5mo ago
I feel like I have different memories about the instability of Internet services in the past than some people do. Common IRC servers were not without problems. I think it was just more common to shrug it off and do something else until the problems went away.
- cogman10 5mo ago
  The difference was it wasn't one global server for everyone. I think that's why the past feels like it was more stable. Now, aws or cloudflare gets a hickup and half the internet is nuked. The old internet was far more federated so doing something else meant to me "Welp, anandtech is down, let's go to pcper, digg, tomshardware, slashdot, etc" Sure stuff would go down, but it would be just that small community rather than most of chat for the internet.
  - filoleg 5mo ago
    Yeah, but (as a user) I would rather have one global server crash for 1-2hrs two-three times per year, as opposed to having each individual server randomly crash once a month for at least one each time. The more I sit down and try to remember how it actually was to use internet in late 00s, the only thing that always comes up is "there is no way people today would tolerate it nearly as well as we did back then".
- bayindirh 5mo ago
  I never seen long-running problems in big server-federations like DALNet. Our local "big" IRC servers were generally down for 10 minutes at most. They were not empty either. Simple services recover faster. Federated infrastructure is much more resilient. We had slower computers, more considerate coders, and simpler software; so everything was snappier, even with 56K modems. For example, navigate to https://git.sr.ht/~bayindirh/ https://git.sr.ht/~bayindirh/. No scripts, pure HTML. running on a single server. Served instantly. This is possible. We, as in the world, just ignore it for shinier stones. Now, a small VPS in an AWS server lapses for 5 seconds, and half of internet is toast. Centralization for the PWN!
- piva00 5mo ago
jstummbillig 5mo ago
I don't exactly know what you are comparing. No popular IRC network came anywhere near what we would find acceptable in terms of reliability today. It was an absolutely (wonderful) wildfire.
- anyfoo 5mo ago
  Netsplits, where the entire IRC network would "split" into two (or more) effectively independent networks because some a link between two servers went down, were extremely common. I don't know if daily, or weekly, but common enough to be perceived as normal and expected in any case. In the earlier days of IRC, netsplits were sometimes used for channel takeover. If someone was on a split off part of the net where there were so few people in the channel that they could obtain op status, they could kill and ban the "legitimate" ops when the nets joined back together.
  - BoredPositron 5mo ago
    It was so much fun placing some eggdrops on servers that usually split to takeover channels.
wetpaws 5mo ago
[dead]

113 5mo ago

It's not a novel opinion but I'm tired of things being fucked all the time.

grim_io 5mo ago
Can't wait for Cloudflare to go the way of GitHub :)
this_user 5mo ago
It's gonna get a lot worse with all the AI slopcode that is about to be pushed directly to production.
- majorchord 5mo ago
  You mean all the AI "slop" that's finding and writing new kernel exploits every day? And submitting hundreds of previously-unknown security bugs in critical software?
  - lpcvoid 5mo ago
    [dead]
  - suprjami 5mo ago
    What are you saying here? An LLM helped humans do something right once therefore it's perfect to use in every other situation too?
    - deleted 5mo ago
      [deleted]
  - dakolli 5mo ago
    llms aren't doing any of that. Some smart people are using llms find those vulns, there's a huge difference.
    - 5mo ago

uproarchat 5mo ago

I keep second guessing whether or not to mention Uproar here out of fear we'll get roasted, because it's still got its blemishes, but here goes: https://uproar.chat https://uproar.chat We're hoping to do better than discord, hopefully you get some use from it! Backend is written in go, frontend is vanilla html/js/css, TOS and PP are readable in one breath each. edit: looks like nobody can see this unless it's vouched. I guess because of the link and VPN.

dang 5mo ago
(It doesn't look like you've been spamming HN so I restored the comment)
- uproarchat 5mo ago
  Thanks Dang!
stryan 5mo ago
From a normie perspective: - No screenshots on front page, I have no idea what it looks like - no video chat, no screen sharing - No downloadable version isn't a feature. What's a PWA? - "Live audio space" doesn't explain whether it's drop-in voice channels like discord/slack huddles or scheduled audio calls - The name makes it sound like a Discord clone From a technical perspective: - Not FOSS, can't self-host or federate. What makes this less likely to rug pull than Discord/any of the other alternatives - No information on who is making this - No information on how messages are encrypted - Webpage looks vaguely AI generated - Bot API is A) hidden at the bottom of the very long tutorial, B) seems to be limited to normal user actions (I could be wrong!), and C) desperately needs an index or sidebar - Unclear whether anonymous channels are truly anon or just anon on the client side Some stuff seems neat: I am intrigued by anonymous channels and from your feature table it hits more table-stakes features than most Discord alternatives. But I would give it a few touch ups if you want it to stand out.
- uproarchat 5mo ago
  Thanks for checking it out and taking the time to give feedback. You made some good points. After staring at the site for so long, you go blind to these types of things, so you're helping a lot here. >no video chat, no screen sharing Screen sharing and video chat were added just a few days ago, and I wanted to get that tested more before mentioning it so prominently. >No downloadable version isn't a feature. What's a PWA? At the moment, we're a small team, and not all engineers. None of us know Swift or Kotlin, but aren't against learning. Usage is something we want to see before investing that much time into writing these, so in the meantime, a PWA is the best middle ground. One key differentiator from Discord is that we're not against third party clients, so if in the meantime someone comes out with one we're not going to make any efforts to stop it. >The name makes it sound like a Discord clone I see what you mean, if you had a magic wand, what would you call it? >"Live audio space" doesn't explain whether it's drop-in voice channels like discord/slack huddles or scheduled audio calls We have video now, but to answer directly, it's similar to a Space on X. Thank you for the feedback, I will be working this weekend to make these things better articulated on the website!

hohithere 5mo ago

[flagged]

coreylane 5mo ago

im surprised the aws outage hasn't been bigger news today https://www.cnbc.com/2026/05/08/aws-outage-data-center-fanduel-coinbase.html https://www.cnbc.com/2026/05/08/aws-outage-data-center-fandu...

jeffwask 5mo ago
It was one AZ. Kinda surprised those guys are built in a way where a single AZ failure takes them down.
- dietr1ch 5mo ago
  AWS makes it annoying to be resilient as AZs aren't transparent to their users, so I'm more surprised some were prepared for it. It seems to me these day people are OK with AWS going down and just blaming it on AWS rather than on themselves for not being prepared for big outages. "Oh, nothing we can do because AWS/Cloudflare is down"
  - baronvonsp 5mo ago
    > AWS makes it annoying to be resilient as AZs aren't transparent to their users What does transparent mean here? AWS is super clear what resources are zonal and provides tons of guidance around making things multi-AZ. AZ outages aren't exactly frequent but they're reasonably likely. Being susceptible to AZ (or region) outages is very much an architectural decision. Or a bug that needs to be fixed (I'm sure Coinbase didn't YOLO single-AZ, they've undoubtedly learned about some edge case that needs to be fixed). Sure it may not be worth the cost/complexity for some systems but resiliency is like job one for anything in the cloud that costs money when it's down.
    - dietr1ch 5mo ago
      > What does transparent mean here? Transparent as a system/box can be, meaning that you can't see / know about it. (Yeah, I guess you can read that not transparent as obscure in disclosure of how their system works, but it shouldn't make much sense) > AWS is super clear what resources are zonal and provides tons of guidance around making things multi-AZ. Yeah, they allow people cheaping out for zonal resources and then going down with their zone.

huxflux 5mo ago

IRC unite!

n80sire 5mo ago

Likely due to aws incident.

ChrisArchitect 5mo ago

Can update to the direct link: https://discordstatus.com/incidents/4hpm4454hxtx https://discordstatus.com/incidents/4hpm4454hxtx

throwawayk7h 5mo ago

matrix.org alive and well