3 ms·

It's easy to scale to 5m websockets with almost any languages, the real question is what those connections are doing, looking at Discord we can assume that a la

by Thaxll 8y ago

It's easy to scale to 5m websockets with almost any languages, the real question is what those connections are doing, looking at Discord we can assume that a large % of users are idle for example. My Discord client is always on but I do nothing on it, I'm pretty sure it's the case for most people since it starts at windows startup. I actually think that Elixir / Erlang are actually not great for those kind of problems because they're slow language and consume a lot of memory. They allow you to do easy message passing + horizontal scaling but the runtime is inefficient. Java / C# net core / Rust / C++ / Go are much faster than Elixir. ( if you actually read on it you'll see that they use a lot of C / C++ / Rust to make it fast which is not something you need to with the above languages ). And for deployment / scaling just use Kubernetes or equivalent, better than BEAM trust me.

jhgg 8y ago
Since this blog post, we are well over 5m ccu (I don't think we have shared our current peaks publicly yet). We do egress 4m-6m websocket messages/sec through our system though at peak. So it's a lot more than just holding onto idle websockets :P The power of BEAM is that although the performance may not be the best, throughput and response time is consistently low throughout the system - allowing no single process (or actor) from monopolizing resources of the system. When you use a homogenous server configuration like we do, this makes a lot of sense. Our largest guilds (100k ccu, 350k members) are on the same nodes as all of our other servers. And when they're busy, the small guilds notice no performance degradation. Other chat products out there (that I hear use the JVM for their real time stuff) have to spin up dedicated resources to hosting their larger servers/clients - and even then cannot handle servers with as many users or concurrents that we can. Every single discord server runs in a homogenous cluster, without special dedicated resources for our largest instances.

by Thaxll 8y ago

jhgg 8y ago

Since this blog post, we are well over 5m ccu (I don't think we have shared our current peaks publicly yet). We do egress 4m-6m websocket messages/sec through our system though at peak. So it's a lot more than just holding onto idle websockets :P The power of BEAM is that although the performance may not be the best, throughput and response time is consistently low throughout the system - allowing no single process (or actor) from monopolizing resources of the system. When you use a homogenous server configuration like we do, this makes a lot of sense. Our largest guilds (100k ccu, 350k members) are on the same nodes as all of our other servers. And when they're busy, the small guilds notice no performance degradation. Other chat products out there (that I hear use the JVM for their real time stuff) have to spin up dedicated resources to hosting their larger servers/clients - and even then cannot handle servers with as many users or concurrents that we can. Every single discord server runs in a homogenous cluster, without special dedicated resources for our largest instances.