3 ms·

Meh. Realtime pub/sub at scale is easy as pie. I wrote a WebSocket client/server toolkit which can handle unlimited users with automatic channel sharding and au

Meh. Realtime pub/sub at scale is easy as pie. I wrote a WebSocket client/server toolkit which can handle unlimited users with automatic channel sharding and auto-scaling on Kubernetes as a weekend project. It was too easy. I made it open source and for some reason it only ended up being used by some crypto companies and a few adult chat websites though one of them said they were handling 120K concurrent users with a couple of instances.

esafak 3y ago
Sounds fun. Is the repo still up?
immibis 3y ago
Did you actually test it with unlimited users?
- jongjong 3y ago
  I tested it with hundreds of thousands of concurrent connections running on multiple hosts yes.
  - immibis 3y ago
    Thousands is nothing. A single thread can handle thousands. You don't need any of this sharding or Kubernetes stuff. I'd wager 120K connections can be handled by one fast instance. Above that, you have basic sharding techniques like assigning each channel to an instance. This should work up to some thousands of moderately active chatters in a single channel - at which point nobody can read all the messages anyway so it's not working for non-technical reasons. Slack is trying to solve the problem of having thousands of channels with millions of people, as well as millions of channels with thousands of people. In other words, about 6 orders of magnitude bigger than your load test.
    - jongjong 3y ago
      I was limited by time and budget there. My solution is open source after all and embarrassingly parallel so can potentially handle sharding across thousands of hosts. If I remember correctly, my calculations suggested that a cluster with 2k hosts could handle 40 million concurrent users if you push a message to every user every 5 to 10 seconds with some randomness. The per-channel limit at that rate (1 message to every users every 5 to 10 seconds) was 20K concurrent connections but the number of channels is unlimited since you can add more capacity by adding more hosts and the sharding of channels across available hosts is automatic. The only limit was number of hosts since there is only one coordinator instance in the cluster though the cluster can continue to operate without any downtime while the coordinator instance is down. The coordinator instance only needs to be up while the cluster is scaling up or down. That said it should be able to handle up to 5k hosts, potentially even much higher.

esafak 3y ago

Sounds fun. Is the repo still up?

immibis 3y ago

Did you actually test it with unlimited users?

jongjong 3y ago
I tested it with hundreds of thousands of concurrent connections running on multiple hosts yes.
- immibis 3y ago
  Thousands is nothing. A single thread can handle thousands. You don't need any of this sharding or Kubernetes stuff. I'd wager 120K connections can be handled by one fast instance. Above that, you have basic sharding techniques like assigning each channel to an instance. This should work up to some thousands of moderately active chatters in a single channel - at which point nobody can read all the messages anyway so it's not working for non-technical reasons. Slack is trying to solve the problem of having thousands of channels with millions of people, as well as millions of channels with thousands of people. In other words, about 6 orders of magnitude bigger than your load test.
  - jongjong 3y ago
    I was limited by time and budget there. My solution is open source after all and embarrassingly parallel so can potentially handle sharding across thousands of hosts. If I remember correctly, my calculations suggested that a cluster with 2k hosts could handle 40 million concurrent users if you push a message to every user every 5 to 10 seconds with some randomness. The per-channel limit at that rate (1 message to every users every 5 to 10 seconds) was 20K concurrent connections but the number of channels is unlimited since you can add more capacity by adding more hosts and the sharding of channels across available hosts is automatic. The only limit was number of hosts since there is only one coordinator instance in the cluster though the cluster can continue to operate without any downtime while the coordinator instance is down. The coordinator instance only needs to be up while the cluster is scaling up or down. That said it should be able to handle up to 5k hosts, potentially even much higher.