6 ms·

Million WebSockets and Go (2017)

210 points by riobard 7y ago

jbmsf 7y ago
There's a brief mention of the load-balancer (nginx) in front of the Go servers; I'm curious if there's anything interesting happening there. I'd imagine that if you lose a server, all of the clients will try to reconnect and traffic will be spread across the existing servers. That's all find and good, but presumably when you bring up a new server to replace the failed on, it'll be seriously underutilized. Is there some easy solution here in nginx-land?
- toredash 7y ago
  For websocket? Yes (https://github.com/SocketCluster/loadbalancer https://github.com/SocketCluster/loadbalancer), but you would have to introduce another layer (AFAIK) that would detect failure and reconnect to a healthy target without informing the client.
andrewmatte 7y ago
This is still super interesting, two years later but does anyone have an update? Susheel Aroskar, a Netflix engineer, did a talk about push notifications https://www.infoq.com/presentations/neflix-push-messaging-scale/ https://www.infoq.com/presentations/neflix-push-messaging-sc... (2018)
- andrewmatte 7y ago
  https://lwn.net/Articles/775238/ https://lwn.net/Articles/775238/ Dave Doyle and Dylan O'Mahony did something pretty amazing related too with websockets for Bose.
maurodelazeri 7y ago
nothing beats this https://github.com/uNetworking https://github.com/uNetworking
phoboslab 7y ago
It's surprising to me that you apparently have to fight for memory usage for these cases when using Go. A while ago I ran a (quite naively written) nodejs application that maxed out at ~700k WebSocket connections per server - using only 4GB of RAM. Here CPU became the bottleneck.
- sansnomme 7y ago
  Go's concurrency design trades off memory usage for productivity; instead of red-blue functions where you have to explicitly design for function interrupt/yield points with the async keyword, you can just write sequential code and the runtime will handle the rest. The downside to this approach is that often the stack will have to be copied during the switching process vs the stackless approach preferred by Node.js, Rust, C# etc. See the excellent Fibers Under a Magnifying Glass paper by Microsoft Research: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
  - nielsbjerg 7y ago
    Productivity in this case being in the eye of the beholder? I’d argue that people experienced in how node works, wouldn’t have to think too much, since async is the default. I agree with your general sentiment though.
    - sansnomme 7y ago
      Async by default can often lead to weird race conditions if you are not careful.
      - networkimprov 7y ago
        Yes; you still need to lock/synchronize shared resources in an async context. Even if the environment is single-threaded.
        7y ago
maxpert 7y ago
So some time ago when I was playing around with my toy project (RaspChat) I noticed creating 2 channels and a go routine for every incoming websocket connection is not the answer. I was designing RaspChat to work on a 512MB Raspberry Pi; and I was bottle-necked by GC, and memory consumption around 3 - 4K connections. After loads of optimizations I got it around 5K. Digging deeper and found well I have to maintain a pool of go routines (like threadpool) and I have to write event loop. I was instantly pulling my hair. I was sacrificing so much simplicity, and flexibility of Node.js just because I was trying to avoid event loop and wanted to use channels (I did too much Erlang months before starting project and couldn't think anything other than process and messages). I got a backlash on my release (https://github.com/maxpert/raspchat/releases/tag/v1.0.0-alpha https://github.com/maxpert/raspchat/releases/tag/v1.0.0-alph...) from go community telling me how I was using desierializers/leaving loop holes in file upload and I didn't know shit about language. At that time I found uws (https://github.com/uNetworking/uWebSockets.js https://github.com/uNetworking/uWebSockets.js) that easily got me to 10K easily, and I was like "I would rather bet on a community investing on efficient websocket event loop rather than me writing my own sh*t". Don't get me wrong; I love Golang! Seriously I love it so much I have been pushing my company to use Golang. I just don't want to glorify the language for being silver bullet (which it's fanboys usually do). I would never implement complicated business logic that involves many moving pieces. When my business requires dealing with shape of an object and mixing matching things to pass data around; I would rather choose a language that lets me deal with shapes of object. Go has it's specific use-cases and strengths, people advertising it as move it to go and it would be faster than Java/C#/Node.js etc. have not done it or have not dealt with complexity of maintaining it.
- hnarn 7y ago
  > When my business requires dealing with shape of an object and mixing matching things to pass data around; I would rather choose a language that lets me deal with shapes of object. Could you elaborate on this a little?
  - nesarkvechnep 7y ago
    Since he mentioned Erlang, I bet he's talking about pattern matching.
fasteo 7y ago
For mail.ru, I was expecting [1] you would use tarantool for this task [1]https://hackernoon.com/tarantool-when-it-takes-500-lines-of-code-to-notify-a-million-users-11d340523493 https://hackernoon.com/tarantool-when-it-takes-500-lines-of-...

210 points by riobard 7y ago

There's a brief mention of the load-balancer (nginx) in front of the Go servers; I'm curious if there's anything interesting happening there. I'd imagine that if you lose a server, all of the clients will try to reconnect and traffic will be spread across the existing servers. That's all find and good, but presumably when you bring up a new server to replace the failed on, it'll be seriously underutilized. Is there some easy solution here in nginx-land?

toredash 7y ago
For websocket? Yes (https://github.com/SocketCluster/loadbalancer https://github.com/SocketCluster/loadbalancer), but you would have to introduce another layer (AFAIK) that would detect failure and reconnect to a healthy target without informing the client.

andrewmatte 7y ago

This is still super interesting, two years later but does anyone have an update? Susheel Aroskar, a Netflix engineer, did a talk about push notifications https://www.infoq.com/presentations/neflix-push-messaging-scale/ https://www.infoq.com/presentations/neflix-push-messaging-sc... (2018)

andrewmatte 7y ago
https://lwn.net/Articles/775238/ https://lwn.net/Articles/775238/ Dave Doyle and Dylan O'Mahony did something pretty amazing related too with websockets for Bose.

maurodelazeri 7y ago

nothing beats this https://github.com/uNetworking https://github.com/uNetworking

phoboslab 7y ago

It's surprising to me that you apparently have to fight for memory usage for these cases when using Go. A while ago I ran a (quite naively written) nodejs application that maxed out at ~700k WebSocket connections per server - using only 4GB of RAM. Here CPU became the bottleneck.

sansnomme 7y ago
Go's concurrency design trades off memory usage for productivity; instead of red-blue functions where you have to explicitly design for function interrupt/yield points with the async keyword, you can just write sequential code and the runtime will handle the rest. The downside to this approach is that often the stack will have to be copied during the switching process vs the stackless approach preferred by Node.js, Rust, C# etc. See the excellent Fibers Under a Magnifying Glass paper by Microsoft Research: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1364r0.pdf http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136...
- nielsbjerg 7y ago
  Productivity in this case being in the eye of the beholder? I’d argue that people experienced in how node works, wouldn’t have to think too much, since async is the default. I agree with your general sentiment though.
  - sansnomme 7y ago
    Async by default can often lead to weird race conditions if you are not careful.
    - networkimprov 7y ago
      Yes; you still need to lock/synchronize shared resources in an async context. Even if the environment is single-threaded.
      - 7y ago

maxpert 7y ago

So some time ago when I was playing around with my toy project (RaspChat) I noticed creating 2 channels and a go routine for every incoming websocket connection is not the answer. I was designing RaspChat to work on a 512MB Raspberry Pi; and I was bottle-necked by GC, and memory consumption around 3 - 4K connections. After loads of optimizations I got it around 5K. Digging deeper and found well I have to maintain a pool of go routines (like threadpool) and I have to write event loop. I was instantly pulling my hair. I was sacrificing so much simplicity, and flexibility of Node.js just because I was trying to avoid event loop and wanted to use channels (I did too much Erlang months before starting project and couldn't think anything other than process and messages). I got a backlash on my release (https://github.com/maxpert/raspchat/releases/tag/v1.0.0-alpha https://github.com/maxpert/raspchat/releases/tag/v1.0.0-alph...) from go community telling me how I was using desierializers/leaving loop holes in file upload and I didn't know shit about language. At that time I found uws (https://github.com/uNetworking/uWebSockets.js https://github.com/uNetworking/uWebSockets.js) that easily got me to 10K easily, and I was like "I would rather bet on a community investing on efficient websocket event loop rather than me writing my own sh*t". Don't get me wrong; I love Golang! Seriously I love it so much I have been pushing my company to use Golang. I just don't want to glorify the language for being silver bullet (which it's fanboys usually do). I would never implement complicated business logic that involves many moving pieces. When my business requires dealing with shape of an object and mixing matching things to pass data around; I would rather choose a language that lets me deal with shapes of object. Go has it's specific use-cases and strengths, people advertising it as move it to go and it would be faster than Java/C#/Node.js etc. have not done it or have not dealt with complexity of maintaining it.

hnarn 7y ago
> When my business requires dealing with shape of an object and mixing matching things to pass data around; I would rather choose a language that lets me deal with shapes of object. Could you elaborate on this a little?
- nesarkvechnep 7y ago
  Since he mentioned Erlang, I bet he's talking about pattern matching.

fasteo 7y ago

For mail.ru, I was expecting [1] you would use tarantool for this task [1]https://hackernoon.com/tarantool-when-it-takes-500-lines-of-code-to-notify-a-million-users-11d340523493 https://hackernoon.com/tarantool-when-it-takes-500-lines-of-...