7 ms·
The Road to 2M Websocket Connections in Phoenix
- toast0 11y agoIf you listen on multiple ports, you can get way more than 64k connections per client. If the workload is mostly idle, tsung should be able to manage quite a few connections too.
- chrismccord 11y agoWe'd like to explore a private network so we can get away with more than 64k per client, as it starts to get expensive spinning up 50 servers whenever we want to test. Unfortunately, the instances that were lent to us came with no way for us to assign additional IPs.
- toast0 11y agoYou don't need multiple IPs, tcp connections are a unique 4-tuple: {ClientIP, ServerIP, ClientPort, ServerPort}. If you add more ServerPorts, you can re-use the client ports. Now, if you want to test past 4 billion connections, you need more IPs. :)
- noir_lord 11y ago> You don't need multiple IPs, tcp connections are a unique 4-tuple: {ClientIP, ServerIP, ClientPort, ServerPort}. Sometimes someone throws something out so casually and yet you realise the innate sense of what they just said, I literally stopped with coffee halfway to my mouth. I'd never have thought of doing it that way.
- toast0 11y agoI may have connection tested a few things in the past ;) If you're benchmarking with tsung, it's nice, because it has hooks to let you pick the client port when making the connection; you'll probably need this, because the OS default assignment algorithms tend to get it wrong with this much stuff going on.
- eridius 11y agoWait wait wait, you can re-use client ports for multiple connections? Does this actually make it past NATs? Why does every TCP connection get a unique client port by default if there's no problem with reusing them as long as you're talking to a different {ServerIP,ServerPort} pair?
- chrismccord 11y agoMy ElixirConfEU keynote gave a high-level overview of the channels design for those interested - quick jump to place in presentation: https://youtu.be/u21S_vq5CTw?t=22m34s https://youtu.be/u21S_vq5CTw?t=22m34s I'm also happy to answer any other questions. The team is very excited about our channel and pubsub layers progress over the last year.
- bcardarella 11y agoWe've been using Phoenix for several applications for over a year. The channel functionality is so easy to use and implement. High recommend people checking this out: http://www.phoenixframework.org/docs/channels http://www.phoenixframework.org/docs/channels
- jsmoov 11y agoI'd also recommend the Programming Phoenix book, it's still in beta but the chapter on Channels just came out: https://pragprog.com/book/phoenix/programming-phoenix https://pragprog.com/book/phoenix/programming-phoenix
- ch4s3 11y agoThis is really cool, I'm hoping to see more small companies experimenting with Phoenix. It seems like a great tool.
- paulus_magnus2 11y agoisn't there a 65k limit of number of open ports on the TCP stack??
- asonge 11y agoAs noted above, TCP connection limits are only on a unique 4-tuple of local-ip, local-port, remote-ip, remote-port.
- Thaxll 11y agoIs it a relevant benchmark to open 2M connexions without doing anything in it?
- sugarpirate 11y agoThere is a video (https://www.youtube.com/watch?v=N4Duii6Yog0 https://www.youtube.com/watch?v=N4Duii6Yog0) at the end of the post which shows Chris demoing the chat room app.
- chrismccord 11y agoI was worried some might not make it that far into the post and come away with that thought, but such is the nature of long-form prose :). This is more than idle WS connections. Every channel client connects to the sever (via WS), then subscribes via our PubSub layer to the chat room topic. Once we connect all 2M clients, we use the real chat app to broadcast a message to all 2M users. Broadcasts to out to all WS clients in about 2s. 2M people in the same chat room might not be a real-world fit, but it stressed out PubSub layer to its max, which was one of the goals for these tests. We were actually shocked at how well broadcasts hold up at these levels.
- BenoitP 11y agoWhat this benchmark shows is the lightweightness of the memory needed per websocket. 2M users is really impressive. 2M users per machine is great if they are mostly idle. This is the use case of WhatsApp, and their stats are[1]: > Peaked at 2.8M connections per server > 571k packets/sec > >200k dist msgs/sec Not every app is meant to have mostly idle users. Can a real-time MMO FPS be done in phoenix is the question (lets limit each user's neighbourhood to the 10 closest players). I'd be very interested in the other corners of the envelope for phoenix: A requests/second benchmark over websockets, with the associated latency HdrHistogram, like [2] [1] http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi... [2] http://www.ostinelli.net/a-comparison-between-misultin-mochiweb-cowboy-nodejs-and-tornadoweb/ http://www.ostinelli.net/a-comparison-between-misultin-mochi...
- polskibus 11y agoGreat job Phoenix team! I wonder how much of this wouldn't even be possible if not for the great BEAM platform and Cowboy web server. Whatsapp famously works with 2m connections on a FreeBSD box (this number is old, I bet they've beaten that number). http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-facebook-bought-for-19-billion.html http://highscalability.com/blog/2014/2/26/the-whatsapp-archi... I wonder whether these two cases are in any way comparable. Different stacks, different machines, different test. The only similarity being BEAM/Erlang used as a platform. Speaks well of its scalability!
- chrismccord 11y agoNone of it would be possible without BEAM and Cowboy :) Cowboy is a huge driver for our great numbers. It handles the tcp stack and websocket connections for us. As I said elsewhere in this thread, we are doing more than just holding idle WS connections open, but we're standing on the shoulders of giants as far as cowboy and erlang are concerned.
- mtw 11y agoI'd be interested in benchmarks vs other similar frameworks, for the same use case. These had json/db queries benchmarks however Phoenix lags behind (will be at the top soon?) https://www.techempower.com/benchmarks/previews/round11/ https://www.techempower.com/benchmarks/previews/round11/
- out_of_protocol 11y agoTake a look at source code. Phoenix in question 1) outdated, version used is really really old (0.8 or so) 2) Database pool set to really low value unlike other participants (~10x lower) 3) Configured to do many things other frameworks skip, like generating csrf token All these points were fixed already but, most likely, new results will go to Round 12
- mtw 11y agook. will see round 12. But why did you downvote my comment? I was just reporting a fact, a benchmark that was performed by a third-party. It's not like I was voicing a weird opinion or fabricating truth.
- out_of_protocol 11y agoI didn't (nor have such rights nor intention). Also, valid question imo. Btw, tested it [old version] locally with a single change - set pool size to 256 (similar to other frameworks). Result: ~10x performance boost
- pekk 11y agoThis is par for the course on the TechEmpower benchmarks. Certain technologies' benchmarks have always been relentlessly optimized while others are left to "the community" which means you can submit PRs, but they'll likely be rejected even if they reflect the usual deployment patterns for a high proportion of users of the language/framework being represented.
- jkarneges 11y agoI love this kind of stuff. Awhile back I attempted to do similar benchmarking with Pushpin: http://blog.fanout.io/2013/10/30/pushing-to-100000-api-clients-simultaneously/ http://blog.fanout.io/2013/10/30/pushing-to-100000-api-clien... Like the Pheonix test, this tested broadcast throughput. However we couldn't push more than 5000 messages per second out of a single EC2 m1.xlarge instance otherwise latency would increase. My theory is that we were hitting the maximum throughput of the underlying NIC, causing packet loss and requiring TCP to retransmit (hence the latency). At some point I want to try adding multiple virtual NICs to the same EC2 instance and see if that helps.
- philsnow 11y agoec2 instances (at least in ec2 classic) indeed have a limit of ~120K packets per second. I don't know if this is per NIC, so I would be interested in seeing your results.
- arrty88 11y agoam i reading this correctly... 83gb of ram in use for 2m connections? seems like a lot of overhead. i have achieved 500k concurrent connections using atmosphere, consuming around 10gb of ram.
- chrismccord 11y agoThat's correct. We have not made any attempt at optimizing connection size yet, so there should be room to trim this down. But honestly, I'm thrilled with the numbers as is. This box on rackspace will run you $1500/mo, which is going to be peanuts for a company requiring this kind of scale, but the more we can trim down the connection size the better.
- darthdeus 11y ago> which is going to be peanuts for a company requiring this kind of scale That depends. Two million connections from people in an e-commerce site is a lot, but two million connections for a side-thingy like some analytics/ads/background-job-whatever doesn't have to be that much, especially if you consider 3rd party code.
- chrismccord 11y agoCases will vary, but if you have 2 million active users at a time and can't cover $1500/mo, then I'm not sure what your options will be. Along these lines, I'm really excited about what kinds of creations Phoenix will enable exactly because the kind of scale it gives you for current hardware. I think we'll see disruptive applications/services come out because of what you can get out of a single machine, so I find stressing price as not affordable in this context really bizarre.
- darthdeus 11y agoI understand, I do like the idea of Phoenix, and it is definitely great to be able to keep 2M connections open at all. I'm just wondering where is the per-connection overhead going into. Is there some inherent limitation of the WebSocket protocol that forces the server to keep large buffers or something? Not trying to bash on Phoenix, I'm just genuinely interested in what is the lowest possible overhead one could achieve while keeping an open WS connection.
- rozap 11y agothe other day i set up hot code upgrades (in a single node, production instance) with exrm and phoenix. it was a breeze and took about 30 minutes of reading and setup. and now i can upgrade the backend while all the websocket connections stay running. so cool!
- alberth 11y agoI'd be interested in seeing this same performance benchmark run against various other OSes. In particular DragonflyBSD since it has a lockless network stack. And FreeBSD for good measure given that Whatsapp runs an Erlang/FreeBSD stack and has documented 2-3M connections themselves.
- chrismccord 11y agoWe were shocked how well stock ubuntu performed under these tests. With the stories of freebsd tuning, we expected to hit an OS ceiling, but other than a dozen shell script lines setting some sysctrl limits, everything else was stock. I would also be very interested in how other OS's perform.
- nulltype 11y agoNow that you can buy servers in tiny linear increments, it seems more interesting (but less headline-y) to talk about KB per connection or CPU per connection. It looks like this is about 41KB per connection. For comparison, I checked my not-very-optimized go server, and it looks like (based on VSZ value) to use 25KB per connection in go, with 43KB per connection used by nginx (to provide SSL termination).
- chrismccord 11y agoLike I highlighted elsewhere, did your Go server setup a separate multiplexed process for the PubSub channel and track the PubSub subscribers on each client connection? Comparing the client size to raw WS connections isn't accurate in this context. Horizontal scaling is nice, but you also have to consider the overhead in broadcasts that now are distributed over dozens/hundreds of nodes vs a handful of large nodes.
- nulltype 11y agoI track the pubsub subscribers on each client connection, but I'm not sure what you mean by a multiplexed process for the pubsub channel. Depending on your traffic patterns, considering the broadcast activity is potentially important, and then you might worry more about CPU than RAM. I use redis myself, but gnatsd or redis cluster may help with scaling the pubsub part of the system.
- ntrepid8 11y agoI've been using elixir/cowboy/Phoenix for the past few months and it's been great. It seems like the elixir ecosystem is still being built out but there are a lot of erlang libraries to use also.
- eridius 11y agoGreat article! One minor nit: When explaining the bag vs duplicate_bag thing, it says > The difference between bag and duplicate_bag is that duplicate_bag will allow multiple entries for the same key. bag also allows multiple entries per key. After reading the ETS documentation, it appears that duplicate_bag allows the same object instance to appear as a value for a key multiple times, whereas bag only allows an object instance to be added once to a key (e.g. so if you add the exact same object instance to the table using a given key, bag will only end up with one value for the key, but duplicate_bag will happily have many identical values for that key). The following sentence is still fine: > Since each socket can only connect once and have one pid, using a duplicate bag didn't cause any issues for us.