5 ms·

am i reading this correctly... 83gb of ram in use for 2m connections? seems like a lot of overhead. i have achieved 500k concurrent connections using atmospher

by arrty88 11y ago

am i reading this correctly... 83gb of ram in use for 2m connections? seems like a lot of overhead. i have achieved 500k concurrent connections using atmosphere, consuming around 10gb of ram.

chrismccord 11y ago
That's correct. We have not made any attempt at optimizing connection size yet, so there should be room to trim this down. But honestly, I'm thrilled with the numbers as is. This box on rackspace will run you $1500/mo, which is going to be peanuts for a company requiring this kind of scale, but the more we can trim down the connection size the better.
- darthdeus 11y ago
  > which is going to be peanuts for a company requiring this kind of scale That depends. Two million connections from people in an e-commerce site is a lot, but two million connections for a side-thingy like some analytics/ads/background-job-whatever doesn't have to be that much, especially if you consider 3rd party code.
  - chrismccord 11y ago
    Cases will vary, but if you have 2 million active users at a time and can't cover $1500/mo, then I'm not sure what your options will be. Along these lines, I'm really excited about what kinds of creations Phoenix will enable exactly because the kind of scale it gives you for current hardware. I think we'll see disruptive applications/services come out because of what you can get out of a single machine, so I find stressing price as not affordable in this context really bizarre.
    - darthdeus 11y ago
      I understand, I do like the idea of Phoenix, and it is definitely great to be able to keep 2M connections open at all. I'm just wondering where is the per-connection overhead going into. Is there some inherent limitation of the WebSocket protocol that forces the server to keep large buffers or something? Not trying to bash on Phoenix, I'm just genuinely interested in what is the lowest possible overhead one could achieve while keeping an open WS connection.
      - asonge 11y ago
        Doing some rough math and my limited understanding of linux network internals, it's about 40KB per connection in this benchmark. I know that cowboy is going to require ~4KB or so per connection. Consulting a local ubuntu install, the default minimum buffer sizes in TCP will be at least 4KB each (2* for read and write), but by default 16KB each, and by default the max goes to 1.5MB or so each. This is required for TCP retransmits and such. If you have clients on shoddy connections or see packet loss, your memory could skyrocket on you. I remember reading of a case where someone had a service die despite memory overhead of 33% when the TCP packet loss rate went up (still under 1%), but it caused their buffer sizes to grow large enough to run out of memory. So that's 8KB (will be higher with more usage) for TCP buffers in the kernel, 4KB or so for cowboy, and 28KB or so left for various other bits of the system when amortized per connection.
darthdeus 11y ago
That's only about twice as much as in your case (I'm not saying it's not a lot, just pointing it out). Still, I wonder how low could one get when implementing this at a much lower level. 41kB per connection seems like a lot of bookkeeping for something that's essentially a handle to a socket? Yes there's a process overhead in BEAM, but based on the Erlang docs, this should be only 309 memory words.
chrismccord 11y ago
Also keep in mind our clients aren't just idle WS processes. They start a channel process for each topic they join ("rooms:lobby") in this case. So 1 WebSocket connection will spawn two processes. One for the transport (WS handler), and one for the channel. The channel also holds its own state, which is mostly empty, but carries metadata, so it's hard to compare to your 500k raw connections.

by arrty88 11y ago

am i reading this correctly... 83gb of ram in use for 2m connections? seems like a lot of overhead. i have achieved 500k concurrent connections using atmosphere, consuming around 10gb of ram.

chrismccord 11y ago

That's correct. We have not made any attempt at optimizing connection size yet, so there should be room to trim this down. But honestly, I'm thrilled with the numbers as is. This box on rackspace will run you $1500/mo, which is going to be peanuts for a company requiring this kind of scale, but the more we can trim down the connection size the better.

darthdeus 11y ago
> which is going to be peanuts for a company requiring this kind of scale That depends. Two million connections from people in an e-commerce site is a lot, but two million connections for a side-thingy like some analytics/ads/background-job-whatever doesn't have to be that much, especially if you consider 3rd party code.
- chrismccord 11y ago
  Cases will vary, but if you have 2 million active users at a time and can't cover $1500/mo, then I'm not sure what your options will be. Along these lines, I'm really excited about what kinds of creations Phoenix will enable exactly because the kind of scale it gives you for current hardware. I think we'll see disruptive applications/services come out because of what you can get out of a single machine, so I find stressing price as not affordable in this context really bizarre.
  - darthdeus 11y ago
    I understand, I do like the idea of Phoenix, and it is definitely great to be able to keep 2M connections open at all. I'm just wondering where is the per-connection overhead going into. Is there some inherent limitation of the WebSocket protocol that forces the server to keep large buffers or something? Not trying to bash on Phoenix, I'm just genuinely interested in what is the lowest possible overhead one could achieve while keeping an open WS connection.
    - asonge 11y ago
      Doing some rough math and my limited understanding of linux network internals, it's about 40KB per connection in this benchmark. I know that cowboy is going to require ~4KB or so per connection. Consulting a local ubuntu install, the default minimum buffer sizes in TCP will be at least 4KB each (2* for read and write), but by default 16KB each, and by default the max goes to 1.5MB or so each. This is required for TCP retransmits and such. If you have clients on shoddy connections or see packet loss, your memory could skyrocket on you. I remember reading of a case where someone had a service die despite memory overhead of 33% when the TCP packet loss rate went up (still under 1%), but it caused their buffer sizes to grow large enough to run out of memory. So that's 8KB (will be higher with more usage) for TCP buffers in the kernel, 4KB or so for cowboy, and 28KB or so left for various other bits of the system when amortized per connection.

darthdeus 11y ago

That's only about twice as much as in your case (I'm not saying it's not a lot, just pointing it out). Still, I wonder how low could one get when implementing this at a much lower level. 41kB per connection seems like a lot of bookkeeping for something that's essentially a handle to a socket? Yes there's a process overhead in BEAM, but based on the Erlang docs, this should be only 309 memory words.

chrismccord 11y ago

Also keep in mind our clients aren't just idle WS processes. They start a channel process for each topic they join ("rooms:lobby") in this case. So 1 WebSocket connection will spawn two processes. One for the transport (WS handler), and one for the channel. The channel also holds its own state, which is mostly empty, but carries metadata, so it's hard to compare to your 500k raw connections.