4 ms·

You've got at least the size of the read and write for every open connection to contend with. Let's be real conservative and clamp that down to 4KiB each. So th

You've got at least the size of the read and write for every open connection to contend with. Let's be real conservative and clamp that down to 4KiB each. So that's 8K per connection. Meaning that a rough lower bound on the memory usage under this scenario would be something like ~8GiB per million open connections, and ~8TiB per billion open connections. You're right that it comes down to the extent to which that's a significant fraction of total system cost. I think I can say with reasonable confidence, though, that this would be a significant fraction of total system cost for the vast majority of systems.

jeffbee 5y ago
Really? I'm not willing to let you wave your arms about that last conclusion. Suppose you had 100 frontends and 10000 backends in a fully-connected manner with 1 million sockets. You spent 8KiB on this state (I think that's a bit low, but I'm using your figure). That would only be a significant fraction of (more than 1% of) 100 * 1E6 * 8KiB == 800GiB, spread across your 10100 containers, meaning they each have only 80MiB of memory? This seems very unlikely. A much more likely scenario is each of your 10100 things has at least 2-4GB of memory per CPU core, meaning at least 20-40TiB of memory in your system, meaning that the million connections cost you 0.02% of your system memory, or essentially nothing.
- AtNightWeCode 5y ago
  Memory when using a workload scheduler can for sure be as low as 80MB. I believe it was set at 100MB for all containers handled by Nomad in a solution I used to work with. It would be insanely expensive to build solutions with your specs to just solve a problem that should be solved by redesigning for fewer connections.
  - jeffbee 5y ago
    36MB is a fatally small container on Linux x86-64, and really nobody should try to use a memcg smaller than 128MB.
    - AtNightWeCode 5y ago
      Yeah, not uncommon that containers crashed and the developers came back and increased the amount of memory once Nomad killed stuff.
- mumblemumble 5y ago
  I think the point of contention here is probably what a typical system looks like. I haven't personally had the pleasure of working on a system that's anywhere close to having 100 frontends and 10,000 backends, and I don't have many friends who have, either, so I assume that's an atypical use case. Perhaps I'm wrong there.
  - jeffbee 5y ago
    Then why would you have a million established connections? That's my whole point here: don't worry about the connection count until you have a concrete need to worry about it.
    - mumblemumble 5y ago
      I think that's actually my whole point here. If I'm running a system on anywhere near the scale I'm used to operating, you damn well better believe that I'm going to worry about "20 billion connections." The concrete reason, which I can give a priori without even waiting for it to happen first, is that the best case scenario for what's going on is probably that the servers are being hugged to death. That and we've violated the laws of physics. Because even by the numbers that you agreed are grossly optimistic (which was deliberate), we're looking at that consuming an order of magnitude or two more RAM than we have across all our servers. So that would admittedly be cool. But still.
- morelisp 5y ago
  > A much more likely scenario is each of your 10100 things has at least 2-4GB of memory per CPU core This is nearly two orders of magnitude off (250-500MB using 4-16 CPU cores) for some of our most connection-intense services. And as you said 8KiB is extremely low, in practice more like 2-10x that once all buffering is taken into account. >5% of my memory spent on connection handling that (by the original assumption) I don't need, yes, that worries me.
  - jeffbee 5y ago
    That just doesn't make any sense. You can't even build a computer with 16 cores and 500MB of memory, so why would you have that ratio in production? This whole thread has been completely surreal.
    - morelisp 5y ago
      > You can't even build a computer with 16 cores and 500MB of memory, so why would you have that ratio in production? Because the service has no substantial local state so doesn't need any more than that? Should we invent unnecessary local state just so we can use more memory? That sounds considerably more surreal to me. Or maybe instead we should use all the memory we saved to keep our 200-300GB working set for a different service in RAM colocated, which is what we do...

jeffbee 5y ago

Really? I'm not willing to let you wave your arms about that last conclusion. Suppose you had 100 frontends and 10000 backends in a fully-connected manner with 1 million sockets. You spent 8KiB on this state (I think that's a bit low, but I'm using your figure). That would only be a significant fraction of (more than 1% of) 100 * 1E6 * 8KiB == 800GiB, spread across your 10100 containers, meaning they each have only 80MiB of memory? This seems very unlikely. A much more likely scenario is each of your 10100 things has at least 2-4GB of memory per CPU core, meaning at least 20-40TiB of memory in your system, meaning that the million connections cost you 0.02% of your system memory, or essentially nothing.

AtNightWeCode 5y ago
Memory when using a workload scheduler can for sure be as low as 80MB. I believe it was set at 100MB for all containers handled by Nomad in a solution I used to work with. It would be insanely expensive to build solutions with your specs to just solve a problem that should be solved by redesigning for fewer connections.
- jeffbee 5y ago
  36MB is a fatally small container on Linux x86-64, and really nobody should try to use a memcg smaller than 128MB.
  - AtNightWeCode 5y ago
    Yeah, not uncommon that containers crashed and the developers came back and increased the amount of memory once Nomad killed stuff.
mumblemumble 5y ago
I think the point of contention here is probably what a typical system looks like. I haven't personally had the pleasure of working on a system that's anywhere close to having 100 frontends and 10,000 backends, and I don't have many friends who have, either, so I assume that's an atypical use case. Perhaps I'm wrong there.
- jeffbee 5y ago
  Then why would you have a million established connections? That's my whole point here: don't worry about the connection count until you have a concrete need to worry about it.
  - mumblemumble 5y ago
    I think that's actually my whole point here. If I'm running a system on anywhere near the scale I'm used to operating, you damn well better believe that I'm going to worry about "20 billion connections." The concrete reason, which I can give a priori without even waiting for it to happen first, is that the best case scenario for what's going on is probably that the servers are being hugged to death. That and we've violated the laws of physics. Because even by the numbers that you agreed are grossly optimistic (which was deliberate), we're looking at that consuming an order of magnitude or two more RAM than we have across all our servers. So that would admittedly be cool. But still.
morelisp 5y ago
> A much more likely scenario is each of your 10100 things has at least 2-4GB of memory per CPU core This is nearly two orders of magnitude off (250-500MB using 4-16 CPU cores) for some of our most connection-intense services. And as you said 8KiB is extremely low, in practice more like 2-10x that once all buffering is taken into account. >5% of my memory spent on connection handling that (by the original assumption) I don't need, yes, that worries me.
- jeffbee 5y ago
  That just doesn't make any sense. You can't even build a computer with 16 cores and 500MB of memory, so why would you have that ratio in production? This whole thread has been completely surreal.
  - morelisp 5y ago
    > You can't even build a computer with 16 cores and 500MB of memory, so why would you have that ratio in production? Because the service has no substantial local state so doesn't need any more than that? Should we invent unnecessary local state just so we can use more memory? That sounds considerably more surreal to me. Or maybe instead we should use all the memory we saved to keep our 200-300GB working set for a different service in RAM colocated, which is what we do...