4 ms·
Nobody should be afraid of 20000 connections or even 20 billion connections. The cost of having a lot of TCP connections hanging around are wildly over-estimate
by jeffbee 6y ago
Nobody should be afraid of 20000 connections or even 20 billion connections. The cost of having a lot of TCP connections hanging around are wildly over-estimated. You should not even think about subsetting unless/until you can measure the cost in terms of CPU spent on keepalive messages or memory spent on connection state, or you're getting more than millions of file descriptors on a single machine. Even if you are subsetting, very small subsets (like 10) are definitely harmful, and hiding the full set of candidate backends is also harmful. If you are subsetting, allow your clients to observe the global state and make their own decisions. What's it going to cost you? A few thousand IP addresses and port numbers in memory? That's basically nothing.
- morelisp 6y agoI also struggle to see the case the parent is trying to solve, but I would be concerned purely on a memory basis if my service opens even millions of unnecessary connections, let alone 20 billion.
- jeffbee 6y agoAcross what size of fleet? It's meaningless to say you'd be concerned about millions of connected sockets, except to the extent that it was a significant fraction of total system cost.
- mumblemumble 6y agoYou've got at least the size of the read and write for every open connection to contend with. Let's be real conservative and clamp that down to 4KiB each. So that's 8K per connection. Meaning that a rough lower bound on the memory usage under this scenario would be something like ~8GiB per million open connections, and ~8TiB per billion open connections. You're right that it comes down to the extent to which that's a significant fraction of total system cost. I think I can say with reasonable confidence, though, that this would be a significant fraction of total system cost for the vast majority of systems.
- jeffbee 6y agoReally? I'm not willing to let you wave your arms about that last conclusion. Suppose you had 100 frontends and 10000 backends in a fully-connected manner with 1 million sockets. You spent 8KiB on this state (I think that's a bit low, but I'm using your figure). That would only be a significant fraction of (more than 1% of) 100 * 1E6 * 8KiB == 800GiB, spread across your 10100 containers, meaning they each have only 80MiB of memory? This seems very unlikely. A much more likely scenario is each of your 10100 things has at least 2-4GB of memory per CPU core, meaning at least 20-40TiB of memory in your system, meaning that the million connections cost you 0.02% of your system memory, or essentially nothing.
- AtNightWeCode 6y agoMemory when using a workload scheduler can for sure be as low as 80MB. I believe it was set at 100MB for all containers handled by Nomad in a solution I used to work with. It would be insanely expensive to build solutions with your specs to just solve a problem that should be solved by redesigning for fewer connections.
- jeffbee 6y ago36MB is a fatally small container on Linux x86-64, and really nobody should try to use a memcg smaller than 128MB.
- AtNightWeCode 6y agoYeah, not uncommon that containers crashed and the developers came back and increased the amount of memory once Nomad killed stuff.
- mumblemumble 6y agoI think the point of contention here is probably what a typical system looks like. I haven't personally had the pleasure of working on a system that's anywhere close to having 100 frontends and 10,000 backends, and I don't have many friends who have, either, so I assume that's an atypical use case. Perhaps I'm wrong there.
- strken 6y agoI'm not an expert on this, but I think what's costly is storing and communicating information about the load of each server a client is connected to, not the overhead of TCP. I found http://liblb.com/learn.html http://liblb.com/learn.html useful for understanding how different load balancing algorithms work.