3 ms·
Depends on several other variables. If you use default docker options, you'll be creating a veth pair per container. You might run into a limit there at around
by TheDong 9y ago
Depends on several other variables.
If you use default docker options, you'll be creating a veth pair per container. You might run into a limit there at around 1024 containers. You also might hit ulimit if your system isn't well configured.
If you use --net=none, you won't hit that issue, and you'll probably be able to manage quite a few
The resource usage ends up being roughly 4 bytes rss for the executable in the container and around 3.5MB for the "containerd-shim" go binary that parents the container.
"containerd" and "dockerd" both probably have a little extra resource usage per container they're managing, but I'd guess that's on the order of about 200KB per at most.
The next big limit you'll hit is the process/pid limit (/proc/sys/kernel/pid_max) which defaults to 32k.
Fortunately, due to the memory overhead of a bit under 4MB, you probably won't get there on your 64GB of ram server and might cap out at around 15k containers total.
Experimentally, my linux laptop (running docker 17.06) is able to run 1100 copies of that sleeping-beuaty container using almost exactly 2GB RSS additional memory and no noticeable additional cpu
This is even better than I calculated above, possibly due to shared memory for containerd-shim. I'm not investigating further.
- e12e 9y agoWhat happens when you hit the limit? Thrashing swap or oom kills? (And what kind of overcommit policy did you use?)
- TheDong 9y agoI didn't hit any limit, but since I don't have swap it would likely be oom kills. The first part of my post is speculating, the second part I ran an arbitrary number and observed resource usage to allow extrapolation, but didn't hit a limit.
- ffk 9y agoThanks for reproducing the experiment. :) I sent a message to tianon asking him if he remembers what the original numbers were. He told me long ago, I think it was around 1000. This was well before the go shim or dockerd existed.
- logicallee 9y agothanks, this was interesting! That's better than I expected.