5 ms·
Honestly, I'd very much like to see more deep studies into container overhead - not just from a kernel standpoint, but actual tit-for-tat measures for storage a
by rcarmo 10y ago
Honestly, I'd very much like to see more deep studies into container overhead - not just from a kernel standpoint, but actual tit-for-tat measures for storage and networking overhead that spanned the different filesystem options (I know that aufs is old hat, but I've used overlay and btrfs too) and, probably most importantly, the networking overhead.
I've had slowdowns in the order of 30% in terms of requests per second on some of my stuff inside Swarm as compared to running processes outside containers (arguably with four or five moving parts and entirely anecdotal, but still enough to give me pause), and I'd really like to understand how to shave this yak.
- gtirloni 10y agoIt really depends on what is being used. There are studies showing the overhead for containers is minimum, CPU-wise. However, you might see docker-proxy eating a lot of CPU due to increased traffic. Or dockerd adding overhead to collect statistics, manage log buffers, etc. My point is, pure containers using basic kernel features don't seem to add a lot of overhead. It's the sugar on top that sometimes is the problem, but that will vary depending on the container runtime, the app being contained, extra features that were enabled, etc. I've seen CouchDB perform just fine in a container when using Docker's "bridge" network and drop to a halt if using Docker's "host" network. It seemed to dislike the situation and was doing way more syscalls than usual. Just to show the app might not like a certain environment. Even with some overhead, it's a trade-off I'm willing to accept considering the advantages in the development workflow and managing the infrastructure. Most of what I've seen are rough edges that will get ironed out with time.
- rcarmo 10y agoYeah, of course. On the network overhead aspect, and as an extra data point, I'm using macvlan with great success on the Raspberry Pi (https://github.com/rcarmo/docker-plex-armhf/blob/master/Makefile https://github.com/rcarmo/docker-plex-armhf/blob/master/Make...), but on cloud services that's pretty much impossible... (I've also investigated the Ubuntu fan technique to lower overhead, but am looking for a more definitive solution)
- annnnd 10y ago> However, you might see docker-proxy eating a lot of CPU due to increased traffic. You can say that again. Tried running Docker container with large range of exposed ports once (because SIP)... Each port got its own docker-proxy and the system basically suffocated. Not a nice view. :)
- user5994461 10y agoFYI: VMWare is 0% CPU overhead. CPU instructions are executed as-is. Running on VmWare, ESX, Xen, Docker, LXD makes no difference. It's a different story with network and storage access. Impact of containerization/virtualization is variable(5 to 95% performance drop).
- AstralStorm 10y agoUsually true. That said, some instructions and hardware are emulated or virtualised. (E.g. APIC, clocks, privileged instructions) The main overhead is in IO though. Closer to 10% with KVM virtio than 95%.
- brianwawok 10y agoAnd this is why friends don't let friends run databases in containers.
- chc 10y agoDon't Virtio and devicemapper have different performance characteristics, though?
- brianwawok 10y agoIf any of them give near native IO, would love to see. Haven't tested anything like that yet myself.
- deleted 10y ago[deleted]
- dawnerd 10y agoDocker is great cpu wise unless you're running on macos. It still likes to go nuts every now and then.