4 ms·
Right. Most cloud systems we have are user heavy, there's not much kernel, usually <3%. They cited a Google paper with 15-20% kernel, which is a lot. Although I
by brendangregg 9y ago
Right. Most cloud systems we have are user heavy, there's not much kernel, usually <3%. They cited a Google paper with 15-20% kernel, which is a lot. Although I've heard from other sources that Google is kernel heavy, so this further confirms that. Different workloads.
It'd be interesting if someone like sysdig would share (if they could) the usr/sys breakdown for their entire customer base (I seem to recall they were sharing other such statistics).
It's a bit hard to see how much an exokernel or unikernel would help given, say, a workload that was 5% kernel. You might say it would only improve performance by less than 5%. But that's just considering the cycles saved. Such kernels should also be smaller, and consume the CPU hardware caches less, leaving more room for the application to execute on those precious resources. That would improve the performance gain further than just the reduction in cycles alone.
- pjmlp 9y agoSecurity and boot times are also relevant, no? Unikernels are much easier to manage than containers.
- brendangregg 9y agoSure, both are relevant. Cutting boot time only helps so far: we also have to tune application warmup to be faster as well. This is a little dated (we've tuned it down since then), but here's a microservice with a 10 minute application warmup time: http://www.brendangregg.com/blog/2016-09-28/java-warmup.html http://www.brendangregg.com/blog/2016-09-28/java-warmup.html
- yaantc 9y agoCould reduced MMU overhead be significant on a server? I work in the embedded space, so very different with small CPUs. Using an MMU compared to a simple MPU can waste 25 to 30% of a CPU cycles easily. I wouldn't expect such overhead on a big server CPU with much bigger TLB (but also bigger working sets?), still a unikernel could use a very simple mapping with in some case a single large page, and see very little MMU management overhead. I have no idea on the possible gain on a server, but I'm curious. Any hindsight welcome.
- jnwatson 9y ago25-30% is highly unusual. There's something extraordinarily inefficient going on in your system.
- yaantc 9y agoThis was on an ARM926 running a quite large code base. The ARM926 TLB has only 64 entries, which is not much when running an OS with 4 kB pages. So that's quite an extreme (and old...) case, but interesting when dealing with rather small CPU with not so small context. The MMU has a performance cost to keep in mind. A server is very different. Still, a blog on transparent huge pages on Linux [1] shows an example where a high load JVM application server spends over 10% of its time doing page walk: "Yes, you see it right! More than 10% of CPU cycles were spent doing the page table walking.". So there's a massively bigger TLB, but also the server run much larger application with a large footprint. In the end, for some apps the MMU overhead can be significant in the server space. In a unikernel, there's a single address space used and no isolation. You could map it with only huge pages, and dramatically reduce TLB misses even for a large footprint application (assuming a server running mostly such unikernels, as another regular OS apps could trash the TLB otherwise I guess). Now, I'm not sure there's any such large footprint application running an a unikernel yet, but it may be a possible gain for unikernels on principle. [1] https://alexandrnikitin.github.io/blog/transparent-hugepages-measuring-the-performance-impact/ https://alexandrnikitin.github.io/blog/transparent-hugepages...