3 ms·

Thank you very much for your response. Some follow-up questions, in the same order as your responses: 1. So, in production, we do this all the time with the k

Thank you very much for your response. Some follow-up questions, in the same order as your responses: 1. So, in production, we do this all the time with the kernel. We have specific .config's that we use on a per-app basis so I'm not sure why this point is made against the kernel? Is it because in general people run what they're provided by the distro? I don't really understand the LTO argument as well, is it that the kernel is smaller? When I compared IncludeOS against the production kernels we run for memcache they were 1.5x bigger. In general, how much of an issue is kernel code size? a 50M kernel, while big pales in comparison to the amount of memory required for runtime operation. Have you done any measurements on runtime costs? 2. Do you have any benchmarks/evidence you could point me to? My benchmarks showed the exact opposite at a huge development cost (debugging unikernels is very painful - at least for me) 3. What do you mean by "same old same old"? The primitives for isolation are provided by the hardware with VT-x and VT-d and your point about "removing the ability to even talk to the hypervisor after configuration stage" is referring to VT-x from what I gather. All modern hypervisors use these primitives and in combination with VT-x and PCIe VFs provide the illusion of complete access to hardware so I think I'm missing your point. 4. I'm sorry but I find the argument that "threads automatically mean worse" hard to understand. The cost of a context switch is the cost of a context switch. The cost of a context switch between threads/processes on the same CPU is still much cheaper than the cost of a context switch between VMs on the same CPU. If a program requires threading then, with the Unikernel either I need an abstraction for threading (e.g. co-operative scheduling) that bypasses the cost of the switch _or_ I need to write the program differently. In your answer, you make the point you don't support threads which means that either I move to an event-driven model (which I can do in Linux too) until I run out of CPU or I switch between VM contexts, which in my understanding is much more costly. Do you mind elaborating further? 5. Do you have an official line on this? I would assume the people you're selling against will be deploying containers so it would be good to get a feel for unikernels on containers. Thanks again for your response. I found it most helpful.

fwsgonzo 8y ago
Hey, IncludeOS is mostly a work in progress, and so the answers will reflect that. 1 and 2: IncludeOS is a service where more stuff is compiled and inlined as a whole, compared to a normal kernel. The whole thing will be baked into one, in many cases inline. This gives the compiler a chance to simplify more things and optimize better, and while the size doesn't have to be smaller, we do have less code overall. Especially after the linker removes unused objects. IncludeOS is not being built with LTO at all presently (although it does have that ability on the Linux userspace platform), nor does it have any major performance work done other than making sure there are no big performance bugs around. I am working on this, but it's not a trivial thing. In a perfect world I would just enable LTO and it would just work, like I already do on the userspace Linux platform of IncludeOS. I have written a beginners-kernel project where I show that LTO reduces image size to 2/3 from 32kb to 11kb: https://github.com/fwsGonzo/barebones https://github.com/fwsGonzo/barebones 2. There is a paper showing this linked to in the repo itself. You can at least build the unikernel on linux userspace, so it shouldn't be too hard to debug it while you are creating it. Other than that, debugging is not easy in production just like you say. 3. Traditional hypervisors manage access to hardware all the time, while a new type of unikernel hypervisor wouldn't. It would just sit there after the configuration/startup phase and never be exited into. The number of VMEXITs something has to do for each packet is a good estimator for how performant something is. If the number is zero, then it must be performant. See bareflank and others. I can't elaborate further because it's an area of research, and I'm not an expert. Afaik there is nothing like this running in production yet, even if we can prove that it's possible to run exitless and with no kernel privileges! Maybe next year we will know more. 4. We absolutely support threads, its just not working right now. We also have an SMP API that is quite good, and the virtual CPUs are scheduled on the host side. We also have fibres. While we do have tests, I don't know the state of things. We have so many other things going trying to make the OS stable in a production environment. I couldn't find a single thing proving my assertion when I made a quick search, so I will just retract until I can find it again. As I'm sure you know unikernels tend to have a single address space, which removes the need to do a full context switch. 5. I don't think our customers will be deploying containers. Rather, I think they will be running hundreds of instances per physical host in their private clouds. Instances will act as Internet gateways, firewalls etc. Some network function. That is the stage we are at now. I really don't see containers being used after the binaries are built, but I could be wrong.

fwsgonzo 8y ago

Hey, IncludeOS is mostly a work in progress, and so the answers will reflect that. 1 and 2: IncludeOS is a service where more stuff is compiled and inlined as a whole, compared to a normal kernel. The whole thing will be baked into one, in many cases inline. This gives the compiler a chance to simplify more things and optimize better, and while the size doesn't have to be smaller, we do have less code overall. Especially after the linker removes unused objects. IncludeOS is not being built with LTO at all presently (although it does have that ability on the Linux userspace platform), nor does it have any major performance work done other than making sure there are no big performance bugs around. I am working on this, but it's not a trivial thing. In a perfect world I would just enable LTO and it would just work, like I already do on the userspace Linux platform of IncludeOS. I have written a beginners-kernel project where I show that LTO reduces image size to 2/3 from 32kb to 11kb: https://github.com/fwsGonzo/barebones https://github.com/fwsGonzo/barebones 2. There is a paper showing this linked to in the repo itself. You can at least build the unikernel on linux userspace, so it shouldn't be too hard to debug it while you are creating it. Other than that, debugging is not easy in production just like you say. 3. Traditional hypervisors manage access to hardware all the time, while a new type of unikernel hypervisor wouldn't. It would just sit there after the configuration/startup phase and never be exited into. The number of VMEXITs something has to do for each packet is a good estimator for how performant something is. If the number is zero, then it must be performant. See bareflank and others. I can't elaborate further because it's an area of research, and I'm not an expert. Afaik there is nothing like this running in production yet, even if we can prove that it's possible to run exitless and with no kernel privileges! Maybe next year we will know more. 4. We absolutely support threads, its just not working right now. We also have an SMP API that is quite good, and the virtual CPUs are scheduled on the host side. We also have fibres. While we do have tests, I don't know the state of things. We have so many other things going trying to make the OS stable in a production environment. I couldn't find a single thing proving my assertion when I made a quick search, so I will just retract until I can find it again. As I'm sure you know unikernels tend to have a single address space, which removes the need to do a full context switch. 5. I don't think our customers will be deploying containers. Rather, I think they will be running hundreds of instances per physical host in their private clouds. Instances will act as Internet gateways, firewalls etc. Some network function. That is the stage we are at now. I really don't see containers being used after the binaries are built, but I could be wrong.