6 ms·
It's never been clear to why very simple unikernels are better than running essentially the same code as a normal user program.
by amluto 9y ago
It's never been clear to why very simple unikernels are better than running essentially the same code as a normal user program.
- andrewstuart 9y agoThere's a whole zoo of "little operating systems" out there that more or less have unikernel-like properties. Some more closely meet the definition of a Unikernel, others are more like stripped down Linux, others are more exotic OS's that are nothing more than a network driver and disk driver in a virtual machine, such as Solo5. I got Bootrino (see elsewhere) to boot all of these types of operating systems including embedded Linux (Yocto), Solo5, TinyCore, Alpine, and some other weird run-from-RAM OS configurations. I got MirageOS running on all of the above, so unikernels can fit within other OSs or live on their own.
- wmf 9y agoA user process running on an OS running on a hypervisor has certain overhead, so you can either remove the hypervisor (native containers) or you can remove the OS (unikernels). Arguably the hypervisor has a smaller attack surface than an OS. Also, there's a small set of programs that don't get along with the abstractions provided by OSes and run better on a virtual hardware abstraction instead.
- first_amendment 9y agoUnfortunately containers aren't considered secure enough for malicious users while VMs are.
- AstralStorm 9y agoNeither are most unikernels. Especially the weak points are file system and network handling. Generally the security of VMs is enforced by hardware and improved by limited attack surface - few drivers to audit, limited communication APIs. You can do the same with general purpose OS by cutting options.
- first_amendment 9y agoI'm not talking about unikernels. I'm talking about the VMs that isolate them from each other. The VM isolation mechanism is considered secure enough to isolate malicious users, while the container isolation mechanism isn't (at least Linux-based containers).
- cyphar 9y agoSolaris/illumos Zones and FreeBSD Jails most definitely are considered secure for malicious users. And they've been secure for more than a few years.
- amscanne 9y agoYou realize that the guest OS _does_ things right? It's not just overhead. You can't just remove the Linux network stack, you have to replace it with something. (And the Linux network stack is pretty damned good at what it does.) The intrinsic overhead of the OS (i.e. costs of mode switching) is much smaller than you might think. It's true that some programs don't really want an OS getting in their way, but a modern operating system provide decent facilities for getting _out_ of the way (e.g. O_DIRECT for block I/O, pinning tasks to cores) and that's usually what those things use.
- amluto 9y agoThis is backwards. A userspace process on x86 has a certain amount of overhead, namely the overhead needed to switch between it and the host kernel. This takes the form of SYSCALL/SYSRET (mostly) for system calls and interrupts/IRET (mostly) for interrupt returns. SYSCALL and SYSRET are quite fast. interrupts and IRET are slow. A VM guest needs to do a VM exit (which is quite a bit slower than even an interrupt, I believe) and then fix up a bunch of host state that doesn't get restored by the CPU. Then, to get back to guest mode, it needs to manually restore some guest state and do VMRESUME. Regular user mode wins hands down. To be fair, unikernels get to directly create page tables, which is potentially very useful. But, for the simpler applications, it's probably not. (I more or less maintain the glue between x86 Linux user code and the kernel. I do, in fact, know exactly how this works.)
- wmf 9y agoThe number of calls probably wouldn't be the same, though. A process might make hundreds of system calls while a unikernel might make one hypercall to do the same work (because virtio has batching). Anyway, I don't personally think reducing system call overhead is interesting but it seems to be one of the arguments unikernel people make.
- amluto 9y agoI see no reason whatsoever that you can't do virtio without virtualization.
- cturner 9y agoYesterday there was a post about a coming linux kernel feature, kernel TLS sockets. (https://news.ycombinator.com/item?id=15164568 https://news.ycombinator.com/item?id=15164568) One of the justifications: it removes the need to copy data between kernel/user-space. KTLS strikes me as inelegant. Socket programming requires you to reason about your frames on the wire. Proto-socket features complicate this. A unikernel approach would allow us to eliminate unnecessary copies, and could be done with more elegance than KTLS.
- trapperkeeper74 9y agoThe Linux kernel is a dumping-ground, kitchen-drain of all features (20 megaLoC with no end in sight) because of their inability to isolate concerns and provide fast IPC.
- AstralStorm 9y agoYou also forgot about security in this IPC which is very much not a priority for Linus. With good reasons - security community has no consensus there.
- orf 9y agoAnd yet able to capture the server and phone market so completely. Best isn't always best.
- xxs 9y agoShared memory and busy spinning work normally when you need that latency (high freq. trading for instance)
- api 9y agoIs the obsession with avoiding memory copying really justified? I'd like to see some data. Is this really a bottleneck in modern systems? Is TLS a bottleneck at all for most work loads?
- justincormack 9y ago
- rwmj 9y agoAll the replies so far miss amluto's point: A unikernel running on a hypervisor is a process, just with an awkward syscall interface and inflexible memory management. Especially when the hypervisor is KVM which runs VMs as regular Linux processes, this is very clear.
- eeZah7Ux 9y ago> A unikernel running on a hypervisor is a process, just with an awkward syscall interface and inflexible memory management. ...and with no way to run ancillary scripts and cronjobs e.g. log management, backups, security tools... Nor ssh into the VM to inspect it and run debuggers or monitoring and metric agents. Essentially you are giving away all the practical advantages of modern OSes. There's a reason why the usual famous tech companies don't use unikernels, generally speaking.
- anonymousDan 9y agoNot the case, Google for example are looking at using the Linux Kernel Library: https://netdevconf.org/1.2/session.html?jerry-chu https://netdevconf.org/1.2/session.html?jerry-chu
- mato 9y agoVM interfaces are awkward due to emulating hardware so that they can run existing operating systems. A VT-backed unikernel interface does not need to inherit this awkwardness. As for memory management, in the Solo5/ukvm case the amount of memory you grant to the guest (monitor process) is essentially the maximum you expect the guest to need, ever. So it's similar in terms of overhead to many usermode allocators which just mmap() a huge block of memory at start and then hand out to the application from that.
- amscanne 9y agoThat's how a regular KVM works as well (mmap a physical memory block) and the mismatch comes from the fact that all the regular OS memory management mechanisms -- reclaim, swap, page cache, etc. -- are all broken by the abstraction provided. FV guests can similarly have a balloon driver, that doesn't mean the interface isn't awkward.
- jlg23 9y agoThe smaller attack surface and the possibility to implement optimizations across boundaries (userspace vs kernelspace, for example) are the selling points. I'm not buying into this myself as of yet but at least in theory this could lead to.an "application provably correct all the way down the stack".
- amluto 9y agoWhat smaller attack surface? If you compare the attack surface of a Linux user process that is very tightly seccomped to the attack surface of a whole guest VM (with all the awkward emulation needed), it's not at all clear to me that guests are better.
- lmm 9y agoBetter isolation and a cleaner interface. There's a huge hodgepodge of different ways user programs can communicate and interfere with each other, many of which are rarely used and don't make sense for most programs but have to be retained lest removing them break something.
- fulafel 9y agoIn which case something like seccomp is a better vessel for these programs? VM interfaces have lots of surface area.
- lmm 9y agoQuite the opposite. The Unix userland <-> kernel and IPC interfaces are enormous: hundreds of system calls, often overlapping, a dozen different socket types, multiple different kinds of magic filesystem interfaces (some of which take complex datastructures in arbitrary unstandardized text formats)... Whereas the interface between a VM and its host is pretty small.
- fulafel 9y agoseccomp is a Linux system for restricting a process to a small # of syscalls.
- lmm 9y agoSure, but the set you'd restrict to would look rather like the VM interfaces, only more awkward because it's had to be retrofitted on to what already exists, and often more permissive than it should be because a library assumes all syscalls are available (why would it not?)
- fulafel 9y agoThe virtualized hardware interfaces also enormous. And more cumbersome to restrict than seccomp, and more cumbersome to use than syscalls. I'm not sure I follow the logic in your harder-to-retrofit opinion. In unikernels the central idea is retrofitting the unikernel around the host interface and the application. It seems obvious to me that this would be more easier to develop and debug inside a standard process, than an emulation of bare iron, even if you would have to call it a different platform due to a radically smaller syscall interface.
- mato 9y agoBy their nature, unikernels are extremely well suited to running on top of a simple and well-defined interface. By extension, such interfaces are easy to audit and secure. Why is this better than running as a normal user process? Look at the complexity inherent in securely sandboxing applications using existing interfaces -- the Linux system call interface amounts to hundreds of system calls, with different mutations for different architectures and many calls multiplexing even more operations (eg. ioctl()) the numbers grow even higher. Unikernels give us a shot at redefining the process interface, while still being able to run on existing infrastructure (host OS or hypervisor). This means we can work toward a system where the application (unikernel) has least privilege by default. With Solo5 [1] we're working towards defining such an interface. We have a way to go yet [2], and are still very much in the "early adopter who knows what a unikernel is" stage, but the possibilities are exciting, not only from the application security point of view. [1] https://github.com/solo5/solo5 https://github.com/solo5/solo5 [2] I'd be the first to say that the current Solo5 interfaces are suboptimal, but we're working on it: https://github.com/Solo5/solo5/pull/201 https://github.com/Solo5/solo5/pull/201
- unilynx 9y agoHow is this any different from running a process with a very short whitelist of system calls? Compare to the original linux seccomp, which only allowed read, write, exit and sigreturn.