13 ms·
Unikernels: The Next Stage of Linux's Dominance [pdf]
- ch_123 7y agoIs there any way I can read this if I don't have an ACM subscription?
- anonymousDan 7y agohttps://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-hotos19.pdf https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-h...
- enriquto 7y agoIn this case the authors themselves put the pdf on their website. In the general case, you can almost always download the article from scihub by searching for the DOI of the publication.
- beefhash 7y ago> In this case the authors themselves put the pdf on their website. OT: This should honestly be a criminal offense, at least if done before a year passes. The publisher does valuable work in vetting contributions and checking the paper before publishing. They shouldn't be cheated out of their fair share of money that comes with due process just because authors go rogue.
- vidarh 7y agoIf "authors go rogue" they have recourse today. But determining whether or not authors have "gone rogue" depends on their agreement with the publisher. You do not have a basis for claiming they have done so unless you know whether or not they have obtained permission or whether or not they never handed over rights in the first place.
- tecleandor 7y agoCan't really say if this is sarcasm or not, but journals are paid to do that checking, it's not free, and they use unpaid people for peer review, so I'd say they aren't really losing money.
- rapsey 7y ago> Unikernels have demonstrated enormous advantages over Linux in many important domains, causing some to propose that the days of Linux's dominance may be coming to an end. Where are unikernels widely used?
- ailideex 7y agoresearch papers I would guess.
- pjmlp 7y agoAnd stuff like SQL Server for Linux or the original WSL implementation, via picoprocesses. Or the Docker TCP/IP stack on macOS, using parts of MirageOS.
- c0l0 7y agoI don't think that this statement describes the reality we live in. To me, unikernels feel quite a bit like statically linked server binaries running under an unprivileged UID - but you're choosing not to trust Linux' (or any other kernel's) user separation facilities, but your hypervisor's domU separation facilities instead. In exchange, you lose virtually all of your existing OS's amazing debugging and performance analysis/tuning tools. It's not a tradeoff I'd readily consider.
- squarefoot 7y agoThe article is Tl;Wbmloc;Dr (too long, way beyond my level of comprehension, didn't read), so forgive any stupid questions. Could the unikernel technology be used to build extremely small Linux distros that contain just the bare minimum to satisfy all dependencies for running say a single software or a small group of them. I'm not interested in virtual machines for network related services but rather in small embedded systems where the user might need to use just one single application for the entire board lifetime. Security wouldn't be a concern there, assuming the user already has complete access to the hardware, that layer could be moved outside of the box by inspecting and firewalling the network traffic. If yes, does anyone know if there is/will be any automated way say to analyze a software, example: a .deb package containing an user software, then build a list of dependencies and create a bootable image for the requested architecture which will run just that software?
- Maakuth 7y agoThis is how embedded linux systems are built already, AFAIK. There are vendors, such as MontaVista, that provide a toolchain to build images in this manner. There are free software alternatives too to do that, such as Embedded Gentoo.
- vidarh 7y agoIt doesn't really make much difference in terms of size in itself, I suspect. You're still carrying along the whole kernel and the application code, they're just linked together into a single binary. You can certainly then trim things down a lot by stripping the kernel to its bones by removing drivers etc., but you can do that even without linking it together, and you can go a lot further than what the normal kernel configuration mechanism allows if you're prepared to strip out functionality you know you'll never need, but I'm hard pressed to think of approaches like that which you can apply to the kernel that will only work if you move your application code to kernel space. In the past I've brought up Linux on embedded boards with 4MB RAM and 4MB flash for both the kernel and the application, and we gained most by linking statically to a smaller libc on the user side and stripping out drivers on the kernel side, ditching bash for ash (currently dash) and using a stripped down barebones init. We could have ditches ash and init as well and booted straight into the app, but ash made debugging and testing slightly easier and we could fit it in. 4MB at the time was not even a challenge. I think the gap from something that you can fit a Linux kernel and an app mashed together into a unikernel up to being able to run them separately will be very small, so in most instances if you can make it work as a unikernel odds are you can produce a nearly a small solution without it. Going full on unikernel potentially costs you a lot of resilience against failure, it's not just about security. The main reason for wanting to do it is to reduce the amount of context switches and data copying to reduce latency.
- fulafel 7y agoCan anyone commnent on how this approach differs from LinuxKit? Or is the LinuxKit plan at all unikernel (MirageOS) based anymore? I hope people will only use unikernels with memory-safe languages or otherwise well sandboxed runtimes.
- eyberg 7y agoIt's worth noting that Linux itself is written in C (not rust).
- rwmj 7y agoOh, it's a paper I am (rather peripherally) involved with. If you have questions then ask away, although it's possible I might not have all the answers ... Edit: In case you can't read the paper there's a copy here: https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-hotos19.pdf https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-h... (Thanks anonymousDan in the comments below for linking to it)
- antsoul 7y agoHow do they relate to microkernels ? Since last HN post about GNU Hurd, I'm wondering why not try to join this Free microkernel project instead of looking in another kernel's direction ?
- panpanna 7y agoArchitecturally, microkernels and unikernels are direct opposites. Unikernels strive to minimize communication complexity (and size and footprint but that is not relevant to this discussion) by putting everything in the same address space. This gives them many advantages among which performance is often mentioned but ease of development is IMHO equally important. However, the two are not mutually exclusive. Unikernels often run on top of microkernels or hypervisors.
- rwmj 7y agoRight answer. While a unikernel might be structured to some extent internally into modules, it's basically all linked into a single blob and running in a single address space. Some programming languages may support some kind of PL-level separation, but there is no hardware enforcement. In the case of what Ali is doing because all the code is written in C (it's all Linux, glibc and memcached) there is neither software nor hardware separation internally.
- ratmice 7y agoTo give a list of these PL-level separation mechanisms, and their languages, these are the kernels I know of that work in this fashion. Spin OS (Modula-3), Singularity (Sing#), Midori (M#), TockOS (safe rust). As far as I know TockOS is the only one I know of which has some form of both PL-level and hardware enforcement of separation, albeit on an MPU rather than a full MMU, PL-level for kernel modules, and an MPU protected userspace. I at least think it is worth addressing that none of these separation mechanisms are actually mutually exclusive.
- mtgx 7y agoI'd rather see unikernels written in Rust or any other memory-safe language. If we're going to start from scratch with this, let's do it right this time. 70%-90% of bugs seem to be memory-related. Let's end that with a little bit of effort upfront for much fewer headaches in the long-term. https://twitter.com/LazyFishBarrel/status/1153410070929203201 https://twitter.com/LazyFishBarrel/status/115341007092920320... https://www.zdnet.com/article/microsoft-70-percent-of-all-security-bugs-are-memory-safety-issues/ https://www.zdnet.com/article/microsoft-70-percent-of-all-se... https://security.googleblog.com/2019/05/queue-hardening-enhancements.html https://security.googleblog.com/2019/05/queue-hardening-enha...
- rwmj 7y agoThe entire point of this paper is not to start over from scratch, but to reuse existing software (Linux and memcached in this case), and fiddle with the linker command line and a little bit of glue to link them into a single binary. If you want to start over from scratch using a safe language then see MirageOS. Edit: I don't mean to say you have to use C for the "userspace" part of this. It should be possible -- in future -- to use any language for that. However at the end of the day you'll still be linking that with Linux (written in C) and glibc into a single binary that runs in one address space.
- edwintorok 7y agoSo this would be basically like rumpkernels but with Linux instead of BSD?
- rwmj 7y agoIt's kind of the opposite (AIUI). Rumpkernels run kernel code in userspace. We run userspace code and the kernel in kernel space.
- panpanna 7y agoWe need Rust to mature a bit before we can do that. Everything looks great on surface but as soon as you start doing low level coding a lot of issues pop up and you need the nightly compiler and xargo and some undocumented library for the boot sequence and whatnot. TL;DR: great idea. Let's just wait 2 years
- deleted 7y ago[deleted]
- ggm 7y agoLong ago, an OS I worked inside had a tool which took shared library indirect calls, and wired them out so you did not have a nested indirect function call cost, but went direct from the call to the backend without the pass-through the null function mapping into the specific .so library. It made things faster by one layer of the onion skin. Strip, as a UNIX tool removes the textual crap in a debuggable binary, to leave behind only the bits you needed. It feels to me like if you can do runtime code call checks, and confirm which actual calls you make, then stripping the bits of libc and associated libraries out, being left with only the strictly required calls, and then by extension the syscalls, and then by extension the kernel elements, is actually possible a lot of the time. So, library -> reduced library -> reduced calls -> reduced syscalls -> reduced kernel state is a sequence or set or something, of applied minimisations which can be done, if you can predict all the call paths in your code and their dependencies. But then its not a generic OS any more: its an application specific binding to a general purpose CPU. Why not go all the way, and work into the ALU and remove the bits you don't need? And then go into the micro code, and the associated FPGA, and everything else..
- sp332 7y agoYou have to make sure your test cases exercise every possible code path. German demogroup .theprodukkt made a demo one time, .kkreiger, a first-person shooter game in 96kb. They used a technique like this to strip every line of source code that was not hit during a playtest. But the playtester didn't press the up-arrow in the main menu, so that doesn't work. More importantly they didn't get hit by any bullets, so in the final release the player is invulnerable because the code that handles damage was never compiled. Edit: more informative link https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for-madmen/ https://fgiesen.wordpress.com/2012/04/08/metaprogramming-for...
- ggm 7y agoYes. And its almost impossible to do this without a huge investment in time and effort, except: If you use languages with FP, and are rigorous, then I believe (possibly wrongly) its actually somewhat simpler to understand because the style of coding in FP exposes much of this "better" than in classic imperative coding So yes, I think you're right: great dream, hugely hard to do, if not actually impossible in most cases near as damnit, but there are places where program proofs take you to this, Military/Space/Medical needs to know the code calls don't have unexpected outcomes. And language choices which express as "how" you express code, can also help. Probably not enough.
- vectorEQ 7y agoall unikernels so far suck ALOT regarding security. wouldn't recommend any of them to run in any production environment unless you want to lose all of your data. NCC group did a good article on unikernels and how crap they are if you want to know. A better idea, like already suggested would be to create an OS which builds itself according to a target application and system, to host that application on said system specifically. that would reduce about as much code, but keep potential security mechanisms like aslr, stack protection , user / kernel separation etc. in tact. (now kernels / oses build to target system ,but not application! -> application would only use subset of kernel, and thus kernel can be built to target application, reducing kernel to whats needed). don't try to be cheap for performance and skip security, we're not in the damned 80s anymore. /endrant
- arbarb 7y agoIf you read that article, they tested two unikernels and the key point was that unikernels don't implement security mechanisms present in a normal OS. This Linux Unikernel, otoh, does not delete any of that security functionality. The boot up code is the same, it just calls a specific code instead of bringing up the general userspace. Unikernels and security don't have to be mutually exclusive.
- pizzazzaro 7y agoI wonder if there's a specific unikernel linux kernel that we can "diff -u /usr/src/foo /usr/src/bar > gimme_dat.patch" against its upstream. I'd love to see if we could get some KSPP/hardened/etc-derived unikernels.
- roddux 7y agoI'd argue the Mirage Unikernel (built almost wholly in OCaml) is one of the most robust platforms out there. The NCC paper you talk about looks at two rather old fashioned unikernels in isolation. I don't think the idea of unikernels should be discarded because the current implementations are slightly lacklustre -- it just shows that there's a fair way to go yet. >[..] A better would be to host that application and reduce the kernel to whats needed This is a unikernel.
- ElijahLynn 7y agoThis is a paid article. Am I missing something?
- iamnotacrook 7y agoI'm getting this link: https://dl.acm.org/citation.cfm?id=3321445 https://dl.acm.org/citation.cfm?id=3321445 The link the PDF contains IDs and stuff which differ each time I visit, which seems unnecessary. Maybe get it from here: https://www.reddit.com/r/programming/comments/cb17mn/read_a_paper_unikernels_the_next_stage_of_linuxs/ https://www.reddit.com/r/programming/comments/cb17mn/read_a_... which links to: https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-hotos19.pdf https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-h...
- gandalfgeek 7y agoFor those who don't want to read the whole paper, I'd made a short ten minute explainedy video going through it: https://youtu.be/3NWUgBsEXiU https://youtu.be/3NWUgBsEXiU
- dwheeler 7y agoThis is really cool, though it looks like the security impact is nuanced. Unikernel plus: A lot of code, including code with vulnerabilities, can disappear. So some vulnerabilities disappear. Unikernel minus: The unikernel and application don't have any security isolation, since the kernel is essentially linked into the appl. As a result, the application has all the privileges of its virtual machine (not the subset that would normally be imposed by the kernel), which would normally be more privileges than it really needs. So any remaining vulnerabilities can have a more devastating effect. That trade-off in terms of security is really hard to evaluate. But the kinds of performance improvement shown here, with relatively modest changes, is a really really big deal. So I'd expect a lot of people to investigate this further; it certainly seems promising.
- edwintorok 7y agoIf you write your unikernel in a high-level language then you can mitigate some of your security concerns. The unikernel should only have access to the data it needs to, so doing multi-tenancy or isolation inside the unikernel is probably not what you want: just spin up a new unikernel for each tenant/thing you want to isolate. The hypervisor would provide isolation in this case. There is a downside, if there is a vulnerability the exploit can probably make hypercalls straight to the hypervisor, but a hypervisor can have less of an attack surface than a full OS.
- compilers 7y agoAnil Madhavapeddy's work on unikernels is also quite interesting to look at.
- pkilgore 7y ago> a support layer for Emacs The best part of this joke was that I can't tell if it's shade or earnest appreciation.