22 ms·
Searching for the killer app of unikernels
- andrewstuart 9y agoUnikernels don't need a killer app, they need millisecond boot and teardown time from the major cloud vendors, along with suitable pricing. Unikernels are a great technology and very much ready for prime time, but unless cloud vendors provide the millisecond boot/teardown/pricing, Unikernels will remain nothing more than a niche special interest technology. All the technology is in place for Unikernels to take off once the cloud vendors get behind them. I wrote Bootrino specifically for the purpose of booting Unikernels (and a whole host of other exotic tiny operating systems) on the major clouds and it works fine - you can see a demo here: https://www.youtube.com/watch?v=zJhxmVlR46c https://www.youtube.com/watch?v=zJhxmVlR46c (Bootrino not released to public as yet). However, the real draw for Unikernels is short lifetime function as a service type applications.
- microcolonel 9y agoTo add to this, I think attention needs to shift from "booting" unikernels to sorta... "instantiating" them. Booting involves enumeration, initialization, and teardown of devices. If unikernels could be resumed from a compact memory image without having to enumerate or initialize any (virtual) devices then I figure things would get a lot easier. For what it's worth, though, there's a lot of value in long-running unikernels. There are good reasons not to pay even the cost of a memcpy of the unikernel image, especially when it comes to making good use of system resources. I'm mostly interested in unikernels on bare metal (or something close, like device-specific PCIe passthrough). There's a lot of potential in an application having full and exclusive access to a hardware platform; it means that your application can make creative use of isolation primitives, be aware of its true resource limits, and choose scheduling and work sharing primitives which make sense for the application. In many ways, good unikernel environments are similar to DOS, I'm amused to see the resurgence.
- andrewstuart 9y agoCheck out Solo5. There's nothing much more in there than a virtual network driver and a virtual disk driver. Solo5 acts as the host for MirageOS and maybe other Unikernels, or even just for raw C++ code.
- noway421 9y agoDo they or would they support libc? And how to go around compiling other important libs which would be required to get python/node/go working?
- fiokoden 9y agoWhy do you need a unikernel for those? Just run your OS from RAM.
- lmm 9y agoThere are plenty of standalone libcs that can be compiled like any other code, they don't need special support from the host. Likewise you could effectively statically link any libraries you wanted into your unikernel image - though if you just build the normal interpreter then you're not getting a lot of advantage out of having a unikernel, part of the point is to build your program in a more direct form without so many extra layers of translation.
- justincormack 9y agoIt supports at least IncludeOS too. It has 1ms boot times.
- andrewstuart 9y agoBe nice to see the clouds support those juicy quick boot times.
- _pmf_ 9y agoResuming a suspended instance should work, no?
- deleted 9y ago[deleted]
- bluejekyll 9y agoA question that constantly reoccurs for me is this: who does the scheduling of threads and/or processes in a unikernel? I know unikernels are single process, but are they also intended to be single core? If not, without the kernel scheduler, what is providing that function to the unikernel? Is it just an optional module, say in the case of rumprun?
- yazaddaruvala 9y agoThe whole point is there are many ways to do it and the application knows best. The API specifics are probably different but it's just setting some registers and the the program counter on that CPU core. The code likely looks very similar to a context switch.
- derefr 9y agoConsider that any complex language runtime (e.g. Erlang's BEAM, the JVM, etc.) has its own scheduler. If you convert such a runtime into a unikernel (such as in Erlang on Xen), the runtime scheduler is "the" scheduler. Besides languages, though, plenty of complex applications have their own schedulers. Databases, for example. Postgres-as-a-unikernel would make a lot of sense, because it's already doing its own equivalent to everything that OSes do. If your program is complex enough, it likely will already have an "optimized" form of internally-managed shared memory/message-passing multithreading (with core affinities et al) to allow it to allocate tasks to cores better than the OS can without the "insider knowledge" of what it's trying to accomplish. At that point, your program will end up with a scheduler "inside" whether you set out to build one or not. On the other hand, if your program isn't that complex, then it's very likely you'll get better performance out of your code by just making each process into a single-core unikernel and then packing instances onto however many vCPUs the hypervisor host offers.
- bluejekyll 9y agoI don't know much about the other Runtimes that you mention, but correct me where I'm wrong: In the case of the JVM, green threads were dropped a long time ago, and under the current paradigm, native threads are used (on all platforms?). So doesn't this need to be written into the JVM? Not impossible of course (or provided by the unikernel wrapper as it is in OSv). Similarly with Databases, Oracle and Postgres both use a process per connection client/server model. So a unikernel for these would be impractical without rearchitecting those to run in that kind of environment. I was initially very excited for Unikernels, but because of this question (and others), I've really started becoming less enamored by them. I'm sure there are many places where they can be put to use, but for most of the software that I'm familiar with, there are still a bunch of kernel services that are quite handy. At what point of building these services into your unikernel/userspace application do you cross-over to basically building a new kernel. It really feels like the micro-kernel architecture would be a better fit in many of these cases. And yes, I understand that we can think of unikernels as VMs based micro-kernel services, but they have more isolation than traditional variations specifically allowing them to be used in AWS, etc. But still...
- fulafel 9y agoA malicious DRM operating system running in my browser sounds pretty bad.
- amluto 9y agoIt's never been clear to why very simple unikernels are better than running essentially the same code as a normal user program.
- andrewstuart 9y agoThere's a whole zoo of "little operating systems" out there that more or less have unikernel-like properties. Some more closely meet the definition of a Unikernel, others are more like stripped down Linux, others are more exotic OS's that are nothing more than a network driver and disk driver in a virtual machine, such as Solo5. I got Bootrino (see elsewhere) to boot all of these types of operating systems including embedded Linux (Yocto), Solo5, TinyCore, Alpine, and some other weird run-from-RAM OS configurations. I got MirageOS running on all of the above, so unikernels can fit within other OSs or live on their own.
- wmf 9y agoA user process running on an OS running on a hypervisor has certain overhead, so you can either remove the hypervisor (native containers) or you can remove the OS (unikernels). Arguably the hypervisor has a smaller attack surface than an OS. Also, there's a small set of programs that don't get along with the abstractions provided by OSes and run better on a virtual hardware abstraction instead.
- first_amendment 9y agoUnfortunately containers aren't considered secure enough for malicious users while VMs are.
- AstralStorm 9y agoNeither are most unikernels. Especially the weak points are file system and network handling. Generally the security of VMs is enforced by hardware and improved by limited attack surface - few drivers to audit, limited communication APIs. You can do the same with general purpose OS by cutting options.
- ww520 9y agoWould IoT be the killer app for unikernel? Since most IoT apps are simple single purpose app and require fast boot time.
- stingraycharles 9y agoAren't most of these developed using realtime kernels on platforms such as VxWorks ? (last time i looked into hardware dev was 10 years ago, so my knowledge might be outdated)
- ww520 9y agoI really have no clue since I haven't done embedded dev for a long time (VxWorks proprietary?). Just throw ideas out and see what stick. :)
- icebraining 9y agorequire fast boot time Why? Aren't they either usually on, or at most sleeping? EDIT: no → on
- ww520 9y agoIt's powered on and it's expected to be up right the way?
- icebraining 9y agoSorry, I mistyped in my previous post. Aren't IoT devices usually running all the time anyway?
- ww520 9y agoIt's the initial user interaction. Users usually expect appliances to be powered on and be running right the way. If it takes 30 seconds to start, the user is not sure whether it's not working or it's just initializing. These things don't have great visual feedback to start with.
- MrTonyD 9y agoI wonder if functional programming might be something to implement with a "flock" of unikernels? I'm not even sure what I mean, but it seems like decomposing into unikernels might allow entirely new architectures. (Perhaps analogous to a VM allowing better use of hardware resources, unikernels might allow complete abstraction away from, and across, hardware??)
- Ultimatt 9y agoYou mean like some kind of "lambda" maybe in the Amazon cloud.... https://aws.amazon.com/lambda https://aws.amazon.com/lambda
- zvrba 9y ago> This VM would not have an IP stack but a small, secure, and well defined interface. It can basically do two or three things to communicate with the host: [...] All of these mechanisms can be transported out of the VM through hypercall (like VMCALL). So how is this better than using existing IPC mechanisms and using the "user" abstraction to separate privileges? And how is this more secure then IPC to a process running under another user given > The browser takes a careful look at the pointer and the data it is pointing at. This code will need to be properly vetted as a buffer overflow here could potentially allow the unikernel to execute code outside its hardware prison. (The long list of CVEs tells us how good programmers are at "properly vetting" buffers).
- noway421 9y agoAs far as i understood, it is single point of failure and something that browser developers would need to ensure, not end developers. It's possible to cover that specific part with extensive testing and fuzzing to make sure it's robust.
- lmm 9y ago> So how is this better than using existing IPC mechanisms and using the "user" abstraction to separate privileges? It's a smaller, cleaner, interface that's been designed from day 1 to be secure, rather than having security awkwardly partially retrofitted. > (The long list of CVEs tells us how good programmers are at "properly vetting" buffers). Right, which is exactly why it's vital to keep those interfaces as small as possible, rather than having dozens of different IPC mechanisms for vulnerabilities to hide in.
- erikb 9y agoI don't know, it's pretty unlikely at this point that anything will replace Linux as The Kernel. It's like replacing Google as the search engine. On top of that users are currenlty moved away one additional abstraction layer from the kernel, to work with apps inside of containers (see Android, Kubernetes). The OS underneath becomes a static one-version-one-set-of-packages layer. That reduces the amount of people who can even decide to exchange a kernel for another, and these are mostly people who are rewarded for stability instead of experiments. Who would still like to write Android apps, if Android would start to crash more often or change its API drastically? This is a battle that can't be won. Let's move on to one that is still undecided, e.g. which container engine will finally win. Although Docker seems to be the most likely favorite already at this point, I'd still like to see if nspawn will do a systemd move and just replace the alternatives.
- coldtea 9y ago>I don't know, it's pretty unlikely at this point that anything will replace Linux as The Kernel Depending on the domain (embedded, IoT, industrial, medical, etc), tons of other kernels are used (and prevail). We're not necessarily talking web services here. >This is a battle that can't be won. Let's move on to one that is still undecided, e.g. which container engine will finally win. That's a totally different concern.
- AstralStorm 9y agoThose tend to be tiny realtime kernels, single core, limited memory size etc. The solutions used there tend to not scale to bigger hardware.
- erikb 9y agoYes there are always special cases that use special tools. But add all computers in data centers + all smart phones and you may have a factor bigger than 1000:1. For the broad mass of people it's totally irrelevant that there is an edge case that makes power use of special kernels. And no the container war is not a totally different concern. That's the current main concern which influences most people and how software will be administrated (and therefore developed) in the next 20 years.
- bubblethink 9y agoThe usecase of executable code in the browser (including DRM) sounds quite similar to what SGX is trying to do.
- perbu 9y agoThere is no opposition here. You can, unless I'm missing something, run a unikernel within a SGX enclave. They are complementary. Since a unikernel provides a self-contained system it could provide very clear and reasonably secure interfaces to the enclave.
- _pmf_ 9y agoI think unikernels are a use case where something like Rust could really, really shine without any feelings of NIH or reinventing the wheel.
- buiductuan182 9y agowell. that is great DucTuan: https://www.quangminhhanoi.com/ https://www.quangminhhanoi.com/
- buiductuan182 9y agohttp://suachuadieuhoa.info.vn/ http://suachuadieuhoa.info.vn/ https://dailydieuhoataihanoi.blogspot.com/ https://dailydieuhoataihanoi.blogspot.com/
- Ceezy 9y agoMirageOS hello world app is pretty ugly. Everything remains to be done before being understood by a wider public.
- perbu 9y agoI think few of us like OCaml for its syntactical beauty. People are drawn to MirageOS for its semantics. Type safety, etc. However, there are other Unikernels. IncludeOS uses modern C++ so the Hello World looks like this: #include <os> int main() { printf("Hello World!"); }
- talex5 9y agoFor reference, the ugly OCaml equivalent unikernel is: let () = print_endline "Hello, world!" (from http://roscidus.com/blog/blog/2014/07/28/my-first-unikernel/ http://roscidus.com/blog/blog/2014/07/28/my-first-unikernel/)
- justaaron 9y agodeterministic low-latency audio?
- lazyjones 9y agoJust call this "cloud serverless architecture" and you're done... (yes, I know this already exists and it's container-based, but similar enough)
- fulminatorz68 9y agoThis culture of optimization and mutation is pretty out of control.