6 ms·

MirageOS: A Unikernel For The Xen Hypervisor

99 points by godisdad 12y ago

derefr 12y ago
See also: Erlang on Xen (http://erlangonxen.org/ http://erlangonxen.org/).
- alco 12y ago
  Also http://corp.galois.com/halvm http://corp.galois.com/halvm. Those three have in common the trait that they support single language runtime: Erlang for Erlang on Xen, OCaml for MirageOS, and Haskell for HaLVM.
  - derefr 12y ago
    Which seems sensible; if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network. It's the natively-compiled, statically-linked equivalent to putting your runtimes in Docker containers. I was thinking, earlier, that this approach could be taken even further: rather than relying on Xen, it'd be nice to extract the drivers from the Linux kernel into their own project, an exokernel library with a defined ABI. Platform runtimes like Erlang, OCaml, Haskell, etc. could each build a "complete bare-metal kernel-binary" version of themselves, simply by linking to that exokernel library. (And Linux could obviously move over to consuming the exokernel library itself. I guess a POSIX environment would technically just be the "complete bare-metal C kernel-binary.")
    - drdaeman 12y ago
      > if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network Those "components" and "virtualized network" really resemble "processes" and "IPC" to me. Except that in MirageOS/Foo-on-Xen/etc each process has its own device drivers. And, indeed, those beg either for putting into a sort of shared component. But... aren't we almost reinventing microkernels here? With the only distinction of using shared libraries instead of server processes?
      - derefr 12y ago
        Yes, we are. I wouldn't say re-inventing, though. Hypervisors are effectively an optimization of microkernels: modern ISAs (e.g. x86-64) have special instructions to make "hypercalls" faster than plain-old context switches, so any modern implementation of a microkernel would be forced by practicality to act as a hypervisor. Though, there's another advantage that comes specifically from thinking of each process as a separate "machine": each process gets to participate fully as an Internet peer, with its own unique, public-routable IPv6 address.
    - justincormack 12y ago
      You can extract the drivers from the NetBSD kernel into exactly that, its called rump kernel http://www.rumpkernel.org http://www.rumpkernel.org and it can run on Xen among other places see https://github.com/rumpkernel/rumpuser-xen https://github.com/rumpkernel/rumpuser-xen or just in userspace under Linux say - it is getting easier to run any application on it, there is still a bit more work needed (thread support is not there yet). Working on it though, and it should get easier to build and run application code over the next few months. One advantage of using a BSD rather than Linux is licensing, as linking Linux kernel code into your application will probably force you to release as GPL. NetBSD also has an initial portability advantage, as you could always cross compile it.
- ams6110 12y ago
  Also https://github.com/kerlnel https://github.com/kerlnel but it doesn't appear to be active.
andrewstuart 12y ago
This is an awesome idea. Every now and then a revolutionary concept comes along that points the way to all future computing. This is the future of any public facing computer. You must understand the concepts here if you are involved with web development at any level. If you haven't yet wrapped your head around this, then you should. The key concept here is about security. Imagine you could compile your web application/website into an application that has nothing else but the required network drivers and minimal hardware drivers needed to execute. That is (as far as I can tell from a brief look) what Mirage does. Using such an approach there's nothing for hackers to log in to - no shell to use, no other insecure cruft on the system potentially leaving holes. The attack surface of such a system is extremely low relative to putting fully powered and configurable interactive operating systems onto the Internet. Hackers are showing that almost everything is hackable without expensive, constant, expert attention to security of every aspect of your systems. Even then zero day exploits expose systems within seconds or minutes of the exploit becoming known. This is not to say that such systems cannot be hacked, but rather to say that in such an approach the attack surface is dramatically lower than in current approaches to deployment. Perhaps more likely to change the way that people think about the concept of web application development in the longer term is that the application/server allows for the possibility of "high resolution server instances" in which a virtual machine can be started in milliseconds, perhaps to service only one query, and then vanish again. Instead of renting your instance from Amazon that runs per hour and does nothing most of the time, your infrastructure will be truly scalable with nothing running at all until its needed and then precisely as much computing power will be used as is required to meet the inbound demand. No longer will you need to "start a web server" which then chugs and rumbles into existence as all its various unneeded-at-most-times processes start. Instead, connect your application/server AMI to the Amazon spot instance marketplace.
- alco 12y ago
  That is partly true. But if there is nothing to hack into, there will also be nothing to log in to for an admin or an ops person. In the case of Erlang, for instance, it provides a built-in remote shell which allows one to connect to a running Erlang process, get multitude of information about the VM, loaded code, etc. As long as there is a convenient way to maintain a running system, that same way can be used for hacking.
harrystone 12y ago
This seems like the rube goldberg version of chroot to me.
p1mrx 12y ago
There was already a "MirageOS", for TI calculators: http://www.ticalc.org/archives/files/fileinfo/139/13949.html http://www.ticalc.org/archives/files/fileinfo/139/13949.html
- Sir_Cmpwn 12y ago
  One with a frustrating name. It's a shell, not an OS. One of my projects is an OS for TI calculators, and loads of people don't actually understand what that means.
  - cube_yellow 12y ago
    You're the Knight OS guy?
gtirloni 12y ago
I understand how using type-safety and streamlined modular systems might be good for security, but I fail to see the point in letting developers be closer to the bare metal. All the security abstractions built into traditional OSes are not there for no reason. Why should I trust the OCaml stack more than the alternatives? This looks like something a really educated/disciplined company could use for creating some crazy cool infrastructure. It's hard to see it taking off for the average Joe.
- amirmc 12y ago
  I think you answered your own question (to some extent). Rewriting the necessary protocols/libraries in a type-safe manner already provides an improvement over the current systems. In addition, (with the standard approach) it's quite hard for a developer to reason about the whole stack they're using and privilege escalation bugs are particularly irksome. Writing the applications with the approach Mirage takes means that there is less code to reason about. An anecdote I use when describing the benefits is the story of a smart fridge that got hacked and became part of a botnet sending spam emails. Why did that fridge even have code that allowed it to send email? It wasn't necessary for its functioning. We should write software differently if were going to be deploying it to 10x the number of devices compared to today. More personally, I worry about the software that's going to find it's way into the embedded health devices of the future (cf pacemakers). These devices will inevitably be 'connected' and I want to make sure that the code they use is safe and secure.
  - brokenparser 12y ago
    > Why did that fridge even have code that allowed it to send email? If it has a remote code execution vulnerability, it's trivial to make it send spam (or do all kinds of things) whether a MUA was already present or not.
- godisdad 12y ago
  OP here. The Unikernel paper someone linked about has more, but I think the main ideas are: incidental complexity mitigation and performance. It's much easier to reason about a bunch of OCaml code, so say the authors, than it is to understand the interaction between independently changing pieces of your "stack" that are written in different languages and integrate in disparate ways.
bjackman 12y ago
I don't work in a sphere that would make this relevant to me, but I think this paradigm is probably the future! "General purpose operating systems are dead!", proclaimed bjackman. Anyway, also see: OSv: http://osv.io/ http://osv.io/ similar idea but with the JVM. I believe some pretty well known Clever People are working on it - can't remember who, though.. anyone know?
- gtirloni 12y ago
  What looks like an alternative in .NET: http://cosmos.codeplex.com/ http://cosmos.codeplex.com/
listic 12y ago
Is there anything similar for KVM?
- justincormack 12y ago
  I don't think so. KVM provides something much more like raw hardware (provided by qemu) while Xen provides a slightly higher level interface, and an example "OS", mini-os to run on it. Your best bet would be to use a virtio based driver set and a minimal initialisation but it will still have to do more.
  - edwintorok 12y ago
    According to the FAQ it should be possible to port it to KVM: http://openmirage.org/wiki/faq http://openmirage.org/wiki/faq
    - justincormack 12y ago
      Sure its possible. You have to also remember there is a historic connection between the Cambridge OCaml people and Xen, with some of Xen being written in OCaml.
amirmc 12y ago
For people interested in this I recommend the ACM article on Unikernels [1]. It's a good overview of the approach and the benefits/trade-offs. Then take a look at http://nymote.org http://nymote.org to see the kind of things we want to start building with this approach. [1] http://queue.acm.org/detail.cfm?id=2566628 http://queue.acm.org/detail.cfm?id=2566628
ForHackernews 12y ago
There's only room for one MirageOS in my heart, and it runs on the TI-83+ http://www.detachedsolutions.com/mirageos/ http://www.detachedsolutions.com/mirageos/
bsaul 12y ago
Tried to read the paper mentionned on the website, and wondered : does "libOS" /unikernel concept means that the application is built as a kernel extension, and thus runs in kernel land ( so no context switching occures) ? Is this an explanation for the performance gains ?
- edwintorok 12y ago
  When you link the application with the Mirage runtime (what you refer to as "libOS") you create a new kernel. You then run that kernel directly on Xen (like you would run a Linux domU), so yes they run in kernel land. I'm not convinced that 'Xen + Mirage unikernel' has a performance advantage over 'Linux baremetal + Mirage direct mode', but I definetely see an advantage compared to 'Xen + Linux domU + Mirage direct mode'.

99 points by godisdad 12y ago

derefr 12y ago

See also: Erlang on Xen (http://erlangonxen.org/ http://erlangonxen.org/).

alco 12y ago
Also http://corp.galois.com/halvm http://corp.galois.com/halvm. Those three have in common the trait that they support single language runtime: Erlang for Erlang on Xen, OCaml for MirageOS, and Haskell for HaLVM.
- derefr 12y ago
  Which seems sensible; if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network. It's the natively-compiled, statically-linked equivalent to putting your runtimes in Docker containers. I was thinking, earlier, that this approach could be taken even further: rather than relying on Xen, it'd be nice to extract the drivers from the Linux kernel into their own project, an exokernel library with a defined ABI. Platform runtimes like Erlang, OCaml, Haskell, etc. could each build a "complete bare-metal kernel-binary" version of themselves, simply by linking to that exokernel library. (And Linux could obviously move over to consuming the exokernel library itself. I guess a POSIX environment would technically just be the "complete bare-metal C kernel-binary.")
  - drdaeman 12y ago
    > if you want to write different components in multiple languages, each component can be its own VM and they can communicate over the (virtualized) network Those "components" and "virtualized network" really resemble "processes" and "IPC" to me. Except that in MirageOS/Foo-on-Xen/etc each process has its own device drivers. And, indeed, those beg either for putting into a sort of shared component. But... aren't we almost reinventing microkernels here? With the only distinction of using shared libraries instead of server processes?
    - derefr 12y ago
      Yes, we are. I wouldn't say re-inventing, though. Hypervisors are effectively an optimization of microkernels: modern ISAs (e.g. x86-64) have special instructions to make "hypercalls" faster than plain-old context switches, so any modern implementation of a microkernel would be forced by practicality to act as a hypervisor. Though, there's another advantage that comes specifically from thinking of each process as a separate "machine": each process gets to participate fully as an Internet peer, with its own unique, public-routable IPv6 address.
  - justincormack 12y ago
    You can extract the drivers from the NetBSD kernel into exactly that, its called rump kernel http://www.rumpkernel.org http://www.rumpkernel.org and it can run on Xen among other places see https://github.com/rumpkernel/rumpuser-xen https://github.com/rumpkernel/rumpuser-xen or just in userspace under Linux say - it is getting easier to run any application on it, there is still a bit more work needed (thread support is not there yet). Working on it though, and it should get easier to build and run application code over the next few months. One advantage of using a BSD rather than Linux is licensing, as linking Linux kernel code into your application will probably force you to release as GPL. NetBSD also has an initial portability advantage, as you could always cross compile it.
ams6110 12y ago
Also https://github.com/kerlnel https://github.com/kerlnel but it doesn't appear to be active.

andrewstuart 12y ago

This is an awesome idea. Every now and then a revolutionary concept comes along that points the way to all future computing. This is the future of any public facing computer. You must understand the concepts here if you are involved with web development at any level. If you haven't yet wrapped your head around this, then you should. The key concept here is about security. Imagine you could compile your web application/website into an application that has nothing else but the required network drivers and minimal hardware drivers needed to execute. That is (as far as I can tell from a brief look) what Mirage does. Using such an approach there's nothing for hackers to log in to - no shell to use, no other insecure cruft on the system potentially leaving holes. The attack surface of such a system is extremely low relative to putting fully powered and configurable interactive operating systems onto the Internet. Hackers are showing that almost everything is hackable without expensive, constant, expert attention to security of every aspect of your systems. Even then zero day exploits expose systems within seconds or minutes of the exploit becoming known. This is not to say that such systems cannot be hacked, but rather to say that in such an approach the attack surface is dramatically lower than in current approaches to deployment. Perhaps more likely to change the way that people think about the concept of web application development in the longer term is that the application/server allows for the possibility of "high resolution server instances" in which a virtual machine can be started in milliseconds, perhaps to service only one query, and then vanish again. Instead of renting your instance from Amazon that runs per hour and does nothing most of the time, your infrastructure will be truly scalable with nothing running at all until its needed and then precisely as much computing power will be used as is required to meet the inbound demand. No longer will you need to "start a web server" which then chugs and rumbles into existence as all its various unneeded-at-most-times processes start. Instead, connect your application/server AMI to the Amazon spot instance marketplace.

alco 12y ago
That is partly true. But if there is nothing to hack into, there will also be nothing to log in to for an admin or an ops person. In the case of Erlang, for instance, it provides a built-in remote shell which allows one to connect to a running Erlang process, get multitude of information about the VM, loaded code, etc. As long as there is a convenient way to maintain a running system, that same way can be used for hacking.

harrystone 12y ago

This seems like the rube goldberg version of chroot to me.

p1mrx 12y ago

There was already a "MirageOS", for TI calculators: http://www.ticalc.org/archives/files/fileinfo/139/13949.html http://www.ticalc.org/archives/files/fileinfo/139/13949.html

Sir_Cmpwn 12y ago
One with a frustrating name. It's a shell, not an OS. One of my projects is an OS for TI calculators, and loads of people don't actually understand what that means.
- cube_yellow 12y ago
  You're the Knight OS guy?

gtirloni 12y ago

I understand how using type-safety and streamlined modular systems might be good for security, but I fail to see the point in letting developers be closer to the bare metal. All the security abstractions built into traditional OSes are not there for no reason. Why should I trust the OCaml stack more than the alternatives? This looks like something a really educated/disciplined company could use for creating some crazy cool infrastructure. It's hard to see it taking off for the average Joe.

amirmc 12y ago
I think you answered your own question (to some extent). Rewriting the necessary protocols/libraries in a type-safe manner already provides an improvement over the current systems. In addition, (with the standard approach) it's quite hard for a developer to reason about the whole stack they're using and privilege escalation bugs are particularly irksome. Writing the applications with the approach Mirage takes means that there is less code to reason about. An anecdote I use when describing the benefits is the story of a smart fridge that got hacked and became part of a botnet sending spam emails. Why did that fridge even have code that allowed it to send email? It wasn't necessary for its functioning. We should write software differently if were going to be deploying it to 10x the number of devices compared to today. More personally, I worry about the software that's going to find it's way into the embedded health devices of the future (cf pacemakers). These devices will inevitably be 'connected' and I want to make sure that the code they use is safe and secure.
- brokenparser 12y ago
  > Why did that fridge even have code that allowed it to send email? If it has a remote code execution vulnerability, it's trivial to make it send spam (or do all kinds of things) whether a MUA was already present or not.
godisdad 12y ago
OP here. The Unikernel paper someone linked about has more, but I think the main ideas are: incidental complexity mitigation and performance. It's much easier to reason about a bunch of OCaml code, so say the authors, than it is to understand the interaction between independently changing pieces of your "stack" that are written in different languages and integrate in disparate ways.

bjackman 12y ago

I don't work in a sphere that would make this relevant to me, but I think this paradigm is probably the future! "General purpose operating systems are dead!", proclaimed bjackman. Anyway, also see: OSv: http://osv.io/ http://osv.io/ similar idea but with the JVM. I believe some pretty well known Clever People are working on it - can't remember who, though.. anyone know?

gtirloni 12y ago
What looks like an alternative in .NET: http://cosmos.codeplex.com/ http://cosmos.codeplex.com/

listic 12y ago

Is there anything similar for KVM?

justincormack 12y ago
I don't think so. KVM provides something much more like raw hardware (provided by qemu) while Xen provides a slightly higher level interface, and an example "OS", mini-os to run on it. Your best bet would be to use a virtio based driver set and a minimal initialisation but it will still have to do more.
- edwintorok 12y ago
  According to the FAQ it should be possible to port it to KVM: http://openmirage.org/wiki/faq http://openmirage.org/wiki/faq
  - justincormack 12y ago
    Sure its possible. You have to also remember there is a historic connection between the Cambridge OCaml people and Xen, with some of Xen being written in OCaml.

amirmc 12y ago

For people interested in this I recommend the ACM article on Unikernels [1]. It's a good overview of the approach and the benefits/trade-offs. Then take a look at http://nymote.org http://nymote.org to see the kind of things we want to start building with this approach. [1] http://queue.acm.org/detail.cfm?id=2566628 http://queue.acm.org/detail.cfm?id=2566628

ForHackernews 12y ago

There's only room for one MirageOS in my heart, and it runs on the TI-83+ http://www.detachedsolutions.com/mirageos/ http://www.detachedsolutions.com/mirageos/

bsaul 12y ago

Tried to read the paper mentionned on the website, and wondered : does "libOS" /unikernel concept means that the application is built as a kernel extension, and thus runs in kernel land ( so no context switching occures) ? Is this an explanation for the performance gains ?

edwintorok 12y ago
When you link the application with the Mirage runtime (what you refer to as "libOS") you create a new kernel. You then run that kernel directly on Xen (like you would run a Linux domU), so yes they run in kernel land. I'm not convinced that 'Xen + Mirage unikernel' has a performance advantage over 'Linux baremetal + Mirage direct mode', but I definetely see an advantage compared to 'Xen + Linux domU + Mirage direct mode'.