7 ms·

Next-Generation Cloud: The Rise of the Unikernel

132 points by sciurus 11y ago

Animats 11y ago
This is encouraging. A path to the post-C, post-Linux world is starting to become clear.
- lbarrow 11y ago
  Exactly. In my mind that's the best thing about these projects: the technical benefits are so amazing they might actually encourage people to look beyond the Linux-and-company systems stack. (Although, to be fair, most people run a Linux dom0.)
  - api 11y ago
    What's weird and funny is the collapse in complexity this represents. Linux, NT, BSD, and other modern kernels implement all kinds of complexity around user/group management, ACLs, security policies, file types and permissions, and so on. If we're moving to unikernels and containers and similar, all that has been replaced with stupid simple access control systems like API access tokens or login/password pairs for cloud service control panels. Your AWS access token is now a single security credential for your entire infrastructure. Of course I'm sure the cycle of reincarnation will continue. Things like Docker or the AWS API will grow in complexity until they at least subsume the entire feature set of Unix UID/groups, Unix/NT ACLs, SELinux, etc.
    - deleted 11y ago
      [deleted]
- pseudoProgress 11y ago
  I would argue that in a world where we've progressed past Linux and C, but still talk about using PHP, MySQL and Java, and other such middleware, including Ruby, Python, Perl and so forth, not much actual progress has been made after all. It would simply feel like standing still, for an overhwelming majority.
- otterley 11y ago
  To what end?
gima 11y ago
I love the idea, always have. Does a unikernel application need statically allocated amount of RAM or is there a way to have unikernel-programs allocate memory dynamically? Maybe this is where garbage-collected language would be useful.
- delinka 11y ago
  Seems to me a unikernel app would have access to all the RAM in the system. It is, after all, the kernel itself.
  - lbarrow 11y ago
    Yea but the hypervisor has to know how much physical memory to give the unikernel.
    - wtallis 11y ago
      The way it usually works is that the guest OS gets a certain allocation of RAM and if it doesn't want to use it all it can inform the hypervisor of which ranges don't need physical backing at the moment. From the guest's perspective this is often implemented using a "balloon" process/driver that reserves a large chunk of guest memory in order to return it to the hypervisor. It can also be handled by the guest having a more general memory hotplug capability.
- rdtsc 11y ago
  Look at a live version of Erlang on Xen. http://erlangonxen.org/ http://erlangonxen.org/ Press ESC on the front page and see the amount of GC ram reclaimed from Erlang's VM. Memory Usage: 12MB -- is also kind of a baffling number.
themartorana 11y ago
I was happy to see Clive, and I imagine a RUMP implementation or other unikernel that supports Rust won't be far away. Being able to compile Go or Rust to metal byte code is absurdly sexy. I don't know how AWS will meter it yet (yup, I love AWS) but I sure hope they're following this trend. AWS support would put so much energy behind this path forward.
- mcguire 11y ago
  Isn't AWS built on XEN? I think the MirageOS folks have said you could deploy one of their unikernel apps on AWS as-is.
  - Sanddancer 11y ago
    Yes it is, and AWS already supports unikernels.
    - pyvpx 11y ago
      unfortunately other providers, like DigitalOcean, don't seem to support rumpkernels or unikernels.
      - mcguire 11y ago
        I believe the problem isn't so much that DO, etc., don't support unikernels, as that they use something other than Xen as the virtualization layer; something-other-than-Xen is still on the to-do list of the unikernel projects.
        anttiok 11y ago
        The rump kernel unikernel (rumprun) will happily work on actual hardware. It's a bit hazy if that's a unikernel or an embedded system in that case (and what's the difference anyway?), but to avoid hairsplitting, you can replace the hardware with KVM and off you go.
    - 11y ago
Hoff 11y ago
What's old is new... Unikernels are quite reminiscent of the VAXELN product from ~30 years ago; write the application code, link in the various services you need, and disk or network or ROM boot the results: http://odl.sysworks.biz/disk$vaxdocdec953/decw$book/d33vaa48.p13.decw$book#23 http://odl.sysworks.biz/disk$vaxdocdec953/decw$book/d33vaa48...
- ChuckMcM 11y ago
  Of course in many ways "cloud" computing is just another implementation of a 370/VM mainframe system. I find the evolution fascinating. Oh and I had a comms processor (terminal server to a modem bank) which turned out to just be a MicroVAX 3 runing VAXELN. The CPU was some what "cut down" (I think optimized was the word they used) to make it more cost effective. VMS would not boot on it.
  - pnathan 11y ago
    Is there a decent technical writeup of how the 370 worked in practice? I love hearing about the old mainframe tech of yore.
    - walterbell 11y ago
      See The Origin of the VM/370 Time-sharing System by R.J. Creasy, http://lass.cs.umass.edu/~shenoy/courses/fall07/papers/vm370.pdf http://lass.cs.umass.edu/~shenoy/courses/fall07/papers/vm370...
- cbd1984 11y ago
  Sounds like LibOS: https://lwn.net/Articles/637658/ https://lwn.net/Articles/637658/ https://news.ycombinator.com/item?id=9259292 https://news.ycombinator.com/item?id=9259292 Or Service Virtual Machines: http://alt.folklore.computers.narkive.com/LXzjsdU7/was-vm-ever-used-as-an-exokernel http://alt.folklore.computers.narkive.com/LXzjsdU7/was-vm-ev... Or Exokernels: http://pdos.csail.mit.edu/exo.html http://pdos.csail.mit.edu/exo.html
  - 11y ago
nl 11y ago
For those who are interested in this, you may also be interested in ZeroVM[1]. To be clear, ZeroVM isn't a Unikernel. Instead, it is a virtualization technology based on NaCL[2] that is designed to quickly create throw-away VMs. The idea is that the VMs is created to service a single request, and disposed of afterwards. Obviously this model has challenges, but it is similar to the model proposed on slide 24 of the OP slidedeck. The security benefits of this are real, even if the programming model is challenging. [1] http://www.zerovm.org/ http://www.zerovm.org/ [2] http://en.wikipedia.org/wiki/Google_Native_Client http://en.wikipedia.org/wiki/Google_Native_Client
bsaul 11y ago
Are there any benchmark comparing speed of execution of a xen+unikernel app vs no virtualization at all ? I wonder if this tech would let you have bare metal speed in a cloud environment.
deleted 11y ago
[deleted]

132 points by sciurus 11y ago

Animats 11y ago

This is encouraging. A path to the post-C, post-Linux world is starting to become clear.

lbarrow 11y ago
Exactly. In my mind that's the best thing about these projects: the technical benefits are so amazing they might actually encourage people to look beyond the Linux-and-company systems stack. (Although, to be fair, most people run a Linux dom0.)
- api 11y ago
  What's weird and funny is the collapse in complexity this represents. Linux, NT, BSD, and other modern kernels implement all kinds of complexity around user/group management, ACLs, security policies, file types and permissions, and so on. If we're moving to unikernels and containers and similar, all that has been replaced with stupid simple access control systems like API access tokens or login/password pairs for cloud service control panels. Your AWS access token is now a single security credential for your entire infrastructure. Of course I'm sure the cycle of reincarnation will continue. Things like Docker or the AWS API will grow in complexity until they at least subsume the entire feature set of Unix UID/groups, Unix/NT ACLs, SELinux, etc.
  - deleted 11y ago
    [deleted]
pseudoProgress 11y ago
I would argue that in a world where we've progressed past Linux and C, but still talk about using PHP, MySQL and Java, and other such middleware, including Ruby, Python, Perl and so forth, not much actual progress has been made after all. It would simply feel like standing still, for an overhwelming majority.
otterley 11y ago
To what end?

gima 11y ago

I love the idea, always have. Does a unikernel application need statically allocated amount of RAM or is there a way to have unikernel-programs allocate memory dynamically? Maybe this is where garbage-collected language would be useful.

delinka 11y ago
Seems to me a unikernel app would have access to all the RAM in the system. It is, after all, the kernel itself.
- lbarrow 11y ago
  Yea but the hypervisor has to know how much physical memory to give the unikernel.
  - wtallis 11y ago
    The way it usually works is that the guest OS gets a certain allocation of RAM and if it doesn't want to use it all it can inform the hypervisor of which ranges don't need physical backing at the moment. From the guest's perspective this is often implemented using a "balloon" process/driver that reserves a large chunk of guest memory in order to return it to the hypervisor. It can also be handled by the guest having a more general memory hotplug capability.
rdtsc 11y ago
Look at a live version of Erlang on Xen. http://erlangonxen.org/ http://erlangonxen.org/ Press ESC on the front page and see the amount of GC ram reclaimed from Erlang's VM. Memory Usage: 12MB -- is also kind of a baffling number.

themartorana 11y ago

I was happy to see Clive, and I imagine a RUMP implementation or other unikernel that supports Rust won't be far away. Being able to compile Go or Rust to metal byte code is absurdly sexy. I don't know how AWS will meter it yet (yup, I love AWS) but I sure hope they're following this trend. AWS support would put so much energy behind this path forward.

mcguire 11y ago
Isn't AWS built on XEN? I think the MirageOS folks have said you could deploy one of their unikernel apps on AWS as-is.
- Sanddancer 11y ago
  Yes it is, and AWS already supports unikernels.
  - pyvpx 11y ago
    unfortunately other providers, like DigitalOcean, don't seem to support rumpkernels or unikernels.
    - mcguire 11y ago
      I believe the problem isn't so much that DO, etc., don't support unikernels, as that they use something other than Xen as the virtualization layer; something-other-than-Xen is still on the to-do list of the unikernel projects.
      - anttiok 11y ago
        The rump kernel unikernel (rumprun) will happily work on actual hardware. It's a bit hazy if that's a unikernel or an embedded system in that case (and what's the difference anyway?), but to avoid hairsplitting, you can replace the hardware with KVM and off you go.
  - 11y ago

Hoff 11y ago

What's old is new... Unikernels are quite reminiscent of the VAXELN product from ~30 years ago; write the application code, link in the various services you need, and disk or network or ROM boot the results: http://odl.sysworks.biz/disk$vaxdocdec953/decw$book/d33vaa48.p13.decw$book#23 http://odl.sysworks.biz/disk$vaxdocdec953/decw$book/d33vaa48...

ChuckMcM 11y ago
Of course in many ways "cloud" computing is just another implementation of a 370/VM mainframe system. I find the evolution fascinating. Oh and I had a comms processor (terminal server to a modem bank) which turned out to just be a MicroVAX 3 runing VAXELN. The CPU was some what "cut down" (I think optimized was the word they used) to make it more cost effective. VMS would not boot on it.
- pnathan 11y ago
  Is there a decent technical writeup of how the 370 worked in practice? I love hearing about the old mainframe tech of yore.
  - walterbell 11y ago
    See The Origin of the VM/370 Time-sharing System by R.J. Creasy, http://lass.cs.umass.edu/~shenoy/courses/fall07/papers/vm370.pdf http://lass.cs.umass.edu/~shenoy/courses/fall07/papers/vm370...
cbd1984 11y ago
Sounds like LibOS: https://lwn.net/Articles/637658/ https://lwn.net/Articles/637658/ https://news.ycombinator.com/item?id=9259292 https://news.ycombinator.com/item?id=9259292 Or Service Virtual Machines: http://alt.folklore.computers.narkive.com/LXzjsdU7/was-vm-ever-used-as-an-exokernel http://alt.folklore.computers.narkive.com/LXzjsdU7/was-vm-ev... Or Exokernels: http://pdos.csail.mit.edu/exo.html http://pdos.csail.mit.edu/exo.html
- 11y ago

nl 11y ago

For those who are interested in this, you may also be interested in ZeroVM[1]. To be clear, ZeroVM isn't a Unikernel. Instead, it is a virtualization technology based on NaCL[2] that is designed to quickly create throw-away VMs. The idea is that the VMs is created to service a single request, and disposed of afterwards. Obviously this model has challenges, but it is similar to the model proposed on slide 24 of the OP slidedeck. The security benefits of this are real, even if the programming model is challenging. [1] http://www.zerovm.org/ http://www.zerovm.org/ [2] http://en.wikipedia.org/wiki/Google_Native_Client http://en.wikipedia.org/wiki/Google_Native_Client

bsaul 11y ago

Are there any benchmark comparing speed of execution of a xen+unikernel app vs no virtualization at all ? I wonder if this tech would let you have bare metal speed in a cloud environment.

deleted 11y ago

[deleted]