6 ms·

A unikernel experiment: A VM for every URL

94 points by amirmc 12y ago

amirmc 12y ago
The fun thing about this is that you can try it out locally and apparently it took (about) an afternoon to put together (most of the work is in rewriting URLs). This kind of fast experimentation is going to lead to some really cool things happening with unikernels.
reilly3000 12y ago
This is something I've been dreaming of. It is lovely to see it coming into the world. Death to the website! Long live the UX in a container!
- agentultra 12y ago
  This is better than a "container." A Mirage unikernel literally compiles in the pieces of an "OS" as libraries. There is no monolithic operating system with code sitting around waiting to be exploited. The web server in this example is the kernel.
digital-rubber 12y ago
Looks interesting, though this will multiply the number of DNS requests for a page with some images significantly.
sp332 12y ago
The next step is to compile it all to JS and just serve static files that boot in the browser! :) Something like http://bellard.org/jslinux/ http://bellard.org/jslinux/
- cookrn 12y ago
  See: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...
hunnypot 12y ago
This is cool, especially when you think of the opportunities it offers for DDoS prevention.
- Nilef 12y ago
  Can you tell us more about these opportunities?
cjg 12y ago
When do we upgrade from IPv6?
lbotos 12y ago
" I have not been able to find a hosting provider that has a minimum configuration which would be appropriate for large or small unikernels (e.g. 8MB vs 64MB RAM)." This is the biggest problem with unikernels right now.
- sp332 12y ago
  Looks like an arbitrage opportunity! Buy 64MB server, sell 8MB servers.
NhanH 12y ago
I've always thought that something really cool could be done with unikernel for SaaS: you let each of your customer have a VM on their own, spinning the unikernel up when the requests came in, and let it process any request come in, after a while, if the specific user stop requesting then you kill the VM. There seems to be a lot of benefits from that: single user authentication, horizontally scaling is simple ("vertical" scaling also get easier as you can just move VM away), app design will easily let it become an on premise installation. Code upgrade is also easier as it won't disturb everyone at once, and you can upgrade when the VM is not running for a specific user (no hotswap needed). There is definitely operational overhead in a DNS server, but besides that it seems like the benefits would outweight the cons. I know that's what DabbleDB was doing back then (in Squeak Smalltalk, each of their users has a smalltalk VM serving them). Is there anyone else doing anything similar nowadays?
- avsm 12y ago
  That sort of design is what motivated Magnus' post that has been linked here. We've been laying the uniSaaS groundwork with a new Xen toolstack daemon called Jitsu ("Just in Time Summoning of Unikernels"). Here's the camera ready paper that we'll be presenting at USENIX NSDI in May: http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf There's a lot more besides just the spawning to build a robust SaaS of course, but you can see the direction we're heading in...
  - jacques_chester 12y ago
    I think the obstacle is that most folk are not prepared to give up their languages for this, so containerising PaaSes are still likely to dominate the scene. Disclaimer: I worked on Cloud Foundry.
    - avsm 12y ago
      > I think the obstacle is that most folk are not prepared to give up their languages s/languages/frameworks, and I agree. One step at a time -- this work is a foundational block that we're building on, not the ultimate solution.
rdtsc 12y ago
Already exists for Erlang. Here is a demo: http://erlangonxen.org/ http://erlangonxen.org/ Site is running on such a VM as well, press escape for a console to see stats. Here is an example output from running their demo: --- 300 sec : is how long it takes to launch a Linux instance to availability on Amazon EC2. 50 sec : is the time between power on and the lock screen for an Android phone. 0.4 sec : ago is when we received your request. Within this time we managed to create a new Xen instance, boot it, and run the application that rendered the page you are viewing. By the time you are done reading this, the instance will be gone. ---
- avsm 12y ago
  Note that both of the setups are quite different in how they boot unikernels. The EoX Zerg demo boots one unikernel per connection by having an HTTP proxy in the frontend spin up the VM. Magnus' version builds a static set of unikernels whose total resource usage is known ahead of time, and boots them based on DNS lookups as necessary. Because there is a race between a DNS response, the first TCP SYN from the client, and the VM booting up to receive it, there is a helper unikernel called "Synjitsu" which terminates TCP connections and replays them to the MirageOS unikernel as soon as it boots up. This is very easy due to the OCaml TCP/IP stack in Synjitsu sharing its state via serialising it into an s-expression. More gory details in http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf Now, imagine a version of Synjitsu that did this handover with TLS connections rather than just TCP. The recipient unikernel wouldn't ever even have to know the private key, just the ephemeral session key...
  - rdtsc 12y ago
    That is just a demo to show startup times. Presumably (just by looking at it from a high level) a proxy instance for handling initial connection + a pre-allocated, or maybe booted but paused, set of worker images would work in either architecture.
thom_nic 12y ago
I seem to remember something similar that was an erlang VM running (I think) directly on Xen. The underlying idea in that case was each request causes a new VM to boot, service that request, then terminate.
- tobz 12y ago
  Mentioned in another top-level comment: http://erlangonxen.org/ http://erlangonxen.org/
mwcampbell 12y ago
I suppose work like this is demonstrating that, contrary to what e.g. the creators of Docker say, a virtual machine image is not too heavyweight to be our industry's equivalent of a shipping container. What is too heavyweight is a VM image of a conventional operating system distribution with all of the baggage that it brings. Would these fine-grained unikernels also run well on KVM, VMware ESXi, Hyper-V, or bhyve (FreeBSD's new VMM)? Or is Xen uniquely suited to this kind of thing?
- amirmc 12y ago
  > "What is too heavyweight is a VM image of a conventional operating system distribution with all of the baggage that it brings." Indeed. I didn't realise that Docker/Container folks had ever said VMs were too heavyweight. Is this linked anywhere? > "Would these fine-grained unikernels also run well on KVM, VMware ESXi, Hyper-V, or bhyve (FreeBSD's new VMM)?" We've been asked this before and for KVM at least, it just boils down to someone suitably motivated to write the boot and VirtIO code [1, 2]. If anyone's interested in this, we'd love to hear about it [3]! [1] http://lists.xenproject.org/archives/html/mirageos-devel/2014-08/msg00103.html http://lists.xenproject.org/archives/html/mirageos-devel/201... [2] http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00034.html http://lists.xenproject.org/archives/html/mirageos-devel/201... [3] http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageo...
  - mwcampbell 12y ago
    > Indeed. I didn't realise that Docker/Container folks had ever said VMs were too heavyweight. Is this linked anywhere? https://github.com/docker/docker#user-content-better-than-vms https://github.com/docker/docker#user-content-better-than-vm...
    - amirmc 12y ago
      Thanks. In hindsight that should have been an obvious place for me to look.
UhUhUhUh 12y ago
I'm not sure what the applications/market for this is or could be, but I love the idea. Everything that goes "micro" I love dearly.
- ccvannorman 12y ago
  I'm no DNS or security expert, but couldn't one application be increased security of connections? E.g. it's harder to do that 0day kernel hack with a buffer overflow if every URL request is sandboxed in a VM? I probably got it wrong but I'd love to hear a more expert opinion on why/why not this could be used for security..
biot 12y ago
Similar to ZeroVM [0], where a VM is spawned and destroyed for each individual request. [0] http://www.zerovm.org/ http://www.zerovm.org/
caligastia 12y ago
This is an interesting experiment in moving the computation to the endpoints of a namespace. I would suggest a variant where instead of a VM you place a Fossil file https://www.fossil-scm.org/index.html/doc/trunk/www/fileformat.wiki https://www.fossil-scm.org/index.html/doc/trunk/www/fileform... Using that you can store a whole filesystem at the URL instead of one file, and you also get synchronization mechanisms built in, so your endpoints can communicate with each other in a structured way that you control.. vs the undefined communication you will get between VM's if you try this with the VM architecture.

94 points by amirmc 12y ago

The fun thing about this is that you can try it out locally and apparently it took (about) an afternoon to put together (most of the work is in rewriting URLs). This kind of fast experimentation is going to lead to some really cool things happening with unikernels.

reilly3000 12y ago

This is something I've been dreaming of. It is lovely to see it coming into the world. Death to the website! Long live the UX in a container!

agentultra 12y ago
This is better than a "container." A Mirage unikernel literally compiles in the pieces of an "OS" as libraries. There is no monolithic operating system with code sitting around waiting to be exploited. The web server in this example is the kernel.

digital-rubber 12y ago

Looks interesting, though this will multiply the number of DNS requests for a page with some images significantly.

sp332 12y ago

The next step is to compile it all to JS and just serve static files that boot in the browser! :) Something like http://bellard.org/jslinux/ http://bellard.org/jslinux/

cookrn 12y ago
See: https://www.destroyallsoftware.com/talks/the-birth-and-death-of-javascript https://www.destroyallsoftware.com/talks/the-birth-and-death...

hunnypot 12y ago

This is cool, especially when you think of the opportunities it offers for DDoS prevention.

Nilef 12y ago
Can you tell us more about these opportunities?

cjg 12y ago

When do we upgrade from IPv6?

lbotos 12y ago

" I have not been able to find a hosting provider that has a minimum configuration which would be appropriate for large or small unikernels (e.g. 8MB vs 64MB RAM)." This is the biggest problem with unikernels right now.

sp332 12y ago
Looks like an arbitrage opportunity! Buy 64MB server, sell 8MB servers.

NhanH 12y ago

I've always thought that something really cool could be done with unikernel for SaaS: you let each of your customer have a VM on their own, spinning the unikernel up when the requests came in, and let it process any request come in, after a while, if the specific user stop requesting then you kill the VM. There seems to be a lot of benefits from that: single user authentication, horizontally scaling is simple ("vertical" scaling also get easier as you can just move VM away), app design will easily let it become an on premise installation. Code upgrade is also easier as it won't disturb everyone at once, and you can upgrade when the VM is not running for a specific user (no hotswap needed). There is definitely operational overhead in a DNS server, but besides that it seems like the benefits would outweight the cons. I know that's what DabbleDB was doing back then (in Squeak Smalltalk, each of their users has a smalltalk VM serving them). Is there anyone else doing anything similar nowadays?

avsm 12y ago
That sort of design is what motivated Magnus' post that has been linked here. We've been laying the uniSaaS groundwork with a new Xen toolstack daemon called Jitsu ("Just in Time Summoning of Unikernels"). Here's the camera ready paper that we'll be presenting at USENIX NSDI in May: http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf There's a lot more besides just the spawning to build a robust SaaS of course, but you can see the direction we're heading in...
- jacques_chester 12y ago
  I think the obstacle is that most folk are not prepared to give up their languages for this, so containerising PaaSes are still likely to dominate the scene. Disclaimer: I worked on Cloud Foundry.
  - avsm 12y ago
    > I think the obstacle is that most folk are not prepared to give up their languages s/languages/frameworks, and I agree. One step at a time -- this work is a foundational block that we're building on, not the ultimate solution.

rdtsc 12y ago

Already exists for Erlang. Here is a demo: http://erlangonxen.org/ http://erlangonxen.org/ Site is running on such a VM as well, press escape for a console to see stats. Here is an example output from running their demo: --- 300 sec : is how long it takes to launch a Linux instance to availability on Amazon EC2. 50 sec : is the time between power on and the lock screen for an Android phone. 0.4 sec : ago is when we received your request. Within this time we managed to create a new Xen instance, boot it, and run the application that rendered the page you are viewing. By the time you are done reading this, the instance will be gone. ---

avsm 12y ago
Note that both of the setups are quite different in how they boot unikernels. The EoX Zerg demo boots one unikernel per connection by having an HTTP proxy in the frontend spin up the VM. Magnus' version builds a static set of unikernels whose total resource usage is known ahead of time, and boots them based on DNS lookups as necessary. Because there is a race between a DNS response, the first TCP SYN from the client, and the VM booting up to receive it, there is a helper unikernel called "Synjitsu" which terminates TCP connections and replays them to the MirageOS unikernel as soon as it boots up. This is very easy due to the OCaml TCP/IP stack in Synjitsu sharing its state via serialising it into an s-expression. More gory details in http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf http://anil.recoil.org/papers/2015-nsdi-jitsu.pdf Now, imagine a version of Synjitsu that did this handover with TLS connections rather than just TCP. The recipient unikernel wouldn't ever even have to know the private key, just the ephemeral session key...
- rdtsc 12y ago
  That is just a demo to show startup times. Presumably (just by looking at it from a high level) a proxy instance for handling initial connection + a pre-allocated, or maybe booted but paused, set of worker images would work in either architecture.

thom_nic 12y ago

I seem to remember something similar that was an erlang VM running (I think) directly on Xen. The underlying idea in that case was each request causes a new VM to boot, service that request, then terminate.

tobz 12y ago
Mentioned in another top-level comment: http://erlangonxen.org/ http://erlangonxen.org/

mwcampbell 12y ago

I suppose work like this is demonstrating that, contrary to what e.g. the creators of Docker say, a virtual machine image is not too heavyweight to be our industry's equivalent of a shipping container. What is too heavyweight is a VM image of a conventional operating system distribution with all of the baggage that it brings. Would these fine-grained unikernels also run well on KVM, VMware ESXi, Hyper-V, or bhyve (FreeBSD's new VMM)? Or is Xen uniquely suited to this kind of thing?

amirmc 12y ago
> "What is too heavyweight is a VM image of a conventional operating system distribution with all of the baggage that it brings." Indeed. I didn't realise that Docker/Container folks had ever said VMs were too heavyweight. Is this linked anywhere? > "Would these fine-grained unikernels also run well on KVM, VMware ESXi, Hyper-V, or bhyve (FreeBSD's new VMM)?" We've been asked this before and for KVM at least, it just boils down to someone suitably motivated to write the boot and VirtIO code [1, 2]. If anyone's interested in this, we'd love to hear about it [3]! [1] http://lists.xenproject.org/archives/html/mirageos-devel/2014-08/msg00103.html http://lists.xenproject.org/archives/html/mirageos-devel/201... [2] http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00034.html http://lists.xenproject.org/archives/html/mirageos-devel/201... [3] http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageos-devel http://lists.xenproject.org/cgi-bin/mailman/listinfo/mirageo...
- mwcampbell 12y ago
  > Indeed. I didn't realise that Docker/Container folks had ever said VMs were too heavyweight. Is this linked anywhere? https://github.com/docker/docker#user-content-better-than-vms https://github.com/docker/docker#user-content-better-than-vm...
  - amirmc 12y ago
    Thanks. In hindsight that should have been an obvious place for me to look.

UhUhUhUh 12y ago

I'm not sure what the applications/market for this is or could be, but I love the idea. Everything that goes "micro" I love dearly.

ccvannorman 12y ago
I'm no DNS or security expert, but couldn't one application be increased security of connections? E.g. it's harder to do that 0day kernel hack with a buffer overflow if every URL request is sandboxed in a VM? I probably got it wrong but I'd love to hear a more expert opinion on why/why not this could be used for security..

biot 12y ago

Similar to ZeroVM [0], where a VM is spawned and destroyed for each individual request. [0] http://www.zerovm.org/ http://www.zerovm.org/

caligastia 12y ago

This is an interesting experiment in moving the computation to the endpoints of a namespace. I would suggest a variant where instead of a VM you place a Fossil file https://www.fossil-scm.org/index.html/doc/trunk/www/fileformat.wiki https://www.fossil-scm.org/index.html/doc/trunk/www/fileform... Using that you can store a whole filesystem at the URL instead of one file, and you also get synchronization mechanisms built in, so your endpoints can communicate with each other in a structured way that you control.. vs the undefined communication you will get between VM's if you try this with the VM architecture.