4 ms·

Rumprun Tutorial: Serve a static website as a Unikernel

50 points by amirmc 11y ago

fennecfoxen 11y ago
What sort of benefits does this sort of an arrangement provide? Is it aimed at stripped-down embedded environments or something?
- yazaddaruvala 11y ago
  My understanding about using unikernels is that it helps you optimize. Unikernels allow you to trade built in safety and portability for customization and performance. If there is only one process, the webserver, why deal with a scheduler? If you're not using the safety features of an OS why have them? Why context switch if its not needed? You're hosting a key value store on an SSD. Why deal with a file system abstraction? Basically, as I understand it, unikernels are trying to democratize application deployment "on the metal" (or currently on xen) without the legacy fundamentals that drove the designs of OSes like Windows or Linux which were built for shared mainframes in university basements. For more info here is Xen talking about unikernels, http://wiki.xenproject.org/wiki/Unikernels http://wiki.xenproject.org/wiki/Unikernels
  - userbinator 11y ago
    In the very beginning, there was no OS and applications ran directly from boot. Then people started realising that applications had lots of duplicate code to do the same tasks, and abstracted that into a library linked into every application, which eventually grew into a full OS. It looks like with all the popularity of VMs/hypervisors, which are sort of like OSs themselves, multiplexing the virtual hardware to real, and the growing complexity of OSs, we're now going back to applications without an OS. It reminds me of these: https://en.wikipedia.org/wiki/PC_booter https://en.wikipedia.org/wiki/PC_booter
    - pjmlp 11y ago
      One thing that helped to steer into that direction is that developers started to realize that programming languages with rich runtimes, can have those runtimes play the OS role anyway. For example, I only care about POSIX when not using "comes with batteries/repository manager" languages.
  - shoo 11y ago
    I think another aspect I recall reading somewhere is security: if you can do the job with a far simpler system, potentially you end up with less attack surface (assuming the fancy new simpler system is actually well built and tested, etc). E.g. > Why use Mirage OS? > > The cloud has so far mostly been used to > consolidate existing operating systems and manage > them more conveniently. The interface between > guest kernels, applications and VMs are all > managed very differently, leading to serious > inefficiencies, unreliability and insecurity. http://www.xenproject.org/developers/teams/mirage-os.html http://www.xenproject.org/developers/teams/mirage-os.html https://mirage.io https://mirage.io
    - GauntletWizard 11y ago
      Further, you have less to lose if that surface is breached; If you penetrate a unikernel static content server, you can't really do much on that system; It won't have a shell, compilers, or any of the tooling needed to expand that hole. Because your OS is so much simpler, the cleanup is simpler too; Nuking the machine and starting over is much simpler when you've designed for lightweight unikernels.
  - amirmc 11y ago
    Each different Unikernel approach has its benefits and trade-offs. Rumprun lets you 'unikernelize' legacy software using well-established OS components (eg out of NetBSD) -- but you're still exposed to classes of bugs and potential run-time errors. MirageOS gives you safety, compile-time checks and produces really small binaries -- but everything is written in OCaml. So it's not quite about trading safety and portability, as you can retain those depending on the approach you take (eg, MirageOS currently lets you target Unix, Xen-on-x86 and Xen-on-ARM from exactly the same code-base). You can also try out static sites in MirageOS [1] and I gave a talk at polyconf last weekend with slides at [2]. [1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/ http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li... [2] http://amirchaudhry.com/unikernels-polyconf-2015/ http://amirchaudhry.com/unikernels-polyconf-2015/
- rumcajz 11y ago
  I would also add that unikernels make sense from the architectural point of view. Instead of running the monolithic behemoth of modern operating system, you can run something very basic, like a hypervisor and have everything else in optionally-linked libraries.
- superuser2 11y ago
  Linux/Windows/the BSDs contain quite a bit of code to support: - Lots of different hardware and peripherals, some legacy (i.e. floppies) - The filesystem, on top of whatever could conceivably be beneath it - An institutional administrator and multiple users who need protection from each other, all going about their business in shells simultaneously - Intelligent resource-sharing between lots of different processes Unikernels rip all that out and say: - There is only one kind of "hardware" in the universe: the supported hypervisor. No need for drivers for anything else. - There is exactly one user with OS-level access and he can be trusted completely. (This is many servers already: the only OS-level user is the sysadmin, all non-root accounts ever do us sudo to a root shell.) - This VM runs exactly one process (the actual application) and it can have direct access to hardware and memory. So no (or drastically reduced) need for memory protection, process scheduling, system calls, etc. It's likely that i.e. a relational database engine with direct access to SSD firmware can be made faster than one which must go through a generic filesystem abstraction layer. Disadvantages: - Largely uncharted waters - Linux is likely to be better at intelligently scheduling resources than a hypervisor, at least for the forseeable future.
  - amirmc 11y ago
    I'm not sure your final disadvantage really applies. If we accept that most deployments to the cloud (eg. EC2) are essentially single-purpose VMs, then Xen is already scheduling resources this way (& presumably well enough).
  - justincormack 11y ago
    If your hypervisor is KVM then Linux is scheduling your resources, so that last argument does not apply.
  - anttiok 11y ago
    > There is only one kind of "hardware" in the universe: the supported hypervisor. No need for drivers for anything else. An explicit goal of rumprun is to support real hardware. Defining the cloud as the only supported platform makes sense if you have to write or port hardware device drivers. For rumprun those drivers come for free as rump kernel components, so if you want to link them in (optional) and run on real hardware, it's possible. Having that possibility makes sense too: your cloud platform and IoT (+) platform can be the same thing, it's just the orchestration layer that differs. +) given a large enough "T"

50 points by amirmc 11y ago

What sort of benefits does this sort of an arrangement provide? Is it aimed at stripped-down embedded environments or something?

yazaddaruvala 11y ago
My understanding about using unikernels is that it helps you optimize. Unikernels allow you to trade built in safety and portability for customization and performance. If there is only one process, the webserver, why deal with a scheduler? If you're not using the safety features of an OS why have them? Why context switch if its not needed? You're hosting a key value store on an SSD. Why deal with a file system abstraction? Basically, as I understand it, unikernels are trying to democratize application deployment "on the metal" (or currently on xen) without the legacy fundamentals that drove the designs of OSes like Windows or Linux which were built for shared mainframes in university basements. For more info here is Xen talking about unikernels, http://wiki.xenproject.org/wiki/Unikernels http://wiki.xenproject.org/wiki/Unikernels
- userbinator 11y ago
  In the very beginning, there was no OS and applications ran directly from boot. Then people started realising that applications had lots of duplicate code to do the same tasks, and abstracted that into a library linked into every application, which eventually grew into a full OS. It looks like with all the popularity of VMs/hypervisors, which are sort of like OSs themselves, multiplexing the virtual hardware to real, and the growing complexity of OSs, we're now going back to applications without an OS. It reminds me of these: https://en.wikipedia.org/wiki/PC_booter https://en.wikipedia.org/wiki/PC_booter
  - pjmlp 11y ago
    One thing that helped to steer into that direction is that developers started to realize that programming languages with rich runtimes, can have those runtimes play the OS role anyway. For example, I only care about POSIX when not using "comes with batteries/repository manager" languages.
- shoo 11y ago
  I think another aspect I recall reading somewhere is security: if you can do the job with a far simpler system, potentially you end up with less attack surface (assuming the fancy new simpler system is actually well built and tested, etc). E.g. > Why use Mirage OS? > > The cloud has so far mostly been used to > consolidate existing operating systems and manage > them more conveniently. The interface between > guest kernels, applications and VMs are all > managed very differently, leading to serious > inefficiencies, unreliability and insecurity. http://www.xenproject.org/developers/teams/mirage-os.html http://www.xenproject.org/developers/teams/mirage-os.html https://mirage.io https://mirage.io
  - GauntletWizard 11y ago
    Further, you have less to lose if that surface is breached; If you penetrate a unikernel static content server, you can't really do much on that system; It won't have a shell, compilers, or any of the tooling needed to expand that hole. Because your OS is so much simpler, the cleanup is simpler too; Nuking the machine and starting over is much simpler when you've designed for lightweight unikernels.
- amirmc 11y ago
  Each different Unikernel approach has its benefits and trade-offs. Rumprun lets you 'unikernelize' legacy software using well-established OS components (eg out of NetBSD) -- but you're still exposed to classes of bugs and potential run-time errors. MirageOS gives you safety, compile-time checks and produces really small binaries -- but everything is written in OCaml. So it's not quite about trading safety and portability, as you can retain those depending on the approach you take (eg, MirageOS currently lets you target Unix, Xen-on-x86 and Xen-on-ARM from exactly the same code-base). You can also try out static sites in MirageOS [1] and I gave a talk at polyconf last weekend with slides at [2]. [1] http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/ http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-li... [2] http://amirchaudhry.com/unikernels-polyconf-2015/ http://amirchaudhry.com/unikernels-polyconf-2015/
rumcajz 11y ago
I would also add that unikernels make sense from the architectural point of view. Instead of running the monolithic behemoth of modern operating system, you can run something very basic, like a hypervisor and have everything else in optionally-linked libraries.
superuser2 11y ago
Linux/Windows/the BSDs contain quite a bit of code to support: - Lots of different hardware and peripherals, some legacy (i.e. floppies) - The filesystem, on top of whatever could conceivably be beneath it - An institutional administrator and multiple users who need protection from each other, all going about their business in shells simultaneously - Intelligent resource-sharing between lots of different processes Unikernels rip all that out and say: - There is only one kind of "hardware" in the universe: the supported hypervisor. No need for drivers for anything else. - There is exactly one user with OS-level access and he can be trusted completely. (This is many servers already: the only OS-level user is the sysadmin, all non-root accounts ever do us sudo to a root shell.) - This VM runs exactly one process (the actual application) and it can have direct access to hardware and memory. So no (or drastically reduced) need for memory protection, process scheduling, system calls, etc. It's likely that i.e. a relational database engine with direct access to SSD firmware can be made faster than one which must go through a generic filesystem abstraction layer. Disadvantages: - Largely uncharted waters - Linux is likely to be better at intelligently scheduling resources than a hypervisor, at least for the forseeable future.
- amirmc 11y ago
  I'm not sure your final disadvantage really applies. If we accept that most deployments to the cloud (eg. EC2) are essentially single-purpose VMs, then Xen is already scheduling resources this way (& presumably well enough).
- justincormack 11y ago
  If your hypervisor is KVM then Linux is scheduling your resources, so that last argument does not apply.
- anttiok 11y ago
  > There is only one kind of "hardware" in the universe: the supported hypervisor. No need for drivers for anything else. An explicit goal of rumprun is to support real hardware. Defining the cloud as the only supported platform makes sense if you have to write or port hardware device drivers. For rumprun those drivers come for free as rump kernel components, so if you want to link them in (optional) and run on real hardware, it's possible. Having that possibility makes sense too: your cloud platform and IoT (+) platform can be the same thing, it's just the orchestration layer that differs. +) given a large enough "T"