10 ms·

UKL: A Unikernel Based on Linux

156 points by perbu 8y ago

seliopou 8y ago
Link to GitHub repo: https://github.com/razaaliraza/ukl The article itself should have linked to it. It's a bad look to announce a project and only link to a bunch of other "competing" projects. Makes it looks like vaporware, but apparently there is code out there.
- ngcc_hk 8y ago
  Cannot see any repo. Just his email.
- codetrotter 8y ago
  Clickable link https://github.com/razaaliraza/ukl https://github.com/razaaliraza/ukl
- joppy 8y ago
  Why post a link which is not clickable?
tinktank 8y ago
How is this different from UML?
- glibgil 8y ago
  No one knows what that is
  - dtzWill 8y ago
    https://en.wikipedia.org/wiki/User-mode_Linux https://en.wikipedia.org/wiki/User-mode_Linux
    - stcredzero 8y ago
      UML used to refer to an entity/relationship+methods modeling language. https://en.wikipedia.org/wiki/Unified_Modeling_Language https://en.wikipedia.org/wiki/Unified_Modeling_Language
      - andrewstuart 8y ago
        Thank goodness we hear almost nothing about UML any more.
        quickthrower2 8y ago
        Why the hate for UML diagrams. They are quite useful.
        pjmlp 8y ago
        Just last week someone on our project was drawing a sequence diagram for an interaction between lambda calls on a cloud provider. UML is pretty much in use on the enterprise space.
- theamk 8y ago
  It's the other direction. UML runs Linux kernel in ring 3. It emulates hardware protection mechanisms in order to make debugging easier and to run VMs when there is no kernel access. UKL runs userspace apps in ring 0. It discards hardware protection mechanisms in order to make processes run faster, decrease boot time, and decrease management overhead.
anonymousDan 8y ago
How does this differ from LKL (the Linux kernel library : https://github.com/lkl/linux https://github.com/lkl/linux )?
- pickdenis 8y ago
  A simple scan of the repo's description and the first sentence of the linked article should answer that question. LKL is a library that allows you to do Linux things (read/write linux filesystems) from outside of Linux, essentially re-using the kernel's code. UKL, is an unikernel which essentially allows you to compile an application into a standalone OS that runs it (not sure if that's entirely correct). Now that I think about it a little more, they're kind of similar but I think unikernels are more of a way to get an application to run on hardware without all the extra bloat of a monolithic kernel (like if you're running a web service on a small virtualized machine) and the LKL is just for re-using Linux code.
hossbeast 8y ago
This is really exciting. Great write-up, and I'll be following with interest.
zippie 8y ago
This is great progress towards making unikernels non-runtime specific. However, I am still skeptical of the idea that unikernels will ever be production friendly. The biggest deficiency, by the definition of a unikernel, is the complete lack of debugging tools available. There is no top, perf, etc in a unikernel. In docker, I can still exec into a running container to debug and investigate. Unikernel issues lend themselves to “just restart it” rather than “let’s debug and fix it” mindsets. I’ve run a k8s environment with kata containers for a while and as time has gone on, I’ve found that Solaris Zones (smartOS) are probably the most scalable way to achieve process/VM isolation for applications.
- eyberg 8y ago
  htop and such don't make sense inside a single process system; logging usually gets sent to remote syslog/elk/etc. many apm solutions work out of the box one of the biggest misconceptions about unikernels is that they are somehow of a slimmed down linux - it's easier to think about them as individual programs provisioned as vms - would you ssh into a process? why? all 'debugging' tools work perfectly fine w/unikernels although I'd draw a fine line between real application level debugging which should never happen on production and ops tooling
  - gigatexal 8y ago
    Yes I agree. We have to refactor the way we debug and realize it’s devugging a single process and that process is the application.
  - rjzzleep 8y ago
    > I'd draw a fine line between real application level debugging which should never happen on production and ops tooling Sorry, but that's just plain wrong. One very common reason why people do that is, because performance debugging may give very different results on dev, staging and production systems that's why people want tooling that gives them as accurate as possible application level debugging information without impeding the behavior of the system. It's a very common problem in large scale systems.
elihu 8y ago
That seems like it could be an interesting way to launch Rust applications. Typically (and I'm assuming UKL is no different), unikernels don't have any memory protection between applications and the OS. That's usually a bad thing; however, if your compiler can verify that memory accesses are all in-bounds, then all the hardware memory checks that an MMU would do in a regular OS are just unnecessary overhead. So, with UKL and a Rust application, you could get approximately the same level of safety as before, but without the performance cost of all that runtime memory protection. Entering the kernel can be just a regular function call, context switches are cheap, etc... That could be a huge performance boon for some apps. In this sort of setup, unsafe Rust is like kernel code -- it effectively has privilege to read/write to any memory. I'd also feel better if the kernel itself were written in something like Rust.
- eyberg 8y ago
  all unikernel implementations I'm familiar with are single process by nature and thus the protection that you need on a multiple process system such as linux is completely un-necessary, especially since they are always deployed as vms
  - elihu 8y ago
    Well, strictly speaking it's safer to have a system where the application can't write into kernel memory space and the kernel can't (accidentally) write into application space. You don't have the risk of leaking secrets or malicious interference between applications that are supposed to be isolated if you've only got one app, but I could still see where someone might rather have memory protection than not in a single-user/single-application environment.
- pjmlp 8y ago
  Most unikernels are written in type safe languages, as such the type system plays the role of a MMU.
erichocean 8y ago
Seeing as how you can already run normal Linux like a unikernel on N-1 cores, and get all of the debugging and management goodness on core 0, I don't see the point. At all.
- eyberg 8y ago
  the big idea is that most companies are heavily virtualized - if you are using aws/gce you most definitely are and many private datacenters are as well - I'm not going to speak for ukl in particular but in general unikernels are vastly more secure than a normal process and much faster (if you are already virtualized which you are on aws/gce) - there is absolutely no reason to have 2 layers of linux if you are in 'the cloud'
  - geggam 8y ago
    Except the fact most production level code has support from the OS running scripts or some other hack to keep the code running. I have yet to see a production stack without this.

156 points by perbu 8y ago

Link to GitHub repo: https://github.com/razaaliraza/ukl The article itself should have linked to it. It's a bad look to announce a project and only link to a bunch of other "competing" projects. Makes it looks like vaporware, but apparently there is code out there.

ngcc_hk 8y ago
Cannot see any repo. Just his email.
codetrotter 8y ago
Clickable link https://github.com/razaaliraza/ukl https://github.com/razaaliraza/ukl
joppy 8y ago
Why post a link which is not clickable?

tinktank 8y ago

How is this different from UML?

glibgil 8y ago
No one knows what that is
- dtzWill 8y ago
  https://en.wikipedia.org/wiki/User-mode_Linux https://en.wikipedia.org/wiki/User-mode_Linux
  - stcredzero 8y ago
    UML used to refer to an entity/relationship+methods modeling language. https://en.wikipedia.org/wiki/Unified_Modeling_Language https://en.wikipedia.org/wiki/Unified_Modeling_Language
    - andrewstuart 8y ago
      Thank goodness we hear almost nothing about UML any more.
      - quickthrower2 8y ago
        Why the hate for UML diagrams. They are quite useful.
      - pjmlp 8y ago
        Just last week someone on our project was drawing a sequence diagram for an interaction between lambda calls on a cloud provider. UML is pretty much in use on the enterprise space.
theamk 8y ago
It's the other direction. UML runs Linux kernel in ring 3. It emulates hardware protection mechanisms in order to make debugging easier and to run VMs when there is no kernel access. UKL runs userspace apps in ring 0. It discards hardware protection mechanisms in order to make processes run faster, decrease boot time, and decrease management overhead.

anonymousDan 8y ago

How does this differ from LKL (the Linux kernel library : https://github.com/lkl/linux https://github.com/lkl/linux )?

pickdenis 8y ago
A simple scan of the repo's description and the first sentence of the linked article should answer that question. LKL is a library that allows you to do Linux things (read/write linux filesystems) from outside of Linux, essentially re-using the kernel's code. UKL, is an unikernel which essentially allows you to compile an application into a standalone OS that runs it (not sure if that's entirely correct). Now that I think about it a little more, they're kind of similar but I think unikernels are more of a way to get an application to run on hardware without all the extra bloat of a monolithic kernel (like if you're running a web service on a small virtualized machine) and the LKL is just for re-using Linux code.

hossbeast 8y ago

This is really exciting. Great write-up, and I'll be following with interest.

zippie 8y ago

This is great progress towards making unikernels non-runtime specific. However, I am still skeptical of the idea that unikernels will ever be production friendly. The biggest deficiency, by the definition of a unikernel, is the complete lack of debugging tools available. There is no top, perf, etc in a unikernel. In docker, I can still exec into a running container to debug and investigate. Unikernel issues lend themselves to “just restart it” rather than “let’s debug and fix it” mindsets. I’ve run a k8s environment with kata containers for a while and as time has gone on, I’ve found that Solaris Zones (smartOS) are probably the most scalable way to achieve process/VM isolation for applications.

eyberg 8y ago
htop and such don't make sense inside a single process system; logging usually gets sent to remote syslog/elk/etc. many apm solutions work out of the box one of the biggest misconceptions about unikernels is that they are somehow of a slimmed down linux - it's easier to think about them as individual programs provisioned as vms - would you ssh into a process? why? all 'debugging' tools work perfectly fine w/unikernels although I'd draw a fine line between real application level debugging which should never happen on production and ops tooling
- gigatexal 8y ago
  Yes I agree. We have to refactor the way we debug and realize it’s devugging a single process and that process is the application.
- rjzzleep 8y ago
  > I'd draw a fine line between real application level debugging which should never happen on production and ops tooling Sorry, but that's just plain wrong. One very common reason why people do that is, because performance debugging may give very different results on dev, staging and production systems that's why people want tooling that gives them as accurate as possible application level debugging information without impeding the behavior of the system. It's a very common problem in large scale systems.

elihu 8y ago

That seems like it could be an interesting way to launch Rust applications. Typically (and I'm assuming UKL is no different), unikernels don't have any memory protection between applications and the OS. That's usually a bad thing; however, if your compiler can verify that memory accesses are all in-bounds, then all the hardware memory checks that an MMU would do in a regular OS are just unnecessary overhead. So, with UKL and a Rust application, you could get approximately the same level of safety as before, but without the performance cost of all that runtime memory protection. Entering the kernel can be just a regular function call, context switches are cheap, etc... That could be a huge performance boon for some apps. In this sort of setup, unsafe Rust is like kernel code -- it effectively has privilege to read/write to any memory. I'd also feel better if the kernel itself were written in something like Rust.

eyberg 8y ago
all unikernel implementations I'm familiar with are single process by nature and thus the protection that you need on a multiple process system such as linux is completely un-necessary, especially since they are always deployed as vms
- elihu 8y ago
  Well, strictly speaking it's safer to have a system where the application can't write into kernel memory space and the kernel can't (accidentally) write into application space. You don't have the risk of leaking secrets or malicious interference between applications that are supposed to be isolated if you've only got one app, but I could still see where someone might rather have memory protection than not in a single-user/single-application environment.
pjmlp 8y ago
Most unikernels are written in type safe languages, as such the type system plays the role of a MMU.

erichocean 8y ago

Seeing as how you can already run normal Linux like a unikernel on N-1 cores, and get all of the debugging and management goodness on core 0, I don't see the point. At all.

eyberg 8y ago
the big idea is that most companies are heavily virtualized - if you are using aws/gce you most definitely are and many private datacenters are as well - I'm not going to speak for ukl in particular but in general unikernels are vastly more secure than a normal process and much faster (if you are already virtualized which you are on aws/gce) - there is absolutely no reason to have 2 layers of linux if you are in 'the cloud'
- geggam 8y ago
  Except the fact most production level code has support from the OS running scripts or some other hack to keep the code running. I have yet to see a production stack without this.