5 ms·

> So, besides performance: what actual, real-life problems does this solve? Well, don't go jumping to "besides performance". Dennard scaling is dead, and the t

> So, besides performance: what actual, real-life problems does this solve? Well, don't go jumping to "besides performance". Dennard scaling is dead, and the things that can interrupt your program in a full preemptive kernel system are myriad. Furthermore, even if your application properly manages and produces backpressure, the OS can introduce buffering where you don't want it, often by necessity (to avoid massive context switch costs). Now, if you just want to manage backpressure over the network, an application-hosted networking stack is probably a fine solution, but if you want to accurately translate disk backpressure to network backpressure it gets more complicated. There are heaps and heaps of ordinary programs written for preemptive systems that would see noticeable, tangible benefits to users if they were run instead on unikernels; but the main thing halting adoption is the inconvenience of adapting programs. I tried getting Capstan/OSv to work the other day and the documentation dragged me through several apparently-outdated methods of achieving the same thing, all of which failed in incomprehensible ways. Tooling for these things could use a lot of work.

MrBuddyCasino 6y ago
So, a niche technology for a handful of HFT firms, betting platforms and hyper-scale cloud vendors. Those already use non-mainstream tech (eg LMAX disruptor) to achieve max performance. This is all nice and good, but the Unikernel guys claim since 20 years its the next big revolutionary thing. In the real world, the evolutionary approach using containers has turned out to solve the problems most people care about. Don’t get me wrong, this is fascinating stuff. But there is a difference between „changes mainstream computing“ and „cool tech to shave off another millisecond of latency“.
- throwaway894345 6y ago
  Unikernels could also be the next phase of cloud orchestration technologies. They would be truly lightweight VMs, and obviate the need for containers (collapse the VM orchestration and container orchestration layers into a single orchestration layer). This would offer better security and better performance without impacting developer/operator friendliness. And a side note about improved performance, in the cloud space, being able to boot in ten ms means you can start a VM in the scope of a single request, which means you can scale up dynamically without keeping a bunch of VMs running idle in the background just in case your traffic spikes. This is real, significant cost savings for cloud providers and customers.
  - rbanffy 6y ago
    Debugging a single-executable container is quite unpleasant. I assume debugging unikernels or thin VMs (with a kernel and a single program) would be a similar experience.
    - zarkov99 6y ago
      Could you not do all your debugging and development on a normal host and only deploy on a unikernel when you are stable? There is already a divide between development and production with debug and release builds the later of which are a massive pain to debug.
      - rbanffy 6y ago
        > debugging and development on a normal host and only deploy on a unikernel when you are stable? Sure! And, still, when you deploy, it fails. At least one of your assumptions that led to your conclusion that your development/testing environment is equivalent to the production one are flawed. Find which ones.
        zarkov99 6y ago
        That is not a new problem. To some extent it already happens with debug and release builds, with running on a VM vs a real host, running on a different distribution than the one you developed in, etc. If the performance gains are significant in some domains that would be more than enough to pay for the extra pain.
        rbanffy 6y ago
        > running on a VM vs a real host, running on a different distribution than the one you developed in Except that, in those cases, you can connect to the server and spin up a shell and actually get an idea, from the file system, from logs or from any of the many ways you have to inspect a system. All that you'll need to provide to your unikernel.
eyberg 6y ago
If you get a chance let me know what you think of our take on https://ops.city https://ops.city && https://github.com/nanovms/nanos https://github.com/nanovms/nanos .
trasz 6y ago
It’s much easier to tweak a normal kernel (or just use whatever is already there, eg real-time scheduling to get rid of preemption) than move to a unikernel.
- eyberg 6y ago
  I kinda disagree. We looked at doing this and just couldn't fathom how it could be done. You might be surprised at the amount of code and entanglement there is to support multiple users, multiple processes - that touches everything from IPC/shared memory to address space to permissions. Very large deep cuts would have to be made and then you'd need to ensure your patchset stays congruent with upstream. The other thing that people will look at is doing something like alpine && a heavy-handed seccomp/apparmor but then you're not really doing any cuts at all.
  - trasz 6y ago
    How does the mere existence of that code hurt your workload?
GregarianChild 6y ago
I'm interested to learn why unikernels would be better at handling backpressure. Because there are fewer sources of interrupts that might interfere with each other?
- microcolonel 6y ago
  Numerous interrupts may have an impact on latency, but for backpressure, buffering in the network layer or the filesystem/block layer can hide the limitations of the system from your application, and when you hit those limits, the hit is harder and can cause oscillations throughout a network or distributed system. Now, of course, applications themselves can fail at this too. Look at Postgres, maybe, where typically people rely on [AUTO]VACUUM, but if your application runs continuously, then there is no right time to run it.

MrBuddyCasino 6y ago

So, a niche technology for a handful of HFT firms, betting platforms and hyper-scale cloud vendors. Those already use non-mainstream tech (eg LMAX disruptor) to achieve max performance. This is all nice and good, but the Unikernel guys claim since 20 years its the next big revolutionary thing. In the real world, the evolutionary approach using containers has turned out to solve the problems most people care about. Don’t get me wrong, this is fascinating stuff. But there is a difference between „changes mainstream computing“ and „cool tech to shave off another millisecond of latency“.

throwaway894345 6y ago
Unikernels could also be the next phase of cloud orchestration technologies. They would be truly lightweight VMs, and obviate the need for containers (collapse the VM orchestration and container orchestration layers into a single orchestration layer). This would offer better security and better performance without impacting developer/operator friendliness. And a side note about improved performance, in the cloud space, being able to boot in ten ms means you can start a VM in the scope of a single request, which means you can scale up dynamically without keeping a bunch of VMs running idle in the background just in case your traffic spikes. This is real, significant cost savings for cloud providers and customers.
- rbanffy 6y ago
  Debugging a single-executable container is quite unpleasant. I assume debugging unikernels or thin VMs (with a kernel and a single program) would be a similar experience.
  - zarkov99 6y ago
    Could you not do all your debugging and development on a normal host and only deploy on a unikernel when you are stable? There is already a divide between development and production with debug and release builds the later of which are a massive pain to debug.
    - rbanffy 6y ago
      > debugging and development on a normal host and only deploy on a unikernel when you are stable? Sure! And, still, when you deploy, it fails. At least one of your assumptions that led to your conclusion that your development/testing environment is equivalent to the production one are flawed. Find which ones.
      - zarkov99 6y ago
        That is not a new problem. To some extent it already happens with debug and release builds, with running on a VM vs a real host, running on a different distribution than the one you developed in, etc. If the performance gains are significant in some domains that would be more than enough to pay for the extra pain.
        rbanffy 6y ago
        > running on a VM vs a real host, running on a different distribution than the one you developed in Except that, in those cases, you can connect to the server and spin up a shell and actually get an idea, from the file system, from logs or from any of the many ways you have to inspect a system. All that you'll need to provide to your unikernel.

eyberg 6y ago

If you get a chance let me know what you think of our take on https://ops.city https://ops.city && https://github.com/nanovms/nanos https://github.com/nanovms/nanos .

trasz 6y ago

It’s much easier to tweak a normal kernel (or just use whatever is already there, eg real-time scheduling to get rid of preemption) than move to a unikernel.

eyberg 6y ago
I kinda disagree. We looked at doing this and just couldn't fathom how it could be done. You might be surprised at the amount of code and entanglement there is to support multiple users, multiple processes - that touches everything from IPC/shared memory to address space to permissions. Very large deep cuts would have to be made and then you'd need to ensure your patchset stays congruent with upstream. The other thing that people will look at is doing something like alpine && a heavy-handed seccomp/apparmor but then you're not really doing any cuts at all.
- trasz 6y ago
  How does the mere existence of that code hurt your workload?

GregarianChild 6y ago

I'm interested to learn why unikernels would be better at handling backpressure. Because there are fewer sources of interrupts that might interfere with each other?

microcolonel 6y ago
Numerous interrupts may have an impact on latency, but for backpressure, buffering in the network layer or the filesystem/block layer can hide the limitations of the system from your application, and when you hit those limits, the hit is harder and can cause oscillations throughout a network or distributed system. Now, of course, applications themselves can fail at this too. Look at Postgres, maybe, where typically people rely on [AUTO]VACUUM, but if your application runs continuously, then there is no right time to run it.