7 ms·

I wouldn't say that unikernels were entirely undebuggable. I spent a few hours hacking and came up with a proof of concept dom0 profiler, and learned some debug

I wouldn't say that unikernels were entirely undebuggable. I spent a few hours hacking and came up with a proof of concept dom0 profiler, and learned some debugging benefits: one symbol table for the entire binary, one place to turn on frame pointers for everything, etc. http://www.brendangregg.com/blog/2016-01-27/unikernel-profiling-from-dom0.html http://www.brendangregg.com/blog/2016-01-27/unikernel-profil...

rjsw 10y ago
There is nothing stopping people from creating a unikernel for a dynamic language that also includes the development tools. A Lisp Machine on Xen would be one model.
- moosingin3space 10y ago
  I feel like Erlang-based unikernels are an extremely compelling alternative to traditional UNIX deployments. Immutable systems with safe hot swap and excellent debugging tools like `observer` and `debugger`.
  - reycharles 10y ago
    http://erlangonxen.org/ http://erlangonxen.org/
- derefr 10y ago
  Erlang on Xen is already that way. You can use the full Erlang profiling/tracing/debugging/observing toolkit on an EoX node.
ryao 10y ago
That requires having access to dom0, which is not available on Amazon EC2 (for good reason). Running in UNIX binary mode in a container on bare metal seems like the way to go here. That does not have the overhead of hardware virtualization. That means no internal memory fragmentation from partitioning RAM, the ability to avoid duplication of cache and any hypervisor upcalls become normal system calls. At that point, the unikernel is just a normal UNIX process that can be debugged with conventional tools.
- brendangregg 10y ago
  We already get hypervisor statistics from dom0, via cloud watch. Could that include on-demand profiling as well? I don't see why not. And that's not the only way to solve this. So work needs to be done to make unikernels profileable & debuggable. I wouldn't claim that this was impossible.
  - ryao 10y ago
    I do not dispute that it is possible to profile/debug unikernels on future cloud infrastructure. However, I am skeptical that unikernels offer any benefit to merit the work of enabling that when the whole system is considered. System calls might be additional overhead when there is a hypervisor, but hypervisors are unnecessary when we have containers. You stand to eliminate much more overhead from eliminating the hypervisor than you stand from eliminating the syscalls. Some of that overhead is internal fragmentation from memory partitioning, duplication of driver code, potential double caching, etcetera. The industry is in the early stages of a transition from hardware virtualization to containers because containers are a better abstraction than hardware virtualization. Joyent offers Illumos zones, Swiscomm offers docker containers with flocker (full disclosure: my employer is the author of flocker), Microsoft has deployed drawbridge on their Azure cloud, etcetera. We will only see more of this in the future. Once the transition is complete, I see no advantage to unikernels. You could use them in UNIX binary mode, but that makes them little more than a standard process on a traditional system. That is a very different role than the one that their creators intended for them.
    - brendangregg 10y ago
      Price/performance always wins. Can an application unikernel on a hypervisor (which is really a lightweight OS that nowadays supports many pass-through features) beat performance of an application on a regular OS? (I didn't mention containers, since they should ultimately be irrelevant to hot path performance). So can it? With a lot of work, I bet they can. So who has has the better price/performance? That's going to depend on how much engineering work it is to adopt, fix, and use unikernels, when they are competing with an established ecosystem around Linux and containers. And that may be where unikernels actually loses on price/performance, where price includes total cost of ownership. We'll see!
jrv 10y ago
That's great, but with processes (containerized or not), I can use the full variety of standard diagnostic tools built over many decades. Of course you can start building similar tools for unikernels, but it's gonna take you a long time before you get to a similar state.
choosername 10y ago
To counter the OP, you'd have to use an existing profiler.

rjsw 10y ago

There is nothing stopping people from creating a unikernel for a dynamic language that also includes the development tools. A Lisp Machine on Xen would be one model.

moosingin3space 10y ago
I feel like Erlang-based unikernels are an extremely compelling alternative to traditional UNIX deployments. Immutable systems with safe hot swap and excellent debugging tools like `observer` and `debugger`.
- reycharles 10y ago
  http://erlangonxen.org/ http://erlangonxen.org/
derefr 10y ago
Erlang on Xen is already that way. You can use the full Erlang profiling/tracing/debugging/observing toolkit on an EoX node.

ryao 10y ago

That requires having access to dom0, which is not available on Amazon EC2 (for good reason). Running in UNIX binary mode in a container on bare metal seems like the way to go here. That does not have the overhead of hardware virtualization. That means no internal memory fragmentation from partitioning RAM, the ability to avoid duplication of cache and any hypervisor upcalls become normal system calls. At that point, the unikernel is just a normal UNIX process that can be debugged with conventional tools.

brendangregg 10y ago
We already get hypervisor statistics from dom0, via cloud watch. Could that include on-demand profiling as well? I don't see why not. And that's not the only way to solve this. So work needs to be done to make unikernels profileable & debuggable. I wouldn't claim that this was impossible.
- ryao 10y ago
  I do not dispute that it is possible to profile/debug unikernels on future cloud infrastructure. However, I am skeptical that unikernels offer any benefit to merit the work of enabling that when the whole system is considered. System calls might be additional overhead when there is a hypervisor, but hypervisors are unnecessary when we have containers. You stand to eliminate much more overhead from eliminating the hypervisor than you stand from eliminating the syscalls. Some of that overhead is internal fragmentation from memory partitioning, duplication of driver code, potential double caching, etcetera. The industry is in the early stages of a transition from hardware virtualization to containers because containers are a better abstraction than hardware virtualization. Joyent offers Illumos zones, Swiscomm offers docker containers with flocker (full disclosure: my employer is the author of flocker), Microsoft has deployed drawbridge on their Azure cloud, etcetera. We will only see more of this in the future. Once the transition is complete, I see no advantage to unikernels. You could use them in UNIX binary mode, but that makes them little more than a standard process on a traditional system. That is a very different role than the one that their creators intended for them.
  - brendangregg 10y ago
    Price/performance always wins. Can an application unikernel on a hypervisor (which is really a lightweight OS that nowadays supports many pass-through features) beat performance of an application on a regular OS? (I didn't mention containers, since they should ultimately be irrelevant to hot path performance). So can it? With a lot of work, I bet they can. So who has has the better price/performance? That's going to depend on how much engineering work it is to adopt, fix, and use unikernels, when they are competing with an established ecosystem around Linux and containers. And that may be where unikernels actually loses on price/performance, where price includes total cost of ownership. We'll see!

jrv 10y ago

That's great, but with processes (containerized or not), I can use the full variety of standard diagnostic tools built over many decades. Of course you can start building similar tools for unikernels, but it's gonna take you a long time before you get to a similar state.

choosername 10y ago

To counter the OP, you'd have to use an existing profiler.