5 ms·

A question that constantly reoccurs for me is this: who does the scheduling of threads and/or processes in a unikernel? I know unikernels are single process, b

A question that constantly reoccurs for me is this: who does the scheduling of threads and/or processes in a unikernel? I know unikernels are single process, but are they also intended to be single core? If not, without the kernel scheduler, what is providing that function to the unikernel? Is it just an optional module, say in the case of rumprun?

yazaddaruvala 9y ago
The whole point is there are many ways to do it and the application knows best. The API specifics are probably different but it's just setting some registers and the the program counter on that CPU core. The code likely looks very similar to a context switch.
derefr 9y ago
Consider that any complex language runtime (e.g. Erlang's BEAM, the JVM, etc.) has its own scheduler. If you convert such a runtime into a unikernel (such as in Erlang on Xen), the runtime scheduler is "the" scheduler. Besides languages, though, plenty of complex applications have their own schedulers. Databases, for example. Postgres-as-a-unikernel would make a lot of sense, because it's already doing its own equivalent to everything that OSes do. If your program is complex enough, it likely will already have an "optimized" form of internally-managed shared memory/message-passing multithreading (with core affinities et al) to allow it to allocate tasks to cores better than the OS can without the "insider knowledge" of what it's trying to accomplish. At that point, your program will end up with a scheduler "inside" whether you set out to build one or not. On the other hand, if your program isn't that complex, then it's very likely you'll get better performance out of your code by just making each process into a single-core unikernel and then packing instances onto however many vCPUs the hypervisor host offers.
- bluejekyll 9y ago
  I don't know much about the other Runtimes that you mention, but correct me where I'm wrong: In the case of the JVM, green threads were dropped a long time ago, and under the current paradigm, native threads are used (on all platforms?). So doesn't this need to be written into the JVM? Not impossible of course (or provided by the unikernel wrapper as it is in OSv). Similarly with Databases, Oracle and Postgres both use a process per connection client/server model. So a unikernel for these would be impractical without rearchitecting those to run in that kind of environment. I was initially very excited for Unikernels, but because of this question (and others), I've really started becoming less enamored by them. I'm sure there are many places where they can be put to use, but for most of the software that I'm familiar with, there are still a bunch of kernel services that are quite handy. At what point of building these services into your unikernel/userspace application do you cross-over to basically building a new kernel. It really feels like the micro-kernel architecture would be a better fit in many of these cases. And yes, I understand that we can think of unikernels as VMs based micro-kernel services, but they have more isolation than traditional variations specifically allowing them to be used in AWS, etc. But still...
perbu 9y ago
Most people consider unikernels to be single process without threads. IncludeOS is rather liberal and has rudimentary multicore support. There is no preemption however so the "threads" will have to yield if there are more threads than cores. So more like fibers than threads, really. The hypervisor typically doesn't care. So if you are using multiple cores in your application it will schedule and de-schedule your VM as per the regular rules. IncludeOS is capable of booting on bare metal and we have some basic SMP tests passing. This is currently mostly of academic interest but if you don't mind having no pre-emption you can, once we improve our hardware support, have an application that runs on multiple physical cores. Perhaps this might be of interest to people with very strict latency requirements.

yazaddaruvala 9y ago

The whole point is there are many ways to do it and the application knows best. The API specifics are probably different but it's just setting some registers and the the program counter on that CPU core. The code likely looks very similar to a context switch.

derefr 9y ago

Consider that any complex language runtime (e.g. Erlang's BEAM, the JVM, etc.) has its own scheduler. If you convert such a runtime into a unikernel (such as in Erlang on Xen), the runtime scheduler is "the" scheduler. Besides languages, though, plenty of complex applications have their own schedulers. Databases, for example. Postgres-as-a-unikernel would make a lot of sense, because it's already doing its own equivalent to everything that OSes do. If your program is complex enough, it likely will already have an "optimized" form of internally-managed shared memory/message-passing multithreading (with core affinities et al) to allow it to allocate tasks to cores better than the OS can without the "insider knowledge" of what it's trying to accomplish. At that point, your program will end up with a scheduler "inside" whether you set out to build one or not. On the other hand, if your program isn't that complex, then it's very likely you'll get better performance out of your code by just making each process into a single-core unikernel and then packing instances onto however many vCPUs the hypervisor host offers.

bluejekyll 9y ago
I don't know much about the other Runtimes that you mention, but correct me where I'm wrong: In the case of the JVM, green threads were dropped a long time ago, and under the current paradigm, native threads are used (on all platforms?). So doesn't this need to be written into the JVM? Not impossible of course (or provided by the unikernel wrapper as it is in OSv). Similarly with Databases, Oracle and Postgres both use a process per connection client/server model. So a unikernel for these would be impractical without rearchitecting those to run in that kind of environment. I was initially very excited for Unikernels, but because of this question (and others), I've really started becoming less enamored by them. I'm sure there are many places where they can be put to use, but for most of the software that I'm familiar with, there are still a bunch of kernel services that are quite handy. At what point of building these services into your unikernel/userspace application do you cross-over to basically building a new kernel. It really feels like the micro-kernel architecture would be a better fit in many of these cases. And yes, I understand that we can think of unikernels as VMs based micro-kernel services, but they have more isolation than traditional variations specifically allowing them to be used in AWS, etc. But still...

perbu 9y ago

Most people consider unikernels to be single process without threads. IncludeOS is rather liberal and has rudimentary multicore support. There is no preemption however so the "threads" will have to yield if there are more threads than cores. So more like fibers than threads, really. The hypervisor typically doesn't care. So if you are using multiple cores in your application it will schedule and de-schedule your VM as per the regular rules. IncludeOS is capable of booting on bare metal and we have some basic SMP tests passing. This is currently mostly of academic interest but if you don't mind having no pre-emption you can, once we improve our hardware support, have an application that runs on multiple physical cores. Perhaps this might be of interest to people with very strict latency requirements.