5 ms·

I am fascinated by Unikernels. But how do they deal with things that need to fork or run multiple processes? Aren't they restricted to single processes?

by onebot 5y ago

actionfromafar 5y ago
The new version of OCaml got multiprocessor support recently. https://www.infoq.com/news/2021/10/ocaml-5-multicore/ https://www.infoq.com/news/2021/10/ocaml-5-multicore/
sanxiyn 5y ago
Yes they are, but you don't need to fork for a lot of workloads.
hannesm 5y ago
We use asynchronous tasks in MirageOS (cooperative multitasking) using lwt http://ocsigen.org/lwt/latest/manual/manual http://ocsigen.org/lwt/latest/manual/manual, so you can serve multiple network connections at the same time. Since there are no processes, there's no concept of "fork", but indeed you can run multiple tasks at the same time (using the same address space). Why again would you need multiple processes? Since the programming language OCaml has a semantics and is memory-safe, there's no strong reason for isolation at execution time (apart from the C bits, which we try to keep to a minimum and compile with runtime safety flags (red-zone for stack protection, mapping execute-only (seems to only reliably work on OpenBSD), etc.)).
- asymmetric 5y ago
  Is this also how you deal with logging/monitoring? Do you have a concept of logging daemon? How would I run prometheus exporter-like things? Is there a FAQ where questions along these lines are answered?
  - avsm 5y ago
    Just substitute 'microservice' with 'unikernel' and you do broadly the same things. There's a prometheus library that you link with the MirageOS unikernel and it exports using that: https://github.com/mirage/prometheus https://github.com/mirage/prometheus No FAQ for this sort of thing yet, but we should start assembling one sometime soon. Questions like this very welcome on the discussion forums: https://discuss.ocaml.org/t/ann-mirageos-4-0/9598 https://discuss.ocaml.org/t/ann-mirageos-4-0/9598 to help us get started. There's a nice collection of unikernels over at: https://github.com/roburio/unikernels https://github.com/roburio/unikernels and https://github.com/tarides/unikernels https://github.com/tarides/unikernels for various infrastructure pieces (like https, smtp, dns, ip filters, etc) that are good to crib from for your own infrastructure.
  - hannesm 5y ago
    Maybe https://hannes.robur.coop/Posts/Monitoring https://hannes.robur.coop/Posts/Monitoring sheds some light how to monitor MirageOS unikernels ;)
- scns 5y ago
  Conurrency instead of parallelism.
  - wmf 5y ago
    No amount of concurrency on a single core can equal the performance of multiple cores.
    - pjmlp 5y ago
      That is why you deploy a process per core, with affinity so that it doesn't jump around.
    - cestith 5y ago
      You can run more than one unikernel per machine. That's why things like KVM or Bhyve are targets instead of just bare metal.
- gpderetta 5y ago
  What about multiprocessing? I assume that MirageOS can take advantage of multiple cores (or do you need separate instances per core?). In this case is the system still shared memory? Also I would say there are reasons for isolation beyond memory safety.
  - hannesm 5y ago
    MirageOS is - similar to the latest OCaml release - only using a single CPU core. You can run multiple unikernels, one on each core. If doing that, you can use Xen vchan (shared memory), or TCP for marshalling. > Also I would say there are reasons for isolation beyond memory safety. Would you mind to elaborate which reasons you are thinking of?
    - 0xbkt 5y ago
      > only using a single CPU core How does it go when you deploy a unikernel on EC2 (or on any IaaS where the hypervisor is managed unlike bare-metal) with multiple cores? Is there a way to start a unikernel per core on a single instance, or are you bound to use single core instance types only?
    - gpderetta 5y ago
      > MirageOS is - similar to the latest OCaml release - only using a single CPU core. Thanks. Is that going to change now that OCaml is finally getting proper multicore support? >> Also I would say there are reasons for isolation beyond memory safety. >Would you mind to elaborate which reasons you are thinking of? Memory safety in a sense protects the integrity of the 'runtime', but only partially help to protect business level integrity. A task might still tricked (by mistake or malice) to access objects it is not supposed to. I'm sure that OCaml has enough abstractions to help prevent that, but full isolation of tasks is a blunt and effective tool.
avsm 5y ago
This all started because we wanted to _get away_ from the need to fork or run multiple processes, since that's so hard in a variety of hardware architectures (like mobile or embedded). Other researchers also share the dislike of fork... https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf https://www.microsoft.com/en-us/research/uploads/prod/2019/0... A 2010 paper where I sketched out some of the early ideas around "multiscale" is here: https://anil.recoil.org/papers/2010-bcs-visions.pdf https://anil.recoil.org/papers/2010-bcs-visions.pdf It took a little longer than I'd planned, but thanks to the hard work of so many MirageOS contributors, now's a pretty good time to glue back personal containers and self-hosted data management infrastructure again! Unikernel-based messaging has really come together in the past couple of years: https://tarides.com/blog/2022-03-08-secure-virtual-messages-in-a-bottle-with-scop https://tarides.com/blog/2022-03-08-secure-virtual-messages-...
rwmj 5y ago
Unikernel Linux (UKL - https://github.com/unikernellinux https://github.com/unikernellinux) actually does allow you to fork. The main unikernel program runs in kernel space linked to Linux, and after a fork you get a new, regular userspace process.

I am fascinated by Unikernels. But how do they deal with things that need to fork or run multiple processes? Aren't they restricted to single processes?

by onebot 5y ago

I am fascinated by Unikernels. But how do they deal with things that need to fork or run multiple processes? Aren't they restricted to single processes?

actionfromafar 5y ago

The new version of OCaml got multiprocessor support recently. https://www.infoq.com/news/2021/10/ocaml-5-multicore/ https://www.infoq.com/news/2021/10/ocaml-5-multicore/

sanxiyn 5y ago

Yes they are, but you don't need to fork for a lot of workloads.

hannesm 5y ago

We use asynchronous tasks in MirageOS (cooperative multitasking) using lwt http://ocsigen.org/lwt/latest/manual/manual http://ocsigen.org/lwt/latest/manual/manual, so you can serve multiple network connections at the same time. Since there are no processes, there's no concept of "fork", but indeed you can run multiple tasks at the same time (using the same address space). Why again would you need multiple processes? Since the programming language OCaml has a semantics and is memory-safe, there's no strong reason for isolation at execution time (apart from the C bits, which we try to keep to a minimum and compile with runtime safety flags (red-zone for stack protection, mapping execute-only (seems to only reliably work on OpenBSD), etc.)).

asymmetric 5y ago
Is this also how you deal with logging/monitoring? Do you have a concept of logging daemon? How would I run prometheus exporter-like things? Is there a FAQ where questions along these lines are answered?
- avsm 5y ago
  Just substitute 'microservice' with 'unikernel' and you do broadly the same things. There's a prometheus library that you link with the MirageOS unikernel and it exports using that: https://github.com/mirage/prometheus https://github.com/mirage/prometheus No FAQ for this sort of thing yet, but we should start assembling one sometime soon. Questions like this very welcome on the discussion forums: https://discuss.ocaml.org/t/ann-mirageos-4-0/9598 https://discuss.ocaml.org/t/ann-mirageos-4-0/9598 to help us get started. There's a nice collection of unikernels over at: https://github.com/roburio/unikernels https://github.com/roburio/unikernels and https://github.com/tarides/unikernels https://github.com/tarides/unikernels for various infrastructure pieces (like https, smtp, dns, ip filters, etc) that are good to crib from for your own infrastructure.
- hannesm 5y ago
  Maybe https://hannes.robur.coop/Posts/Monitoring https://hannes.robur.coop/Posts/Monitoring sheds some light how to monitor MirageOS unikernels ;)
scns 5y ago
Conurrency instead of parallelism.
- wmf 5y ago
  No amount of concurrency on a single core can equal the performance of multiple cores.
  - pjmlp 5y ago
    That is why you deploy a process per core, with affinity so that it doesn't jump around.
  - cestith 5y ago
    You can run more than one unikernel per machine. That's why things like KVM or Bhyve are targets instead of just bare metal.
gpderetta 5y ago
What about multiprocessing? I assume that MirageOS can take advantage of multiple cores (or do you need separate instances per core?). In this case is the system still shared memory? Also I would say there are reasons for isolation beyond memory safety.
- hannesm 5y ago
  MirageOS is - similar to the latest OCaml release - only using a single CPU core. You can run multiple unikernels, one on each core. If doing that, you can use Xen vchan (shared memory), or TCP for marshalling. > Also I would say there are reasons for isolation beyond memory safety. Would you mind to elaborate which reasons you are thinking of?
  - 0xbkt 5y ago
    > only using a single CPU core How does it go when you deploy a unikernel on EC2 (or on any IaaS where the hypervisor is managed unlike bare-metal) with multiple cores? Is there a way to start a unikernel per core on a single instance, or are you bound to use single core instance types only?
  - gpderetta 5y ago
    > MirageOS is - similar to the latest OCaml release - only using a single CPU core. Thanks. Is that going to change now that OCaml is finally getting proper multicore support? >> Also I would say there are reasons for isolation beyond memory safety. >Would you mind to elaborate which reasons you are thinking of? Memory safety in a sense protects the integrity of the 'runtime', but only partially help to protect business level integrity. A task might still tricked (by mistake or malice) to access objects it is not supposed to. I'm sure that OCaml has enough abstractions to help prevent that, but full isolation of tasks is a blunt and effective tool.

avsm 5y ago

This all started because we wanted to _get away_ from the need to fork or run multiple processes, since that's so hard in a variety of hardware architectures (like mobile or embedded). Other researchers also share the dislike of fork... https://www.microsoft.com/en-us/research/uploads/prod/2019/04/fork-hotos19.pdf https://www.microsoft.com/en-us/research/uploads/prod/2019/0... A 2010 paper where I sketched out some of the early ideas around "multiscale" is here: https://anil.recoil.org/papers/2010-bcs-visions.pdf https://anil.recoil.org/papers/2010-bcs-visions.pdf It took a little longer than I'd planned, but thanks to the hard work of so many MirageOS contributors, now's a pretty good time to glue back personal containers and self-hosted data management infrastructure again! Unikernel-based messaging has really come together in the past couple of years: https://tarides.com/blog/2022-03-08-secure-virtual-messages-in-a-bottle-with-scop https://tarides.com/blog/2022-03-08-secure-virtual-messages-...

rwmj 5y ago

Unikernel Linux (UKL - https://github.com/unikernellinux https://github.com/unikernellinux) actually does allow you to fork. The main unikernel program runs in kernel space linked to Linux, and after a fork you get a new, regular userspace process.