30 ms·

I'm really sold on the idea: Instead of a full-blown OS, you compile your application with a thin layer of support libraries that provide the OS features that y

by v1ne 3y ago

I'm really sold on the idea: Instead of a full-blown OS, you compile your application with a thin layer of support libraries that provide the OS features that your application needs (network, I/O) and that talks to a hypervisor. I mean, if your application runs in a virtualized environment, there's little need to SSH into the system in the first place (except for debugging purposes). Thus, why bother with a full-blown operating system? In the virtualized case, the true OS logic is in the host OS anyway, talking to the hardware. Cutting out all those superfluous layers in the app VM makes it small, start quickly, and gives less attack surface. Sounds like a win-win to me. In contrast, FreeBSD on Firecracker is a full-blown OS, but boots in 25 milliseconds on the Firecracker hypervisor.

BrittonR 3y ago
Isn’t that the aim of webassembly?
- ReleaseCandidat 3y ago
  Not really, as the filesystem you access from Webassembly needs to be in a sort of sandbox or container too.
- pjmlp 3y ago
  Not at all, WebAssembly is replacement of PNaCL browser vendors could agree on. Then a bunch of folks, with some VC money, decided to make the second coming of Java and .NET with it.
Osiris 3y ago
Isn’t this the same idea behind unikernels?
- edf13 3y ago
  That’s what this is… > MirageOS is a library operating system that constructs unikernels for secure, high-performance network applications across a variety of cloud computing and mobile platforms.
- repelsteeltje 3y ago
  Mirage is a unikernel. The catch here is Ocaml. You can design your unikernel "bare metal" in any way you want and make it do nothing but the essentials you need ... and design it in any language you want as long as that language is OCaml.
  - deleted 3y ago
    [deleted]
  - samuell 3y ago
    And on that note, I just found this list of UniKernel projects: http://unikernel.org/projects/ http://unikernel.org/projects/ I have especially had hopes for the UniK [1] project, as it was/is written in Go AFAIK. I see now it incorporates work from the Mirage project as well. Not sure what is the status of this project anymore though. [1] https://github.com/solo-io/unik https://github.com/solo-io/unik
    - mk89 3y ago
      I didn't read your message, so I just clicked and I found that great nice project from Solo.io (the company behind Gloo Edge, etc.). Freaking awesome... I thought: > Not sure what is the status of this project anymore what the hell! :D last commit was 4 years ago!
      - samuell 3y ago
        Hah, yea, should have checked that :D ... but yea, I stopped hearing news from solo.io / UniK some time ago, so it makes sense.
    - eyberg 3y ago
      Unik was just a build tool that utilized other projects like Rump, Mirage, IncludeOS, etc. It's now dead since Solo pivoted a very long time ago to service mesh/api gateways. The GoRump port they use was from us and then we realized we needed to code our own from the ground up for many reasons so we wrote https://nanos.org https://nanos.org (runs as a go unikernel in GCP).
- eriangazag 3y ago
  Absolutely, MirageOS is indeed pivotal in the development of unikernels. MirageOS not only developed the unikernel concept but also coined the term 'unikernel'. MirageOS represents a significant shift in cloud and network computing, focusing on building highly specialized, secure, and efficient unikernels that streamline application deployment by shedding unnecessary components. It also puts extra focus on security, with most of the stack — including TCP and TLS — being rewritten in a type-safe language (OCaml). Currently, MirageOS is expanding to include bare-metal and embedded systems, aiming to bring the same level of security and efficiency to this domain. The move towards embedded systems is a natural progression for MirageOS, given our emphasis on minimalism and security in environments where resources are limited and reliability is crucial. See for instance what is happening with SpaceOS: https://tarides.com/blog/2023-07-31-ocaml-in-space-welcome-spaceos/ https://tarides.com/blog/2023-07-31-ocaml-in-space-welcome-s...
rapsey 3y ago
> there's little need to SSH into the system in the first place (except for debugging purposes). Pretty big except though. This concept has been done multiple times and has always failed. It's solving a problem that few care about.
- deleted 3y ago
  [deleted]
- SenAnder 3y ago
  > It's solving a problem that few care about. Security?
- eightnoteight 3y ago
  exactly this, debugging is a pretty big part when it comes to operational aspects. but I feel like similar to kubernetes container injection systems. I suspect in future we would see dynamic injection into the unikernel where you can enable debugging environment instantly. like the library unikernel exposes a plugin system that you can inject in future, this way the unikernel is lightweight and only gets bloated when you eventually want to debug something.
  - eightnoteight 3y ago
    actually I think this might be the intention of mirage as well, especially since they put "modularity" in the title
  - hannesm 3y ago
    What you already can do is attaching gdb -- either live or at a core dump. Please see https://github.com/Solo5/solo5/blob/master/docs/debugging.md https://github.com/Solo5/solo5/blob/master/docs/debugging.md for detailed instructions ;) What you as well can do is record-replay -- well, at least there used to be this option, I'm now lost whether it has never been merged anywhere, or it is stuck in some PR somewhere. This was truly great - since the external interface is so thin, it is easily doable to dump all external events (API calls and returns) onto disk and replay one-by-one, inspecting the state.
lazzlazzlazz 3y ago
But what's the benefit of cutting out the full-blown operating system? It's rare that such things would be the bottleneck.
- pjmlp 3y ago
  Performance and security, basically. Similar to trying to run containers bare metal directly on top of type 1 hypervisors, except unikernels are designed for this purpose.
- fhuici 3y ago
  At least for cloud deployments you don't need a full blown OS when all you want to do is run apps like NGINX or Redis. If you use a unikernel instead, then you severely (orders of magnitude) reduce cold boot times, memory consumption, server density (thousands on hw-isolated instances on a single server) and TCB.
  - lazzlazzlazz 3y ago
    Makes sense. Thank you.
datadeft 3y ago
> I'm really sold on the idea: Instead of a full-blown OS, You mean you are sold on the idea of going back to the 80s. We used to have an operating system with every game released[1]. 1. https://youtu.be/kZRE7HIO3vk?t=1114 https://youtu.be/kZRE7HIO3vk?t=1114
- broken-kebab 3y ago
  Well, technology is known to spiral around ideas with time, so nothing's wrong with that
- reycharles 3y ago
  A big difference in my opinion is that projects like Mirage does not try to reimplement the same hardware drivers as the host system and instead implements much simpler drivers for virtual hardware. For example, how do you write to a block device? You make a hypercall with a "handle", offset into the block device, the number of blocks to write and a pointer to memory to be written. There's no pretending it's a spinny disk with sectors and heads or what have you. https://github.com/Solo5/solo5/blob/bf29b8af11feec9dbc2e74ccdc69bc1e0cbf5dfe/bindings/hvt/block.c#L25-L56 https://github.com/Solo5/solo5/blob/bf29b8af11feec9dbc2e74cc...
  - Veserv 3y ago
    That is just infinitely worse than calling write() on a file handle like a regular application. The hypervisor + OS + application model is stupid. The hypervisor and OS are duplicative. If you can rewrite the application (as is needed for a unikernel design) there is no reason to use a hypervisor in your stack at all. You should just be running a OS directly on the bare metal with no hypervisor at all. The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary. Everything else about them is strictly inferior to a standard application on OS model.
    - 0xDEAFBEAD 3y ago
      >The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary. Thoughts on Qubes?
hannesm 3y ago
To be a bit more precise, what we achieved so far: apart from web servers and TLS reverse proxies (with DNS-backed let's encrypt provisioning), DNS authoritative servers, a CalDAV server, git client and servers, SMTP stack (including dkim etc.), OpenVPN implementation, archive mirror (using tar as file system), monitoring with syslog and influx, ... A big milestone was binary releases and deployment instructions, see https://robur.coop/Projects/Reproducible_builds https://robur.coop/Projects/Reproducible_builds see more at https://blog.robur.coop https://blog.robur.coop https://hannes.robur.coop https://hannes.robur.coop https://blog.osau.re/ https://blog.osau.re/ https://reyn.ir/archive.html https://reyn.ir/archive.html If you're curious how/what to integrate MirageOS into your infrastructure, please reach out (best via email).
troupo 3y ago
> I'm really sold on the idea: Instead of a full-blown OS, you compile your application with a thin layer of support libraries that provide the OS features that your application needs I'm really sad that unikernels never took off, even there seemed to be a lot of excitement around them a few years back.
- fhuici 3y ago
  The second wave is coming :) . At www.unikraft.org we're hard at work providing a Linux-API compatible unikernel (run unmodified apps/langs) and combining it with tooling like docker and k8s.

by v1ne 3y ago

BrittonR 3y ago

Isn’t that the aim of webassembly?

ReleaseCandidat 3y ago
Not really, as the filesystem you access from Webassembly needs to be in a sort of sandbox or container too.
pjmlp 3y ago
Not at all, WebAssembly is replacement of PNaCL browser vendors could agree on. Then a bunch of folks, with some VC money, decided to make the second coming of Java and .NET with it.

Osiris 3y ago

Isn’t this the same idea behind unikernels?

edf13 3y ago
That’s what this is… > MirageOS is a library operating system that constructs unikernels for secure, high-performance network applications across a variety of cloud computing and mobile platforms.
repelsteeltje 3y ago
Mirage is a unikernel. The catch here is Ocaml. You can design your unikernel "bare metal" in any way you want and make it do nothing but the essentials you need ... and design it in any language you want as long as that language is OCaml.
- deleted 3y ago
  [deleted]
- samuell 3y ago
  And on that note, I just found this list of UniKernel projects: http://unikernel.org/projects/ http://unikernel.org/projects/ I have especially had hopes for the UniK [1] project, as it was/is written in Go AFAIK. I see now it incorporates work from the Mirage project as well. Not sure what is the status of this project anymore though. [1] https://github.com/solo-io/unik https://github.com/solo-io/unik
  - mk89 3y ago
    I didn't read your message, so I just clicked and I found that great nice project from Solo.io (the company behind Gloo Edge, etc.). Freaking awesome... I thought: > Not sure what is the status of this project anymore what the hell! :D last commit was 4 years ago!
    - samuell 3y ago
      Hah, yea, should have checked that :D ... but yea, I stopped hearing news from solo.io / UniK some time ago, so it makes sense.
  - eyberg 3y ago
    Unik was just a build tool that utilized other projects like Rump, Mirage, IncludeOS, etc. It's now dead since Solo pivoted a very long time ago to service mesh/api gateways. The GoRump port they use was from us and then we realized we needed to code our own from the ground up for many reasons so we wrote https://nanos.org https://nanos.org (runs as a go unikernel in GCP).
eriangazag 3y ago
Absolutely, MirageOS is indeed pivotal in the development of unikernels. MirageOS not only developed the unikernel concept but also coined the term 'unikernel'. MirageOS represents a significant shift in cloud and network computing, focusing on building highly specialized, secure, and efficient unikernels that streamline application deployment by shedding unnecessary components. It also puts extra focus on security, with most of the stack — including TCP and TLS — being rewritten in a type-safe language (OCaml). Currently, MirageOS is expanding to include bare-metal and embedded systems, aiming to bring the same level of security and efficiency to this domain. The move towards embedded systems is a natural progression for MirageOS, given our emphasis on minimalism and security in environments where resources are limited and reliability is crucial. See for instance what is happening with SpaceOS: https://tarides.com/blog/2023-07-31-ocaml-in-space-welcome-spaceos/ https://tarides.com/blog/2023-07-31-ocaml-in-space-welcome-s...

rapsey 3y ago

> there's little need to SSH into the system in the first place (except for debugging purposes). Pretty big except though. This concept has been done multiple times and has always failed. It's solving a problem that few care about.

deleted 3y ago
[deleted]
SenAnder 3y ago
> It's solving a problem that few care about. Security?
eightnoteight 3y ago
exactly this, debugging is a pretty big part when it comes to operational aspects. but I feel like similar to kubernetes container injection systems. I suspect in future we would see dynamic injection into the unikernel where you can enable debugging environment instantly. like the library unikernel exposes a plugin system that you can inject in future, this way the unikernel is lightweight and only gets bloated when you eventually want to debug something.
- eightnoteight 3y ago
  actually I think this might be the intention of mirage as well, especially since they put "modularity" in the title
- hannesm 3y ago
  What you already can do is attaching gdb -- either live or at a core dump. Please see https://github.com/Solo5/solo5/blob/master/docs/debugging.md https://github.com/Solo5/solo5/blob/master/docs/debugging.md for detailed instructions ;) What you as well can do is record-replay -- well, at least there used to be this option, I'm now lost whether it has never been merged anywhere, or it is stuck in some PR somewhere. This was truly great - since the external interface is so thin, it is easily doable to dump all external events (API calls and returns) onto disk and replay one-by-one, inspecting the state.

lazzlazzlazz 3y ago

But what's the benefit of cutting out the full-blown operating system? It's rare that such things would be the bottleneck.

pjmlp 3y ago
Performance and security, basically. Similar to trying to run containers bare metal directly on top of type 1 hypervisors, except unikernels are designed for this purpose.
fhuici 3y ago
At least for cloud deployments you don't need a full blown OS when all you want to do is run apps like NGINX or Redis. If you use a unikernel instead, then you severely (orders of magnitude) reduce cold boot times, memory consumption, server density (thousands on hw-isolated instances on a single server) and TCB.
- lazzlazzlazz 3y ago
  Makes sense. Thank you.

datadeft 3y ago

> I'm really sold on the idea: Instead of a full-blown OS, You mean you are sold on the idea of going back to the 80s. We used to have an operating system with every game released[1]. 1. https://youtu.be/kZRE7HIO3vk?t=1114 https://youtu.be/kZRE7HIO3vk?t=1114

broken-kebab 3y ago
Well, technology is known to spiral around ideas with time, so nothing's wrong with that
reycharles 3y ago
A big difference in my opinion is that projects like Mirage does not try to reimplement the same hardware drivers as the host system and instead implements much simpler drivers for virtual hardware. For example, how do you write to a block device? You make a hypercall with a "handle", offset into the block device, the number of blocks to write and a pointer to memory to be written. There's no pretending it's a spinny disk with sectors and heads or what have you. https://github.com/Solo5/solo5/blob/bf29b8af11feec9dbc2e74ccdc69bc1e0cbf5dfe/bindings/hvt/block.c#L25-L56 https://github.com/Solo5/solo5/blob/bf29b8af11feec9dbc2e74cc...
- Veserv 3y ago
  That is just infinitely worse than calling write() on a file handle like a regular application. The hypervisor + OS + application model is stupid. The hypervisor and OS are duplicative. If you can rewrite the application (as is needed for a unikernel design) there is no reason to use a hypervisor in your stack at all. You should just be running a OS directly on the bare metal with no hypervisor at all. The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary. Everything else about them is strictly inferior to a standard application on OS model.
  - 0xDEAFBEAD 3y ago
    >The only reason to prefer a unikernel is because you wrongly believe that hypervisors are a security boundary. Thoughts on Qubes?

hannesm 3y ago

To be a bit more precise, what we achieved so far: apart from web servers and TLS reverse proxies (with DNS-backed let's encrypt provisioning), DNS authoritative servers, a CalDAV server, git client and servers, SMTP stack (including dkim etc.), OpenVPN implementation, archive mirror (using tar as file system), monitoring with syslog and influx, ... A big milestone was binary releases and deployment instructions, see https://robur.coop/Projects/Reproducible_builds https://robur.coop/Projects/Reproducible_builds see more at https://blog.robur.coop https://blog.robur.coop https://hannes.robur.coop https://hannes.robur.coop https://blog.osau.re/ https://blog.osau.re/ https://reyn.ir/archive.html https://reyn.ir/archive.html If you're curious how/what to integrate MirageOS into your infrastructure, please reach out (best via email).

troupo 3y ago

> I'm really sold on the idea: Instead of a full-blown OS, you compile your application with a thin layer of support libraries that provide the OS features that your application needs I'm really sad that unikernels never took off, even there seemed to be a lot of excitement around them a few years back.

fhuici 3y ago
The second wave is coming :) . At www.unikraft.org we're hard at work providing a Linux-API compatible unikernel (run unmodified apps/langs) and combining it with tooling like docker and k8s.