5 ms·

IncludeOS – Run your application with zero overhead

96 points by turrini 5y ago

tshanmu 5y ago
How would includeos work with the Amazon specific network hardware or does something like firecracker abstract the network stack fully? Where do the network drivers live?
- ncmncm 5y ago
  I don't really know much specific about Amazon. But any network drivers you need are linked, or compiled, right into your program, instead of in a kernel you run on top of. If Amazon is providing a virtualized NIC, you compile in drivers for that under the IP stack you have also compiled in. Amazon's hypervisor might proxy access to actual or synthetic network devices, or might map a real NIC into your VM, or they might even boot your program on a real machine with real NIC, for all the world as if it is booting a proper kernel. It really is as if your app is your kernel, which just doesn't bother with any sort of user programs, because your code is all right there. Instead of doing system calls, it just does regular function calls to what would have been kernel services. Because it is all just one program, you can inline a lot of stuff that would normally have a system call or module indirection interposed. This would be a good place for LTO, to inline everything that could benefit by it.
  - smackeyacky 5y ago
    So its like a lambda that doesnt have the run time limits Amazon impose? I kinda like the idea (I like the AWS lambdas for short running tasks triggered by events) but if you want what is effectively a long running lambda, then a container is really the next logical step. I guess there is room between a lambda and a container but not much room.
    - curryst 5y ago
      Not really, it effectively adds a kernel to your app so your app is the OS. Like if you were booting from Grub, you would point Grub at your app binary directly. You don't boot into Linux and start your app, your app is the OS. Iirc the main advantages are eliminating a lot of context switching (your app is the kernel, so theres no context switching for kernel calls) and reducing unneeded features (i.e. if your app doesn't need networking, then it won't put any network drivers in your app-OS). Its mostly a performance optimization afaik. There's some supposed security features because it drops parts of the OS you don't need, which reduces the attack surface. I don't know if there are any additional risks from it though; making the app run in ring 0 makes me uneasy. I think that might just be because passing from userland to kernel land is a significant exploit in normal OS'.
esjeon 5y ago
I think the title and the main page are really bad at explaining what IncludeOS actually is. The author probably is trying to impress readers too much. The thing is, this is something called “unikernel[1]”, which basically can be linked directly into your application. With this, you just boot into your application instead of general-purpose OS like Linux. IIRC, it’s even possible to this on top of another OS. This comes with a lot of benefits, but with a lot of caveats, too. [1]: https://en.wikipedia.org/wiki/Unikernel https://en.wikipedia.org/wiki/Unikernel
- ksec 5y ago
  Or the link on the page [Tell me more] [1], >IncludeOS is an operating system written as a library. When building an application, the build system includes the code that would typically reside in your operating system into the application itself. When it is done building a bootloader is added. So, after a successful build, you now have a standalone application that boots without an operating system. The application is now capable of driving the hardware, it has an IP stack and manages its memory. >Unlike more traditional operating systems IncludeOS is meant for single task computers. There is only a single application running. For multiple security domains, we require multiple machines, and we leave the separation task to the hypervisor. They could have just called it Cloud-DOS. [1]https://www.includeos.org/technology.html https://www.includeos.org/technology.html
johannes1234321 5y ago
How well is the project doing? - They had big plans, but it seems the company behind it shut down sometime ago. Lat blog from two years ago, last commit a year ago.
- softblush 5y ago
  It's dead in the water. See also this comment when this project came up the last time on HN https://news.ycombinator.com/item?id=21905955 https://news.ycombinator.com/item?id=21905955 https://github.com/includeos/IncludeOS/issues/2211 https://github.com/includeos/IncludeOS/issues/2211 https://github.com/includeos/IncludeOS/issues/2183 https://github.com/includeos/IncludeOS/issues/2183 https://twitter.com/perbu/status/1215931242593554434?s=19 https://twitter.com/perbu/status/1215931242593554434?s=19
elzbardico 5y ago
Unfortunately this project is dead.
- halation_effect 5y ago
  I recommend Unikraft[1][2][3] as an alternative. It is under active development. [1] https://unikraft.org/ https://unikraft.org/ [2] https://github.com/unikra https://github.com/unikra ft/unikraft [3] https://dl.acm.org/doi/10.1145/3447786.3456248 https://dl.acm.org/doi/10.1145/3447786.3456248

96 points by turrini 5y ago

How would includeos work with the Amazon specific network hardware or does something like firecracker abstract the network stack fully? Where do the network drivers live?

ncmncm 5y ago
I don't really know much specific about Amazon. But any network drivers you need are linked, or compiled, right into your program, instead of in a kernel you run on top of. If Amazon is providing a virtualized NIC, you compile in drivers for that under the IP stack you have also compiled in. Amazon's hypervisor might proxy access to actual or synthetic network devices, or might map a real NIC into your VM, or they might even boot your program on a real machine with real NIC, for all the world as if it is booting a proper kernel. It really is as if your app is your kernel, which just doesn't bother with any sort of user programs, because your code is all right there. Instead of doing system calls, it just does regular function calls to what would have been kernel services. Because it is all just one program, you can inline a lot of stuff that would normally have a system call or module indirection interposed. This would be a good place for LTO, to inline everything that could benefit by it.
- smackeyacky 5y ago
  So its like a lambda that doesnt have the run time limits Amazon impose? I kinda like the idea (I like the AWS lambdas for short running tasks triggered by events) but if you want what is effectively a long running lambda, then a container is really the next logical step. I guess there is room between a lambda and a container but not much room.
  - curryst 5y ago
    Not really, it effectively adds a kernel to your app so your app is the OS. Like if you were booting from Grub, you would point Grub at your app binary directly. You don't boot into Linux and start your app, your app is the OS. Iirc the main advantages are eliminating a lot of context switching (your app is the kernel, so theres no context switching for kernel calls) and reducing unneeded features (i.e. if your app doesn't need networking, then it won't put any network drivers in your app-OS). Its mostly a performance optimization afaik. There's some supposed security features because it drops parts of the OS you don't need, which reduces the attack surface. I don't know if there are any additional risks from it though; making the app run in ring 0 makes me uneasy. I think that might just be because passing from userland to kernel land is a significant exploit in normal OS'.

esjeon 5y ago

I think the title and the main page are really bad at explaining what IncludeOS actually is. The author probably is trying to impress readers too much. The thing is, this is something called “unikernel[1]”, which basically can be linked directly into your application. With this, you just boot into your application instead of general-purpose OS like Linux. IIRC, it’s even possible to this on top of another OS. This comes with a lot of benefits, but with a lot of caveats, too. [1]: https://en.wikipedia.org/wiki/Unikernel https://en.wikipedia.org/wiki/Unikernel

ksec 5y ago
Or the link on the page [Tell me more] [1], >IncludeOS is an operating system written as a library. When building an application, the build system includes the code that would typically reside in your operating system into the application itself. When it is done building a bootloader is added. So, after a successful build, you now have a standalone application that boots without an operating system. The application is now capable of driving the hardware, it has an IP stack and manages its memory. >Unlike more traditional operating systems IncludeOS is meant for single task computers. There is only a single application running. For multiple security domains, we require multiple machines, and we leave the separation task to the hypervisor. They could have just called it Cloud-DOS. [1]https://www.includeos.org/technology.html https://www.includeos.org/technology.html

johannes1234321 5y ago

How well is the project doing? - They had big plans, but it seems the company behind it shut down sometime ago. Lat blog from two years ago, last commit a year ago.

softblush 5y ago
It's dead in the water. See also this comment when this project came up the last time on HN https://news.ycombinator.com/item?id=21905955 https://news.ycombinator.com/item?id=21905955 https://github.com/includeos/IncludeOS/issues/2211 https://github.com/includeos/IncludeOS/issues/2211 https://github.com/includeos/IncludeOS/issues/2183 https://github.com/includeos/IncludeOS/issues/2183 https://twitter.com/perbu/status/1215931242593554434?s=19 https://twitter.com/perbu/status/1215931242593554434?s=19

elzbardico 5y ago

Unfortunately this project is dead.

halation_effect 5y ago
I recommend Unikraft[1][2][3] as an alternative. It is under active development. [1] https://unikraft.org/ https://unikraft.org/ [2] https://github.com/unikra https://github.com/unikra ft/unikraft [3] https://dl.acm.org/doi/10.1145/3447786.3456248 https://dl.acm.org/doi/10.1145/3447786.3456248