4 ms·

HermiTux – A Linux binary-compatible unikernel

umanghere 6y ago
This seems to be quite similar to the other unikernel that was posted recently [1]. It would be useful to have an in-depth comparison of these (and other) unikernels, especially with regard to performance and compatibility. I can see that HermiTux has the ability to select only the syscalls needed by the embedded program, and to transform syscalls to function calls, which seems quite interesting. [1]: https://news.ycombinator.com/item?id=25405672 https://news.ycombinator.com/item?id=25405672
- codetrotter 6y ago
  > transform syscalls to function calls I thought that was kind of the point of a unikernel. Or am I mistaken? Does the other one from the other day not do that?
  - michaelmior 6y ago
    That's certainly part of the point (and probably one of the main performance benefits). However, even if this doesn't happen, a unikernel still gives the advantage that it's much more difficult to run undesired code and is likely to improve security.
- sitkack 6y ago
  https://github.com/cloudius-systems/osv https://github.com/cloudius-systems/osv is an offshot of a project from ScyllaDB.
  - hardwaresofton 6y ago
    Bit of a nitpick, but from what I understand they were making osv and pivoted to ScyllaDB, correct?
    - sitkack 6y ago
      I only know what the readme says. > OSv was originally designed and implemented by Cloudius Systems (now ScyllaDB) however currently it is being maintained and enhanced by a small community of volunteers.
      - hardwaresofton 6y ago
        Yeah that doesn't sound conclusive in either direction -- not clear if they made this on the way to ScyllaDB. I took another look just in case and found a source[0]: > After much research into the market and technology, in mid-2014 the team decided to pivot away from OSv (still a live project). Instead they decided to embrace the Cassandra architecture but rewrite it from scratch in native code with all of the know-how of years of kernel/hypervisor coding. This would become the Scylla database. [0]: https://www.scylladb.com/company/the-scylla-story/ https://www.scylladb.com/company/the-scylla-story/
        PeterCorless 6y ago
        Correct. OSv came first, and then Seastar + Scylla.
Semaphor 6y ago
April 7, 2019 26 comments: https://news.ycombinator.com/item?id=19593497 https://news.ycombinator.com/item?id=19593497
tutfbhuf 6y ago
I would like to see some benchmarks and I would like to know what the limitations are. What kind of Linux software won't run on HermiTux?
- corty 6y ago
  Everything that needs more than a single process won't run, either because it needs a system service or because it does multi-process-sandboxing. Single-address-space is extremely limiting for a lot of stuff, so most webservers are out. Not sure if threads would even work. Any desktop software is out anyways. Basically you will have to write your own software for this model of operation, or pick stuff that is accidentially suited, which won't be a lot.
gandalfgeek 6y ago
Made a short explainer video on this paper: https://youtu.be/wHLqSb4Op4s https://youtu.be/wHLqSb4Op4s I've covered a bunch of other unikernel and VM papers and that might give you a broad idea of the design space here: https://www.youtube.com/playlist?list=PLOuhQnxVenf0dMrMDrq7VT1GwHCib9Sc0 https://www.youtube.com/playlist?list=PLOuhQnxVenf0dMrMDrq7V...
- llimllib 6y ago
  That video is great! thanks
anderspitman 6y ago
I was watching a Jonathan Blow clip recently where he was bemoaning all the layers of abstraction we've put over executables, which are all x86 (or ARM etc) at the end of the day. It got me wondering if it would be feasible to have a single executable format that works across OSes. My OS knowledge is pretty rusty. What are the main roadblocks? Will we always have to have at least some runtime layer like WebAssembly?
- cwyers 6y ago
  Well, there's two things here. One, typically executable code is packaged in a file along with other content, otherwise, the OS doesn't know that the contents are intended to be executed. The broader problem is, if you could somehow work around that... what are you going to _do_? You need OS-specific code to read and write files, to display things, to get input from the keyboard/mouse, to talk to the network. It's not like Ye Olden Days where you can just write directly to the display memory and have things show up, the OS controls your access to all of the hardware. An OS-agnostic executable can't accomplish much on a modern PC.
- corty 6y ago
  There are a few roadblocks that probably will not be overcome, at least not without introducing more of those abstraction layers. The binary format itself is a roadblock, because the loader of your OS needs to know that in order to load and start your program. To be compatible e.g. for PE and ELF, you would either need another loader or binaries that look like both formats at the same time. The latter isn't really possible (except for some interesting puzzles) the former is what binfmt support in Linux does. On other OSes, you do that manually like in "java foobar.class" or "wine something.exe". The syscall convention is another problem, every OS does them somewhat differently, and even the same OS on the same architecture can have different conventions one may use: int 0x80 (or whatever number one likes), call gates, SYSENTER, traps or trampolines. I certainly forgot a few. Within those there is another variability around parameter passing and return values (register, stack, magic location, all of the above), buffer handling for larger parameters and preserved registers. Syscall names, numbers and semantics are all totally different. Then there are mechanisms like unix signals and callbacks from the OS, futures and zero-copy-IO that are very very OS specific. There also is absolutely no smallest common denominator. Memory layout has peculiarities, like randomisation, different sections, who sets up the stack, that kind of stuff. Mostly the linker and loader will do things here, but again, it depends. And then there are libraries. Even if you use a static binary (which you probably would have to), modern OSes still expect you to dynamically link some libraries like all syscall entry points or stubs, things like libnss, etc. All in all quite impossible without at least another layer like wine or the java vm, some big loader switch/case and quite a bit of fiddling to adapt to some hard-to-adapt peculiarities. E.g. the Linux kernel has quite a few things built specially for wine to work.

This seems to be quite similar to the other unikernel that was posted recently [1]. It would be useful to have an in-depth comparison of these (and other) unikernels, especially with regard to performance and compatibility. I can see that HermiTux has the ability to select only the syscalls needed by the embedded program, and to transform syscalls to function calls, which seems quite interesting. [1]: https://news.ycombinator.com/item?id=25405672 https://news.ycombinator.com/item?id=25405672

codetrotter 6y ago
> transform syscalls to function calls I thought that was kind of the point of a unikernel. Or am I mistaken? Does the other one from the other day not do that?
- michaelmior 6y ago
  That's certainly part of the point (and probably one of the main performance benefits). However, even if this doesn't happen, a unikernel still gives the advantage that it's much more difficult to run undesired code and is likely to improve security.
sitkack 6y ago
https://github.com/cloudius-systems/osv https://github.com/cloudius-systems/osv is an offshot of a project from ScyllaDB.
- hardwaresofton 6y ago
  Bit of a nitpick, but from what I understand they were making osv and pivoted to ScyllaDB, correct?
  - sitkack 6y ago
    I only know what the readme says. > OSv was originally designed and implemented by Cloudius Systems (now ScyllaDB) however currently it is being maintained and enhanced by a small community of volunteers.
    - hardwaresofton 6y ago
      Yeah that doesn't sound conclusive in either direction -- not clear if they made this on the way to ScyllaDB. I took another look just in case and found a source[0]: > After much research into the market and technology, in mid-2014 the team decided to pivot away from OSv (still a live project). Instead they decided to embrace the Cassandra architecture but rewrite it from scratch in native code with all of the know-how of years of kernel/hypervisor coding. This would become the Scylla database. [0]: https://www.scylladb.com/company/the-scylla-story/ https://www.scylladb.com/company/the-scylla-story/
      - PeterCorless 6y ago
        Correct. OSv came first, and then Seastar + Scylla.

Semaphor 6y ago

April 7, 2019 26 comments: https://news.ycombinator.com/item?id=19593497 https://news.ycombinator.com/item?id=19593497

tutfbhuf 6y ago

I would like to see some benchmarks and I would like to know what the limitations are. What kind of Linux software won't run on HermiTux?

corty 6y ago
Everything that needs more than a single process won't run, either because it needs a system service or because it does multi-process-sandboxing. Single-address-space is extremely limiting for a lot of stuff, so most webservers are out. Not sure if threads would even work. Any desktop software is out anyways. Basically you will have to write your own software for this model of operation, or pick stuff that is accidentially suited, which won't be a lot.

gandalfgeek 6y ago

Made a short explainer video on this paper: https://youtu.be/wHLqSb4Op4s https://youtu.be/wHLqSb4Op4s I've covered a bunch of other unikernel and VM papers and that might give you a broad idea of the design space here: https://www.youtube.com/playlist?list=PLOuhQnxVenf0dMrMDrq7VT1GwHCib9Sc0 https://www.youtube.com/playlist?list=PLOuhQnxVenf0dMrMDrq7V...

llimllib 6y ago
That video is great! thanks

anderspitman 6y ago

I was watching a Jonathan Blow clip recently where he was bemoaning all the layers of abstraction we've put over executables, which are all x86 (or ARM etc) at the end of the day. It got me wondering if it would be feasible to have a single executable format that works across OSes. My OS knowledge is pretty rusty. What are the main roadblocks? Will we always have to have at least some runtime layer like WebAssembly?

cwyers 6y ago
Well, there's two things here. One, typically executable code is packaged in a file along with other content, otherwise, the OS doesn't know that the contents are intended to be executed. The broader problem is, if you could somehow work around that... what are you going to _do_? You need OS-specific code to read and write files, to display things, to get input from the keyboard/mouse, to talk to the network. It's not like Ye Olden Days where you can just write directly to the display memory and have things show up, the OS controls your access to all of the hardware. An OS-agnostic executable can't accomplish much on a modern PC.
corty 6y ago
There are a few roadblocks that probably will not be overcome, at least not without introducing more of those abstraction layers. The binary format itself is a roadblock, because the loader of your OS needs to know that in order to load and start your program. To be compatible e.g. for PE and ELF, you would either need another loader or binaries that look like both formats at the same time. The latter isn't really possible (except for some interesting puzzles) the former is what binfmt support in Linux does. On other OSes, you do that manually like in "java foobar.class" or "wine something.exe". The syscall convention is another problem, every OS does them somewhat differently, and even the same OS on the same architecture can have different conventions one may use: int 0x80 (or whatever number one likes), call gates, SYSENTER, traps or trampolines. I certainly forgot a few. Within those there is another variability around parameter passing and return values (register, stack, magic location, all of the above), buffer handling for larger parameters and preserved registers. Syscall names, numbers and semantics are all totally different. Then there are mechanisms like unix signals and callbacks from the OS, futures and zero-copy-IO that are very very OS specific. There also is absolutely no smallest common denominator. Memory layout has peculiarities, like randomisation, different sections, who sets up the stack, that kind of stuff. Mostly the linker and loader will do things here, but again, it depends. And then there are libraries. Even if you use a static binary (which you probably would have to), modern OSes still expect you to dynamically link some libraries like all syscall entry points or stubs, things like libnss, etc. All in all quite impossible without at least another layer like wine or the java vm, some big loader switch/case and quite a bit of fiddling to adapt to some hard-to-adapt peculiarities. E.g. the Linux kernel has quite a few things built specially for wine to work.