9 ms·
OSv – Linux binary compatible unikernel for virtualized environments
- peter_d_sherman 6y ago>"OSv has been designed to run unmodified x86-64 Linux binaries as is , which effectively makes it a Linux binary compatible unikernel (for more details about Linux ABI compatibility please read this doc). In particular OSv can run many managed language runtimes including JVM, Python 2 and 3, Node.JS, Ruby, Erlang, and applications built on top of those runtimes. It can also run applications written in languages compiling directly to native machine code like C, C++, Golang and Rust as well as native images produced by GraalVM and WebAssembly/Wasmer. OSv can boot as fast as ~5 ms on Firecracker using as low as 15 MB of memory. OSv can run on many hypervisors including QEMU/KVM, Firecracker, Xen, VMWare, VirtualBox and Hyperkit as well as open clouds like AWS EC2, GCE and OpenStack." Related: http://seastar.io/futures-promises http://seastar.io/futures-promises >"Paradigms for parallellization Solutions for coordinating work across multiple cores are many. Some are highly programmer-friendly and enable development of software that works exactly if it were running on a single core. For example the classic Unix process model is designed to keep each process in total isolation and relies on kernel code to maintain a separate virtual memory space per process. Unfortunately this increases the overhead at the OS level. Software development challenges [...] Hardware has changed to the point where the assumptions originally made on small numbers of CPU cores are no longer valid. Processes are extremely self-contained but have high overhead. Threads impose additional coordination costs on both the programmer and the application infrastructure, and are notoriously difficult to debug. Pure event-driven programming can result in codebases that are difficult to test and extend."
- ttul 6y agoEffectively, event-driven programming puts the task of sorting out process scheduling into the hands of the developer, rather than the OS, which would otherwise have to guess at the best scheduling approach. Yes, there are also speed advantages from losing the context switch overhead, but fundamentally, I believe it is the programmer-controlled scheduling that makes event driven apps faster, all other things being held equal.
- varbhat 6y agoNice Project. But, I think that Linux kernel is more Battle Tested than this. Also, Compiled Linux kernel can weigh 6.7MB or even less. What i am thinking is that we can compile linux kernel , drivers needed for the particular device, compile runtime and application and deploy on bare machine. It will be monolithic approach too. Further i think that unikernel applications are more helpful if application take features of underlying kernel and vice versa . I also think that unikernel approach is not much helpful if that unikernel acts like shim layer to run other applications. Briefly,i am saying that unikernel applications must be built from bottom unikernel to application all integrated. So,how is this unikernel better than Linux based monolithic deployment that i mentioned above? what about security updates in this unikernel ?
- rwmj 6y ago> What i am thinking is that we can compile linux kernel , drivers needed for the particular device, compile runtime and application and deploy on bare machine. It will be monolithic approach too. That's what UKL is doing: https://github.com/unikernelLinux/ https://github.com/unikernelLinux/
- heavyset_go 6y ago> But, I think that Linux kernel is more Battle Tested than this. Also, Compiled Linux kernel can weigh 6.7MB or even less. It can weigh much less.
- q3k 6y agoInteresting. Would like to see some compatibility matrix against gVisor/runsc. I know they're not the same thing, but they attempt to solve the same goal (lightweight VMM-based isolation providing a Linux ABI). Is there a summary list of supported syscalls somewhere?
- eyberg 6y agogVisor is pretty bad perf-wise: https://news.ycombinator.com/item?id=19924036 https://news.ycombinator.com/item?id=19924036 https://gvisor.dev/docs/architecture_guide/performance/ https://gvisor.dev/docs/architecture_guide/performance/ However, it does directly deal with container security issues. If you absolutely must use containers you probably should be using something like that as containers inherently weaken security architecturally.
- angry_octet 6y agoYou've got it backwards. Containers strenghen security. Container mgmt runtimes executing as root causes security problems, like the setuid programs of old.
- eyberg 6y agoYeh, I beg to differ. Containers have an absolutely horrible security record. Just in the past few weeks we've seen yet another container breakout, an "un-fixable" k8s 'issue', another cryptojacker exclusively attacking container infrastructure and more. The huge problem with "containers" is that they break well known security boundaries by sharing the same kernel. This is further exacerbated by software like k8s by allowing an attacker to further extend their attack as soon as they compromise a single instance.
- JoachimSchipper 6y agoI've been a little out of the loop - do you happen to have references?
- dis-sys 6y agoInteresting project. Just wondering is there any ongoing effort to somehow allow users to build a minimum linux kernel that is just enough to support a specified application?
- orumin 6y agoThere is project what you say, that's name "Lupine". paper: https://dl.acm.org/doi/abs/10.1145/3342195.3387526 https://dl.acm.org/doi/abs/10.1145/3342195.3387526 code: https://github.com/hckuo/Lupine-Linux https://github.com/hckuo/Lupine-Linux
- dis-sys 6y agovery cool stuff, thanks for sharing
- rwmj 6y agohttps://github.com/unikernelLinux https://github.com/unikernelLinux
- dis-sys 6y agovery interesting stuff, thanks for sharing
- IfOnlyYouKnew 6y agoI did once spend a few boring hours on a plane on a genetic-algorithm-based optimiser that (tried to) minimise a Linux image (I believe I used Gentoo), eliminating packages first, followed by files, segments, and individual lines of code. From what I remember, (a) you can probably get to an extremely low size, (b) it won't be worth it compared to similar efforts taking a different route and getting 90 % there, because (c) your test will never be sufficient for any real-world usage: I remember going through some of the codepaths that had been disabled, and it included handling of specific dates, any unicode character I had not used, handling of networking and disk access failures that I did not even know of, etc.
- tutfbhuf 6y agoIt sounds like a cool idea, but I'm not convinced yet. Could you tell me more about the advantages that OSv has to offer over a stripped down Linux?
- sanxiyn 6y agoNo system call overhead. System calls are function calls in OSv.
- trasz 6y agoHow do they do fork()?
- ObscureScience 6y agoThey don't have processes. Only threads.
- erk__ 6y agoI wonder how it would compare to something like DragonFlyBSD then
- __s 6y agohttp://osv.io/benchmarks http://osv.io/benchmarks
- tutfbhuf 6y agoAny particular reason why they benchmark against Linux 3.13 a 7 years old kernel?
- coder543 6y agoUnsurprisingly, they benchmarked against the latest that was available at the time. It's not some nefarious trick to make the benchmarks look good. We can see that the heyday of OSv development work was about 7 years ago: https://github.com/cloudius-systems/osv/graphs/contributors https://github.com/cloudius-systems/osv/graphs/contributors Hopefully, OSv is a stable product that continues to work well, so heavy investment in development hasn't been needed on an ongoing basis, primarily just maintenance work. Or, it could be abandonware... but the README gives the impression that it is still being maintained. I would still be hesitant to deploy such a deeply invasive solution, but the idea is interesting.
- deleted 6y ago[deleted]
- caspper69 6y agoI love it. We spent 30 years getting preemptive-multitasking working, and it is such a giant fragile pain in the a, that it is easier to just virtualize the whole damn computer rather than making different software play nice together (maybe run well together is more apropos). We have now come full circle. People want DOS. I have officially lived "long enough." Not too long, mind you, just long enough.
- jonjacky 6y agoit is easier to just virtualize the whole damn computer IBM did this starting in the 1960s as a way of providing time-sharing. It became their VM/CMS OS product line (VM - hypervisor, CMS - single-user guest OS - but any OS could be the guest, including VM itself) CMS influenced the CP/M microcomputer operating system. So yes, it was not unlike DOS. https://en.wikipedia.org/wiki/VM_(operating_system) https://en.wikipedia.org/wiki/VM_(operating_system)
- rwmj 6y agoI've never thought that preemptive multitasking is "such a giant fragile pain in the a". I think it's pretty useful. What has been the problem with it? We could probably do with partitioning the desktop though, but that's easily done with a bit more cgroups.
- caspper69 6y agoThe biggest problem is that context switching overhead becomes the main "work" being done, rather than the actual work of the threads, for some core count n (which depends on workload). You can already saturate an entire core just copying data from a NIC. Of course, enter BPF and all that jazz. Plus it doesn't play as nice with BIG.little multiprocessing (AMP - asymmetric multiprocessing). edit: I should note I am not advocating getting rid of preemption entirely, because it is needed for at least one processor, but having a fixed quantum causing a context switch on application CPUs is highly disruptive, both from the operation itself, and from blowing the cache all the time. Incidentally, one of the main reason for green threads.
- jandrewrogers 6y ago
- secondcoming 6y agoLooks really cool and has quite similar goals to IncludeOS[0]. Stripping the hell out of images can only be a good thing. [0] https://www.includeos.org/ https://www.includeos.org/
- nderjung 6y agoBe sure to also check out Unikraft, a Unikernel SDK [0]! [0]: https://github.com/unikraft/unikraft https://github.com/unikraft/unikraft
- deleted 6y ago[deleted]
- umvi 6y agoSo is this similar to what AWS is using to power Lambda? I'm trying to understand the use case. Is this something that would be used in an IoT device?
- detaro 6y agoAFAIK Lambda runs optimized Linux kernels in VMs (with very limited virtual hardware etc for speed and security).
- orumin 6y agoAmazon optimized VMM and Linux kernel both. https://www.usenix.org/conference/nsdi20/presentation/agache https://www.usenix.org/conference/nsdi20/presentation/agache
- tutfbhuf 6y agoCan it run Docker?