13 ms·
LibOS – A library operating system for Linux
- heinrich5991 12y agoDoes this mean we get to link the kernel as a shared library?
- geofft 12y agoIf you want that, there's always OSv: https://github.com/cloudius-systems/osv https://github.com/cloudius-systems/osv
- chubot 12y agoUh, not sure what's up with lkml.org -- I see some G+ +1 buttons but no actual text, in both Firefox and Chrome. Presumably the article is this: https://lwn.net/Articles/637658/ https://lwn.net/Articles/637658/ in which case it sounds awesome!
- sctb 12y agoThanks, we updated the link.
- hoodoof 12y agoHow might this relate to rump kernels and unikernels?
- amelius 12y agoOr microkernels.
- justincormack 12y agoThe design is quite heavily influenced by the rump kernel design, although Linux has its own issues that make it a bit different. Obviously its only the network stack.
- Rusky 12y agoThe other closest thing is probably an exokernel, which is essentially a rump kernel with all the of the stuff that's optionally in userspace removed from the kernel. MIT did a lot of experimentation there.
- huhtenberg 12y agoThis is excellent. Anyone who's ever faced the daunting task of bolting TCP on top of custom datagram protocol can certainly confirm how useful a library like this is.
- agwa 12y agoAs someone who's currently facing that daunting task myself, may I ask what you ended up doing? Did you have to roll your own reliable stream-oriented protocol?
- geofft 12y agoWhy isn't that "just" either using lwIP, or using your host's TCP running against a TUN/TAP device and then encapsulating the data you read from /dev/net/tun into your custom datagram protocol? (This is not meant to be dismissive, just inquisitive. I'm sure it can't be anywhere near as easy as what I said, I'd just like to learn why.)
- rpcope1 12y agoIt will be interesting to see how this evolves; hopefully it will make it easier to build a userland TCP/IP stack, and maybe even to make it easier to ship a new transport protocol (this might make it easier to push for HTTP over SCTP instead of TCP, which makes far more sense, really). Also this will be nice if it allows userland applications to utilize ICMP (traceroute/nmap would no longer be nerfed without root access). NUSE alone should be incredibly handy.
- justincormack 12y agoYou still need root access to connect to an outside network interface from the userland stack, eg a raw socket... Linux has unpriveleged icmp sockets but no one has ever used them apparently - IPPROTO_ICMP eg see [1] [1] http://stackoverflow.com/questions/14018584/python-with-unprivileged-ping-in-linux-ipproto-icmp http://stackoverflow.com/questions/14018584/python-with-unpr...
- anonbanker 12y agoLennart Poettering would do well to pay attention to people like this, continuing to honor the POSIX way.
- fuuuuuub4r 12y agoSorry but that comment is just BS.
- anonbanker 12y agowhy?
- agashka 12y agoI have to credit this blog for much of the material I've absorbed over the years, it's been fantastic how much I've learned here. I've always managed to learn something new from articles, all this to say that today is different... I really do not understand this library at all. Could someone explain me different possible uses for this library? What would be the uses of this out of the box?
- eps 12y agoYou have a very peculiar comment history - https://news.ycombinator.com/threads?id=agashka https://news.ycombinator.com/threads?id=agashka
- Hello71 12y ago1. Calling LWN a "blog" is vaguely like calling Ars or even the NYT a "blog" because they have textual online content in article format. 2. This isn't even really LWN content, it's a mailing list post. You can click "Archive-link" to see it on Gmane.
- agashka 12y agoalright got it, thanks everyone, i'll stay quiet the next time
- agumonkey 12y agoReminded me of that quote: "An operating system is a collection of things that don't fit into a language. There shouldn't be one." --DanIngalls
- keenerd 12y agoThere are environments that do not have a strong distinction between OS and language. While adored by their users, for some reason they are never very popular. Oberon, Smalltalk and the old Lisp Machines come to mind. (Maybe Forth too, though most "Forth OSes" never tried to do multitasking or even file systems.) I suspect there are anti-network effects holding them back. They support one language and one language only. You can't easily bolt them into whatever OS you already use. None of your familiar tools run on them either. Adoption rates are poor with no way to dip your toe into the environment. What would a "no OS" system that equally supports multiple languages look like? edit: Multiple languages that have nothing in common. Think about the insane variety of languages you have available on *nix.
- chongli 12y agoWhat would a "no OS" system that equally supports multiple languages look like? Racket?
- agumonkey 12y agoa CLR on metal ? I know all these names, unfortunately only by reading and not by experience. Some lispers are also keen on bypassing OS interface and emit binary code on the go.
- icebraining 12y agoMicrosoft has already put CIL on metal, using Bartok¹. They used that to build a whole OS, called Singularity². ¹http://en.wikipedia.org/wiki/Bartok_%28compiler%29 http://en.wikipedia.org/wiki/Bartok_%28compiler%29 ² http://en.wikipedia.org/wiki/Singularity_%28operating_system%29 http://en.wikipedia.org/wiki/Singularity_%28operating_system...
- 12y ago
- arunc 12y agoThe author forgot to add a license for this project. What shall we assume?
- teraflop 12y agoIt was submitted as a patch series to the Linux kernel mailing list, with proper "Signed-off-by" headers in the commit messages. So presumably it's usable under GPLv2. EDIT: I was referring to the kernel support, but it looks like you might have been talking about the code on Github which uses the LibOS API. That does indeed seem to be missing a license.
- mintplant 12y agoDo you mean https://github.com/libos-nuse/net-next-nuse https://github.com/libos-nuse/net-next-nuse? There's a COPYING file: https://github.com/libos-nuse/net-next-nuse/blob/nuse/COPYING https://github.com/libos-nuse/net-next-nuse/blob/nuse/COPYIN... (GPLv2, taken from the kernel)
- teraflop 12y agoI was talking about the repository linked from the original mailing list post: https://github.com/libos-nuse/linux-libos-tools https://github.com/libos-nuse/linux-libos-tools
- d4mi3n 12y agoIf you're in the US, lack of a license generally means standard copyright: http://choosealicense.com/no-license/ http://choosealicense.com/no-license/
- glandium 12y agoIt's a patch to apply to the linux kernel. I guess you can assume GPLv2.
- ufo 12y agoStupid question: why does the Linux Kernel handle the networking in the first place? Why not implement everything in userspace from the start?
- ori_b 12y agoFor a number of reasons: - It used to be impossible to implement securely. Until very recently, there was no hardware support for virtualizing the network buffers, which would mean emulating the network hardware in userspace. This would be very slow. - Because even today, many devices don't have the required hardware virtualization support. For example, many (most?) ARM devices. If you give direct DMA access, you might be allowing anyone to splatter whatever code they want across whatever memory they want. - Without kernel arbitration of some sort, there's no way to do load balancing across services, throttle, or firewall effectively. - The kernel is designed to provide a uniform interface for all programs to the hardware. Putting networking in userspace gets rid of this abstraction, and means that every program has to be aware of the network hardware; software that works with file descriptors directly can't use the same abstraction for files and network. - Because there are no hardware limitations on the amount of multiplexing. I don't remember how much muxing various hardware supports, but hardware has limits for this sort of thing. If you want more than N processes using the virtual network, you might be SOL with a virtualized userspace network stack.
- vidarh 12y agoThese are valid objections to in-process access to the networking hardware. Not to running the networking stack in user space, in a privileged process. For the latter, performance concerns would be a bigger issue.
- justincormack 12y agoTo multiplex multiple users and applications. Which is less necessary in these days of virtual network devices etc, eg SR-IOV can make a PCI device appear as say 64 network devices so every application can have its own.
- thehajime 12y ago
- suprjami 12y agoNow this is userspace networking I can get behind! Rather than re-implement a new network stack and solve all the same problems again, this builds on the years of good work already done and leverages the existing time investment within the existing stack. "Librarising" existing stacks to userspace, great idea, this is the way of the future.
- justincormack 12y agoThe NetBSD rump kernel (rumpkernel.org) does this for every driver in the kernel, and is part of the OS, so they are maintained (there are lots of stacks pulled out but not maintained). Thats why it is important that this project becomes part of upstream so it can be maintained as the stack changes.
- ausjke 12y agoI'm aware of Freescale's USDPAA that runs network stack in user space to achieve wire-speed.
- jrk_ 12y agoMy first thought was that this sounds like an idea taken from microkernels. Is this a correct analogy? Would it be sufficient to use libos and fuse to call the Linux kernel a microkernel? What would be missing?
- Millennium 12y agoMy first thouvht on hearing the name "libOS" was that someone had gotten Linux running on an exokernel, which is in some ways a variation on the microkernel theme. Linux on a microkernel has been done before. The MkLinux projext is one example, though it also doubled as a port to PowerPC. I haven't heard of any exokernel attempts, though.
- anttiok 12y agoFor one, the microkernel would be missing ;) More seriously though, you're absolutely right in that it's a step in the direction of an "optional" microkernel architecture. That's actually how rump kernels on NetBSD started: running the kernel file system driver as a library in userspace on top of a FUSE-like subsystem. It's pretty useful functionality, since it allows you to handle untrusted file system images safely in userspace, while not imposing performance penalty on the trusted images which can be handled by the same driver running in the kernel. Unlike with FUSE-specific drivers, you don't run into issues with unsymmetric driver support in userspace vs. kernel.