9 ms·

Meet LKL: Turn Linux Kernel into a Library

102 points by yarapavan 11y ago

cm3 11y ago
So basically this is Linux's version of NetBSD's Rump, right?
- justincormack 11y ago
  Yes, thats right.
- anttiok 11y ago
  If it's not an instance of a selection of components running on top of externally scheduled threads, it's not a rump kernel in my book. So, strictly speaking no, but the answer really depends on what you want to convey by "basically". Many of the use cases should fit both projects.
  - cm3 11y ago
    I don't know enough about rump to talk more definitively, and in fact I've failed to find an overview about what it is technically, how it works on NetBSD, how it works outside NetBSD, etc. Does running rump on Linux provide the SMP scalability improvements one misses with NetBSD's kernel for instance? Can you point me at something to read that's short enough to digest in an hour?
    - anttiok 11y ago
      Follow the "Getting Started" link on rumpkernel.org. That page includes both short and long reading (should you get more interested ;), and also some "hands on" tutorials should you learn about things better that way (I know I do). SMP scalability is all about bottlenecks. If the driver is the bottleneck, it won't scale no matter where you run it.
  - thehajime 11y ago
    I've been working (though it's too slow) on making answer to this question "yes". https://github.com/thehajime/blog/issues/1 https://github.com/thehajime/blog/issues/1
rwmj 11y ago
I added experimental support for LKL to libguestfs[1]. It's not upstream in libguestfs because LKL isn't upstream in the kernel, but I'm rooting for it because it'll be nice to have a rump kernel based on Linux. It makes it a much easier sell for all the companies currently built around the "Linux ecosystem". One problem is it appears to require that programs are rewritten, replacing all system calls with lkl_sys_* variants. For some programs this is necessary, because you want to differentiate between (eg) lkl_sys_open (open a file through LKL) and regular open (open a file on the host), but it's also a pain for any significant program. [1] https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-backend-for-libguestfs/ https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-b...
- ambrice 11y ago
  It might be useful to make an LD_PRELOAD library that does the translation.
  - dohqu8Zi 11y ago
    Only in corner cases you want to replace all syscalls to LKL calls. Often you need to call to the host as well. Especially LKL itself needs...
  - thehajime 11y ago
    lkl (and rump hijack library) already do the job, though it's not perfect (at least in lkl right now).
bmuk 11y ago
Does this still incur the penalty of context switching from user space to kernel space? Filesystems are an interest of mine, and this would be a nice alternative to FUSE if it could mitigate the context-switching overhead.
- i4k 11y ago
  I have the same question.
- rwmj 11y ago
  If you use it to write a FUSE filesystem, then yes, because it's using FUSE. If you link (-llkl) directly to your program, then no, since it's all in the same process. But this is not exposing the POSIX API or a host filesystem, so that's possibly not useful, depending on what you're trying to do. I compare lkl to libguestfs in my article here: https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-backend-for-libguestfs/ https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-b...
  - bmuk 11y ago
    Ah okay, I think I understand more now. Thanks for the link, I'll check it out! Basically this is a kernel running in userspace, not hooking into the actual kernel running in kernel space?
    - rwmj 11y ago
      Yes, it's the Linux kernel, compiled as a (userspace) library, and linked to applications. The applications use lkl_sys_* functions to call "system calls" in this code-running-as-a-library.
- dohqu8Zi 11y ago
  This does not make any sense. As soon you issue any system call you will trigger a context switch. So, eve without using FUSE and just using LKL as tool to extract your favorite filesystem it will face them. But the context switches not the bottleneck in FUSE. Did you benchmark?
uxcn 11y ago
Since software will be linking directly against LKL, does this mean it also needs to be GPL? Will the same symbol restrictions apply?
ycmbntrthrwaway 11y ago
So, compared to LibOS, it does not emulate timers etc., but provides host system timers, right? There is no way it will replace LibOS in NS-3 DCE [1] and LibOS will continue to be developed out-of-tree, or is it possible that LKL will be ported to NS-3 as an underlying platform just like it supports POSIX and Windows now? [1] https://www.nsnam.org/overview/projects/direct-code-execution/ https://www.nsnam.org/overview/projects/direct-code-executio...
- thehajime 11y ago
  i hope to port libos code to lkl.
signa11 11y ago
couple of questions 1. is the entirety of kernel-networking now available to applications e.g. applications using tcp/udp/sctp/ipsec etc. etc. stacks completely in userland, just need to link with liblkl, and they are all set ? 2. also with dpdk etc. quite a large number of vendors are offering implementation of core networking protocols from l2 all the way up to l7. would this make all of those obsolete ?
- thehajime 11y ago
  1. yes. linking with liblkl or ld_preload. but network stack is not all about open(2)/close(2): bunch of configuration tools are needed to do the job, which is what we want to have in a future. 2. yes and no.
stuaxo 11y ago
This would be great for read/write support of linux filesystems.

102 points by yarapavan 11y ago

cm3 11y ago

So basically this is Linux's version of NetBSD's Rump, right?

justincormack 11y ago
Yes, thats right.
anttiok 11y ago
If it's not an instance of a selection of components running on top of externally scheduled threads, it's not a rump kernel in my book. So, strictly speaking no, but the answer really depends on what you want to convey by "basically". Many of the use cases should fit both projects.
- cm3 11y ago
  I don't know enough about rump to talk more definitively, and in fact I've failed to find an overview about what it is technically, how it works on NetBSD, how it works outside NetBSD, etc. Does running rump on Linux provide the SMP scalability improvements one misses with NetBSD's kernel for instance? Can you point me at something to read that's short enough to digest in an hour?
  - anttiok 11y ago
    Follow the "Getting Started" link on rumpkernel.org. That page includes both short and long reading (should you get more interested ;), and also some "hands on" tutorials should you learn about things better that way (I know I do). SMP scalability is all about bottlenecks. If the driver is the bottleneck, it won't scale no matter where you run it.
- thehajime 11y ago
  I've been working (though it's too slow) on making answer to this question "yes". https://github.com/thehajime/blog/issues/1 https://github.com/thehajime/blog/issues/1

rwmj 11y ago

I added experimental support for LKL to libguestfs[1]. It's not upstream in libguestfs because LKL isn't upstream in the kernel, but I'm rooting for it because it'll be nice to have a rump kernel based on Linux. It makes it a much easier sell for all the companies currently built around the "Linux ecosystem". One problem is it appears to require that programs are rewritten, replacing all system calls with lkl_sys_* variants. For some programs this is necessary, because you want to differentiate between (eg) lkl_sys_open (open a file through LKL) and regular open (open a file on the host), but it's also a pain for any significant program. [1] https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-backend-for-libguestfs/ https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-b...

ambrice 11y ago
It might be useful to make an LD_PRELOAD library that does the translation.
- dohqu8Zi 11y ago
  Only in corner cases you want to replace all syscalls to LKL calls. Often you need to call to the host as well. Especially LKL itself needs...
- thehajime 11y ago
  lkl (and rump hijack library) already do the job, though it's not perfect (at least in lkl right now).

bmuk 11y ago

Does this still incur the penalty of context switching from user space to kernel space? Filesystems are an interest of mine, and this would be a nice alternative to FUSE if it could mitigate the context-switching overhead.

i4k 11y ago
I have the same question.
rwmj 11y ago
If you use it to write a FUSE filesystem, then yes, because it's using FUSE. If you link (-llkl) directly to your program, then no, since it's all in the same process. But this is not exposing the POSIX API or a host filesystem, so that's possibly not useful, depending on what you're trying to do. I compare lkl to libguestfs in my article here: https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-backend-for-libguestfs/ https://rwmj.wordpress.com/2015/11/07/linux-kernel-library-b...
- bmuk 11y ago
  Ah okay, I think I understand more now. Thanks for the link, I'll check it out! Basically this is a kernel running in userspace, not hooking into the actual kernel running in kernel space?
  - rwmj 11y ago
    Yes, it's the Linux kernel, compiled as a (userspace) library, and linked to applications. The applications use lkl_sys_* functions to call "system calls" in this code-running-as-a-library.
dohqu8Zi 11y ago
This does not make any sense. As soon you issue any system call you will trigger a context switch. So, eve without using FUSE and just using LKL as tool to extract your favorite filesystem it will face them. But the context switches not the bottleneck in FUSE. Did you benchmark?

uxcn 11y ago

Since software will be linking directly against LKL, does this mean it also needs to be GPL? Will the same symbol restrictions apply?

ycmbntrthrwaway 11y ago

So, compared to LibOS, it does not emulate timers etc., but provides host system timers, right? There is no way it will replace LibOS in NS-3 DCE [1] and LibOS will continue to be developed out-of-tree, or is it possible that LKL will be ported to NS-3 as an underlying platform just like it supports POSIX and Windows now? [1] https://www.nsnam.org/overview/projects/direct-code-execution/ https://www.nsnam.org/overview/projects/direct-code-executio...

thehajime 11y ago
i hope to port libos code to lkl.

signa11 11y ago

couple of questions 1. is the entirety of kernel-networking now available to applications e.g. applications using tcp/udp/sctp/ipsec etc. etc. stacks completely in userland, just need to link with liblkl, and they are all set ? 2. also with dpdk etc. quite a large number of vendors are offering implementation of core networking protocols from l2 all the way up to l7. would this make all of those obsolete ?

thehajime 11y ago
1. yes. linking with liblkl or ld_preload. but network stack is not all about open(2)/close(2): bunch of configuration tools are needed to do the job, which is what we want to have in a future. 2. yes and no.

stuaxo 11y ago

This would be great for read/write support of linux filesystems.