6 ms·

The Linux Kernel Module Programming Guide

386 points by asicsp 3y ago

totetsu 3y ago
I always liked the Hitchhikers Guide to Kernel module programming, with DONT PANIC on the cover.
packetlost 3y ago
This... is really helpful. I've been trying to contribute some drivers for Pine64 devices lately and am trying to get up to speed on Linux Kernel development. There's... a lot. Does anyone know if this is up to date with current Linux best practices?
- smashed 3y ago
  According to the github readme, the guide has been around since 2001 but it seems fairly up to date.. There are numerous discussion going on the issue tracker, about adding rust chapter, targeting linux 6.1, etc.. Very cool to see github issues and pull request actively used to keep a guide up to date. The guide will walk you through the basic of a kernel module. Exposing virtual files under /proc or /sys or messing around with system calls. It will fall short of helping you with the nitty gritty of writing an hardware driver though. The problem with writing drivers is that you are interfacing with a real-world machine for which you will have very little information about. Pick a device similar to the one you want to support, but that is already supported in linux. Go look at its source code. This will tell you far more about the work involved. You might realize that there is simply not enough information available to support the driver.
- megous 3y ago
  No, it's not uptodate. But it's probably good enough to learn about basic concepts. For the devices you're talking about, you'll need to learn about device trees (didn't exist in 2001 at all in Linux), power management, etc.
  - packetlost 3y ago
    I've been reading up on DTS stuff already, there's some resources out there, but all of the "kernel developer guides" seem to predate that by a fair bit. I suppose I'll have to familiarize myself with U-Boot as well...
    - megous 3y ago
      U-Boot has a sort of baby device model implementation. :) Simpler and perhaps easier to grok than Linux, but using the identical DT based HW representation Linux uses later on during boot (it may even modify DTB a bit in memory to pass some information to kernel). OTOH, depending on platform it may mix this device model with some ad-hoc/hardcoded knowledge about the platform and ifdef mess[1], that is otherwise properly described by DT in Linux. So it can be a mixed bag. [1] eg. https://xff.cz/git/u-boot/tree/board/sunxi/board.c https://xff.cz/git/u-boot/tree/board/sunxi/board.c
bvan 3y ago
Fantastic resource. Thanks for sharing.
asicsp 3y ago
Previous discussions: https://news.ycombinator.com/item?id=28036619 https://news.ycombinator.com/item?id=28036619 (154 points | Aug 2, 2021 | 30 comments) https://news.ycombinator.com/item?id=28283030 https://news.ycombinator.com/item?id=28283030 (339 points | Aug 23, 2021 | 42 comments) See also: https://0xax.gitbooks.io/linux-insides/content/ https://0xax.gitbooks.io/linux-insides/content/ (A book-in-progress about the linux kernel and its insides)
westurner 3y ago
Ctrl-F "rust" https://rust-for-linux.com/ https://rust-for-linux.com/ links to LWN articles at https://lwn.net/Kernel/Index/#Development_tools-Rust https://lwn.net/Kernel/Index/#Development_tools-Rust that suggest that only basic modules are yet possible with the rust support in Linux kernels 6.2 and 6.3. Rust-for-linux links to the Android binder module though: https://rust-for-linux.com/Android-Binder-Driver.html https://rust-for-linux.com/Android-Binder-Driver.html : > Android Binder Driver: This project is an effort to rewrite Android's Binder kernel driver in Rust. > Motivation: Binder is one of the most security and performance critical components of Android. Android isolates apps from each other and the system by assigning each app a unique user ID (UID). This is called "application sandboxing", and is a fundamental tenet of the Android Platform Security Model. > The majority of inter-process communication (IPC) on Android goes through Binder. Thus, memory unsafety vulnerabilities are especially critical when they happen in the Binder driver ... "Rust in the Linux kernel" (2021) https://security.googleblog.com/2021/04/rust-in-linux-kernel.html?m=1 https://security.googleblog.com/2021/04/rust-in-linux-kernel... : > [...] We also need designs that allow code in the two languages to interact with each other: we're particularly interested in safe, zero-cost abstractions that allow Rust code to use kernel functionality written in C, and how to implement functionality in idiomatic Rust that can be called seamlessly from the C portions of the kernel. > Since Rust is a new language for the kernel, we also have the opportunity to enforce best practices in terms of documentation and uniformity. For example, we have specific machine-checked requirements around the usage of unsafe code: for every unsafe function, the developer must document the requirements that need to be satisfied by callers to ensure that its usage is safe; additionally, for every call to unsafe functions (or usage of unsafe constructs like dereferencing a raw pointer), the developer must document the justification for why it is safe to do so. > We'll now show how such a driver would be implemented in Rust, contrasting it with a C implementation. [...] Is this the source for the rust port of the Android binder kernel module?: https://android.googlesource.com/platform/frameworks/native/+/master/libs/binder/rust/src https://android.googlesource.com/platform/frameworks/native/... This guide with unsafe rust that calls into the C, and then with next gen much safer rust right next to it would be a helpful resource too. What of the post-docker container support (with userspaces also written in go) should be cloned to rust first? What are some good examples of non-trivial Linux kernel modules written in Rust?
pushedx 3y ago
Many things that you may have wanted to do with a kernel module can now be done more easily and safely with eBPF. https://ebpf.io/get-started/ https://ebpf.io/get-started/ Nevertheless, the OP is a classic guide and worth a read.
- tetris11 3y ago
  That's pretty tidy. Is there any noticeable overhead in listening to all kernel sockets and hooking in your code when it matches a packet filter, compared to just building it into the kernel?
  - bjackman 3y ago
    It's pretty damm fast. Everything is JITed and there's codegen for the "trampoline" logic so that there aren't even indirect branches to get to the program code. The "overhead" comes from the fact that writing nontrivial BPF programs is quite hard. Getting programs to pass the verifier can become a bit of an art. This does tend to get easier if your production environment has few and recent kernel versions. And then of course this "overhead" totally pays for itself in terms of reliability. It's really hard to accidentally break machines with BPF, while a kernel module in your prod environment is a time bomb :D (Deliberately breaking machines with BPF is very easy though. This sometimes seems to surprise people but if your attacker gets CAP_BPF it's probably game over).
- pcthrowaway 3y ago
  > Nevertheless, the OP is a classic guide and worth a read. For anyone curious why a book published in 2023 is being called a "classic", it appears to be a republication (possible update) of https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf from 2001 There are 2 new authors listed so I'm assuming the content has been refreshed
  - laweijfmvo 3y ago
    I've probably written or dabbled with kernel modules three times in my life: first in college (~2010), second for a take-home interview question (~2015), and third as part of my job duties (~2020), and every time I do so I find that the APIs/best practices have changed. So I really hope there's a definitive guide that stays up to date until at least ~2025 when I next need it.
joanne123 3y ago
[dead]
uraura 3y ago
This one is also good. https://linux-kernel-labs.github.io/refs/heads/master/so2/index.html https://linux-kernel-labs.github.io/refs/heads/master/so2/in...
ejolto 3y ago
Neat! will this finally replace the outdated Linux Device Drivers Third Edition as the de-facto standard reference book for building kernel modules? https://lwn.net/Kernel/LDD3/ https://lwn.net/Kernel/LDD3/
lavventura 3y ago
This is really helpful. Beyond that its open LaTeX format can be used for documentation, which I will do for my own work.
vrglvrglvrgl 3y ago
[dead]

386 points by asicsp 3y ago

I always liked the Hitchhikers Guide to Kernel module programming, with DONT PANIC on the cover.

This... is really helpful. I've been trying to contribute some drivers for Pine64 devices lately and am trying to get up to speed on Linux Kernel development. There's... a lot. Does anyone know if this is up to date with current Linux best practices?

smashed 3y ago
According to the github readme, the guide has been around since 2001 but it seems fairly up to date.. There are numerous discussion going on the issue tracker, about adding rust chapter, targeting linux 6.1, etc.. Very cool to see github issues and pull request actively used to keep a guide up to date. The guide will walk you through the basic of a kernel module. Exposing virtual files under /proc or /sys or messing around with system calls. It will fall short of helping you with the nitty gritty of writing an hardware driver though. The problem with writing drivers is that you are interfacing with a real-world machine for which you will have very little information about. Pick a device similar to the one you want to support, but that is already supported in linux. Go look at its source code. This will tell you far more about the work involved. You might realize that there is simply not enough information available to support the driver.
megous 3y ago
No, it's not uptodate. But it's probably good enough to learn about basic concepts. For the devices you're talking about, you'll need to learn about device trees (didn't exist in 2001 at all in Linux), power management, etc.
- packetlost 3y ago
  I've been reading up on DTS stuff already, there's some resources out there, but all of the "kernel developer guides" seem to predate that by a fair bit. I suppose I'll have to familiarize myself with U-Boot as well...
  - megous 3y ago
    U-Boot has a sort of baby device model implementation. :) Simpler and perhaps easier to grok than Linux, but using the identical DT based HW representation Linux uses later on during boot (it may even modify DTB a bit in memory to pass some information to kernel). OTOH, depending on platform it may mix this device model with some ad-hoc/hardcoded knowledge about the platform and ifdef mess[1], that is otherwise properly described by DT in Linux. So it can be a mixed bag. [1] eg. https://xff.cz/git/u-boot/tree/board/sunxi/board.c https://xff.cz/git/u-boot/tree/board/sunxi/board.c

bvan 3y ago

Fantastic resource. Thanks for sharing.

asicsp 3y ago

Previous discussions: https://news.ycombinator.com/item?id=28036619 https://news.ycombinator.com/item?id=28036619 (154 points | Aug 2, 2021 | 30 comments) https://news.ycombinator.com/item?id=28283030 https://news.ycombinator.com/item?id=28283030 (339 points | Aug 23, 2021 | 42 comments) See also: https://0xax.gitbooks.io/linux-insides/content/ https://0xax.gitbooks.io/linux-insides/content/ (A book-in-progress about the linux kernel and its insides)

westurner 3y ago

Ctrl-F "rust" https://rust-for-linux.com/ https://rust-for-linux.com/ links to LWN articles at https://lwn.net/Kernel/Index/#Development_tools-Rust https://lwn.net/Kernel/Index/#Development_tools-Rust that suggest that only basic modules are yet possible with the rust support in Linux kernels 6.2 and 6.3. Rust-for-linux links to the Android binder module though: https://rust-for-linux.com/Android-Binder-Driver.html https://rust-for-linux.com/Android-Binder-Driver.html : > Android Binder Driver: This project is an effort to rewrite Android's Binder kernel driver in Rust. > Motivation: Binder is one of the most security and performance critical components of Android. Android isolates apps from each other and the system by assigning each app a unique user ID (UID). This is called "application sandboxing", and is a fundamental tenet of the Android Platform Security Model. > The majority of inter-process communication (IPC) on Android goes through Binder. Thus, memory unsafety vulnerabilities are especially critical when they happen in the Binder driver ... "Rust in the Linux kernel" (2021) https://security.googleblog.com/2021/04/rust-in-linux-kernel.html?m=1 https://security.googleblog.com/2021/04/rust-in-linux-kernel... : > [...] We also need designs that allow code in the two languages to interact with each other: we're particularly interested in safe, zero-cost abstractions that allow Rust code to use kernel functionality written in C, and how to implement functionality in idiomatic Rust that can be called seamlessly from the C portions of the kernel. > Since Rust is a new language for the kernel, we also have the opportunity to enforce best practices in terms of documentation and uniformity. For example, we have specific machine-checked requirements around the usage of unsafe code: for every unsafe function, the developer must document the requirements that need to be satisfied by callers to ensure that its usage is safe; additionally, for every call to unsafe functions (or usage of unsafe constructs like dereferencing a raw pointer), the developer must document the justification for why it is safe to do so. > We'll now show how such a driver would be implemented in Rust, contrasting it with a C implementation. [...] Is this the source for the rust port of the Android binder kernel module?: https://android.googlesource.com/platform/frameworks/native/+/master/libs/binder/rust/src https://android.googlesource.com/platform/frameworks/native/... This guide with unsafe rust that calls into the C, and then with next gen much safer rust right next to it would be a helpful resource too. What of the post-docker container support (with userspaces also written in go) should be cloned to rust first? What are some good examples of non-trivial Linux kernel modules written in Rust?

pushedx 3y ago

Many things that you may have wanted to do with a kernel module can now be done more easily and safely with eBPF. https://ebpf.io/get-started/ https://ebpf.io/get-started/ Nevertheless, the OP is a classic guide and worth a read.

tetris11 3y ago
That's pretty tidy. Is there any noticeable overhead in listening to all kernel sockets and hooking in your code when it matches a packet filter, compared to just building it into the kernel?
- bjackman 3y ago
  It's pretty damm fast. Everything is JITed and there's codegen for the "trampoline" logic so that there aren't even indirect branches to get to the program code. The "overhead" comes from the fact that writing nontrivial BPF programs is quite hard. Getting programs to pass the verifier can become a bit of an art. This does tend to get easier if your production environment has few and recent kernel versions. And then of course this "overhead" totally pays for itself in terms of reliability. It's really hard to accidentally break machines with BPF, while a kernel module in your prod environment is a time bomb :D (Deliberately breaking machines with BPF is very easy though. This sometimes seems to surprise people but if your attacker gets CAP_BPF it's probably game over).
pcthrowaway 3y ago
> Nevertheless, the OP is a classic guide and worth a read. For anyone curious why a book published in 2023 is being called a "classic", it appears to be a republication (possible update) of https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf from 2001 There are 2 new authors listed so I'm assuming the content has been refreshed
- laweijfmvo 3y ago
  I've probably written or dabbled with kernel modules three times in my life: first in college (~2010), second for a take-home interview question (~2015), and third as part of my job duties (~2020), and every time I do so I find that the APIs/best practices have changed. So I really hope there's a definitive guide that stays up to date until at least ~2025 when I next need it.

joanne123 3y ago

[dead]

uraura 3y ago

This one is also good. https://linux-kernel-labs.github.io/refs/heads/master/so2/index.html https://linux-kernel-labs.github.io/refs/heads/master/so2/in...

ejolto 3y ago

Neat! will this finally replace the outdated Linux Device Drivers Third Edition as the de-facto standard reference book for building kernel modules? https://lwn.net/Kernel/LDD3/ https://lwn.net/Kernel/LDD3/

lavventura 3y ago

This is really helpful. Beyond that its open LaTeX format can be used for documentation, which I will do for my own work.

vrglvrglvrgl 3y ago

[dead]