3 ms·
Most of my work involves kernel programming (device drivers for hardware or virtualisation companies etc.); almost all of the rest of it is writing networking c
by pmjordan 11y ago
Most of my work involves kernel programming (device drivers for hardware or virtualisation companies etc.); almost all of the rest of it is writing networking code (custom protocol implementations). So my reasons for evaluating[1] Rust are that I think C and C++ are pretty awful languages for writing code that's as security and reliability critical as the code I touch on a day to day basis, while memory-managed languages aren't even on the menu for me - kernel hacking is a fairly unpopular niche, and I'm used to it being ignored in terms of programming language and library design. My opinions/statements below are primarily centred around this set of use cases:
* There is little practical difference between regular threads and green threads in an OS kernel. Most OSes can't handle it if you manually mess with the stack in a kernel context anyway. (thread records are typically accessed by rounding the current stack pointer) So if you want to do synchronous-style I/O with real stacks, you'll need an OS thread for each task.
* Kernel thread stacks are fairly small; 8-16KiB are typical. The reason for this is of course that kernel stacks must use wired (non-pageable) memory, or interrupts will irrecoverably page fault. 8-16KiB is much too small for buffers of course, but also enormous compared to the actual amount of non-buffer state for most I/O tasks. In any case, you never ever want to get too close to utilising the theoretical maximum as you risk crashing the system. So for every kernel thread you create, you know you're wasting precious wired memory. Obviously, this is true for threads created from userspace too, but in the kernel, people usually don't have a choice about running your code, so you try to be as good a citizen as possible, and not fire off hundreds of threads.
* In many contexts, dynamic memory allocations are not reliable, and allocation failure must not impede progress. (I.e. I can't wait for the system to page some memory out to disk if my code is on the critical path for disk I/O.) So typically, it is desirable to pre-allocate enough memory to keep all the state required for a sequence of operations from start to finish. It's even less likely you can just spin up a new thread; so the threaded approach implies keeping a pool of threads around which is guaranteed to be big enough. A.k.a. a waste of resources.
* The chain-of-callbacks approach to I/O is even more awful in languages and environments with manually managed memory and resources.
* If I'm going to pick a fancy new language to write my drivers in, I'm still going to have to use the custom alloc/free functions for each type of kernel object (network packet, etc.) that the OS I'm writing against happens to use.
So typically, you end up either splitting your code into a bunch of callbacks, or you create a complicated explicit state machine with a giant dispatch switch() statement. In both cases all state tracked in a giant struct. Keeping track of control flow is tricky. Maintaining invariants is tricky. Making sure you don't leak (or over-free, or use-after-free) any of the resources you touch is tricky.
It'd be really, really nice if the language could help you out with this. Write it as synchronous code, and the compiler turns each location where execution can be suspended into a callback function. The locals that are used across suspension points are stored in an automatically generated struct (bonus points: unions for state which is guaranteed to not have overlapping lifetimes) which can be preallocated before firing off the "task" in question. As far as I'm aware, such a code transformation would effectively be a CPS-transform, (continuation passing style) which has at least been researched quite a bit in theory, if not so much in non-GC-language practice.
I haven't been able to invest large amounts of time into really learning Rust and applying it to my use case in earnest. It's tough without a remotely compliant standard C library around, and I've struggled a bit with trying to pick and choose bits out of Rust's 'core' library without bringing on an avalanche of dependencies. (any kernel module code is kept in wired memory, so unused code is a waste of resources in the kernel) I'm determined to overcome that though and use Rust for something other than a toy kernel module, and see if and how it improves things over C. If Rust does start supporting some kind of advanced I/O pattern that works in that sort of constrained environment, that seems like a significant competitive advantage.
[1] https://github.com/pmj/rustykext https://github.com/pmj/rustykext
- acconsta 11y agoWhat do you think of C++ coroutines? Two competing standards proposals, already available as libraries: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4453.pdf http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n445... http://blogs.msdn.com/b/vcblog/archive/2014/11/12/resumable-functions-in-c.aspx http://blogs.msdn.com/b/vcblog/archive/2014/11/12/resumable-...
- pmjordan 11y agoThey certainly look useful at first glance. The first's library implementation is based on boost.coroutine which as far as I'm aware uses stack-switching, so this is more of a syntax demo than a demo of the final implementation details. The fact that you can apply sizeof() to the resumable type is important for the kernel-typical pool allocators. The second seems to have preview support in VS2015's compiler. Too bad I'm not much of a Windows programmer these days… It's not quite clear what ther memory-allocation story is from that article. Of course the downside is it's yet another graft onto the C++ language. After ~15 years of using it, my point of view is mainly that C++'s type system and unsafe-by-default behaviour inherited from C are a mistake…
- jpgvm 11y agoI think you should take a look at C#'s async/await infrastructure. I am also from a low-level C background and I think this is the approach I would like to see in Rust. It can be implemented in a way that would be safe to run in kernel space and overall provides a pretty nice programming model. It can also be very performant if it's integrated into the compiler.
- steveklabnik 11y agoWe have a ticket open: https://github.com/rust-lang/rfcs/issues/388 https://github.com/rust-lang/rfcs/issues/388