26 ms·
A pair of Linux kernel modules using Rust
- motiejus 4y agoFolks, there is a good reason this is a subscriber-only content. It takes time and effort (and money!) to keep us up to date. For such a long time. This is the 5'th SubscriberLink over the last week on HN: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=lwn.net%2FSubscriberLink&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... Please, folks, subscribe, and let Corbet put the subscriber links here himself. The LWN weekly editions are definitely worth every penny. Edit: I am paying LWN since 2012 and intend to keep paying for as long as there is LWN.
- smcl 4y agoIs "SubscriberLink" a way to share paywalled LWN stuff with non-subscribers? Damn, I had no idea. I just presumed they un-paywalled stuff every now and again (like after N days or so).
- gnastygnorc 4y agoI believe all paywalled content becomes free to read two weeks after initial publication
- smcl 4y agoAh ok. I should still probably sub, I tend to read LWN when it pops up here and never really gave it a moments thought before :O
- _wolfie_ 4y agoIn my humble opinion it is definitely worth it. All people over at LWN do tremendous job and should really be supported.
- steveklabnik 4y agoI haven’t even used Linux for a few years now. Still pay for LWN. It’s so worth supporting.
- sophacles 4y agoI can honestly say the resources at LWN have helped my career immensely. The in depth series on topics like namespaces and io_uring directly allowed me to tackle some hard projects - without those articles I would have steered clear of "magic i don't understand" and the relevant projects would have suffered. It's definitely worth subscribing if you have the means (and the low cost of subscriptions suggests a lot of readers here have the means).
- couchand 4y agoI'd agree that often with new features and subsytems recent LWN articles are the best documentation around.
- wufocaculura 4y agoHighly recommend. Just to show support for corbet and his team. He could keep the articles paywalled forever, but is making them free after a week. Becoming a paying member is our way to say thank you for his fair attitude (and keeping ads at minimum level). I work at totaly different field in IT, most articles are just "wow, that's interesting" for me and - I still keep LWN's subscriptionm and it is the only one I have.
- thejammahimself 4y agoIndeed. They even provide a mailing list you can subscribe to which will notify you whenever an article goes free. I think you need to have an account with them firstly though.
- jwilk 4y agoIt's (usually?) less than that. https://lwn.net/op/FAQ.lwn https://lwn.net/op/FAQ.lwn > Must I subscribe to read the Weekly Edition? > No, the Weekly Edition becomes freely available to all readers one week after its publication.
- deleted 4y ago[deleted]
- voxadam 4y ago> I just presumed they un-paywalled stuff every now and again (like after N days or so). All of LWN's paywalled articles become free for everyone to view after a week. During the week that articles are paywalled any subscriber can share currently paywalled content by creating a SubscriberLink. LWN produces some of the best highly technical Linux and Linux adjacent content on the web.
- jbirer 4y agoHacker News is not the place to post paywalled content, whether you believe the reasons are valid or not.
- Aissen 4y agoRules are a bit more subtle: https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html (see: "Are paywalls ok?")
- wartijn_ 4y agoIt is according to the faq[0] --- Are paywalls ok? It's ok to post stories from sites with paywalls that have workarounds. In comments, it's ok to ask how to read an article and to help other users do so. But please don't post complaints about paywalls. Those are off topic. More here. --- [0] https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html
- o_1 4y agoi really hope substack takes off, the strategy behind pay content is much more responsible than javascript "walls"
- cycomanic 4y agoIt should be pointed out that Corbet himself has come out in support of subscriber links posted here and at other places. That's why they provided that capability. That said it shouldn't be abused and I suspect if it ends up becoming too much they might remove that ability. I agree with one of the other posters, we should support quality journalism, and lwn definitely is.
- Bancakes 4y agoIf LWN disappears tomorrow, I'll get my news elsewhere. Thank you for your concerns.
- sophacles 4y agoWhere else do you get in depth kernel reporting?
- penguin_booze 4y agoI recommend browsing their index pages, and pause and ponder where else you'd find such high quality, curated, approachable content, in such concentration. You could suggest the kernel's own documentation, but that's more developer-facing.
- jabiko 4y ago> This is the 5'th SubscriberLink over the last week on HN Its the first post to gain any meaningful traction > Please, folks, subscribe, and let Corbet put the subscriber links here himself. The LWN weekly editions are definitely worth every penny. According to corbet its totally fine if every now and then a subscriber link pops up on HN: https://news.ycombinator.com/item?id=1966033 https://news.ycombinator.com/item?id=1966033 > As long as the posting of subscriber links in places like this is occasional, I believe it serves as good marketing for LWN - indeed, every now and then, I even do it myself. We just hope that people realize that we run nine feature articles every week, all of which are instantly accessible to LWN subscribers. The LWN FAQ also considers it no problem: https://lwn.net/op/FAQ.lwn#slinks https://lwn.net/op/FAQ.lwn#slinks > Where is it appropriate to post a subscriber link? > Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared. I think posting this on HN does not defeat the purpose of gaining subscribers, but supports them. At least for me it worked. After reading some LWN articles on HN I decided to become a paying subscriber. Telling people they are not allowed to post such links here actually IMHO hurts LWN.
- capitol_ 4y agoBut they didn't tell them that its not allowed, they just encouraged people to subscribe to this excellent publication.
- yencabulator 4y ago"and let Corbet put the subscriber links here himself"
- jwilk 4y agoAnother, more recent statement from corbet: https://news.ycombinator.com/item?id=31852477 https://news.ycombinator.com/item?id=31852477
- mccorrinall 4y agoI only found out about LEB via links shared on HN. I haven’t subscribed yet but will consider it once I landed a job. I don’t think that sharing their most interesting articles is hurting them, more like the opposite.
- mhss 4y agoThanks for the reminder. I was a subscriber for a few years and then I stopped when I hit a long time crunch with no time to read much news. I just subscribed now that I feel I have some leisure time again for this and HN in general.
- yarg 4y agoIf they wanted to keep it subscriber only, they could achieve that by requiring users to be logged in. It's clear that the intention is to allow for sharing (although there is the question of "at what scale?"). Then there's the page itself: > Welcome to LWN.net > The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net! The people who run the site seem to have less of a problem with sharing its content than you do.
- lofaszvanitt 4y agoGrsec in the kernel eta?
- rs_rs_rs_rs_rs 4y agoI can only see that happening if spender goes insane.
- JoshTriplett 4y ago*goes sane More seriously: many different security improvements are filtering into the kernel idea-by-idea, insofar as folks working on kernel security do the actual work of making them fit in with the kernel, and/or coming up with better alternatives.
- geofft 4y agohttps://kernsec.org/wiki/index.php/Kernel_Self_Protection_Project https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Pr... is basically trying to upstream grsec's good ideas in upstream-friendly ways. (Oddly, grsec itself has expressed opposition to Rust in the kernel, I think on the grounds that their custom GCC plugins can't cross the language boundary. Now that there's work on supporting Rust in GCC itself, I'm not sure if that objection still applies.)
- steeleduncan 4y agoHow do these Rust kernel modules handle out of bounds access in an array? In a C module it would be undefined behaviour leading to kernel panic or weird bugs, in a Rust userspace binary it would be a panic that terminates the process with an error message, what happens here? Would it be a kernel panic as well? or is there a possibility of handling it?
- EugeneOZ 4y agoYou can catch the panic. It will panic, but I don't know if the driver will catch it. I hope so :)
- ncmncm 4y agoThis not actually substantively different from throwing an exception.
- EugeneOZ 4y agoIndeed, but it's much better than undefined behavior :)
- davidatbu 4y agoI believe Rust in linux was made so that it never panics. Here's a patch that removes panicking allocations for example: https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@kernel.org/ https://lore.kernel.org/lkml/20210704202756.29107-1-ojeda@ke... (but I think all other instances of panicking were removed as well). EDIT: Look at replies. "Linus considers panics acceptable in some cases".
- ncmncm 4y ago
- ncmncm 4y agoSpiteful downvotes notwithstanding, there really is nothing Linus can do to stop you coding your eBPF fragments in C++.
- galangalalgol 4y agoeBPF doesn't prevent UB, just keeps that code from hurting the rest of the kernel right? I have written c++ for 28 years and I can only recall letting something rust would have protected me from make it out of test once. That I know of... A coworker found some UB in a rather large c++98 codebase recently. It wasn't exactly idiomatic modern c++. But the same person writing the same code in c++20 would just have had explicit and auto in more places. It was a use after free on the stack that wasn't terribly subtle but I only found it because I knew something was there. Cppcheck and clang-tidy were no help. Debug builds didn't exhibit the UB with g++ or clang, clang didn't exhibit it with -O3 (optimized out), but g++ did. Asan would have caught it, if they used asan (they don't) and if they had unit test coverage for that branch (also no). So experienced c++ developers with years of experience (no novices on that codebase) made an error that can't be caught by static analysis, and requires 100% branch coverage for asan to catch. It eventually popped up in production. I find modern c++ much more pleasant to write than rust. I like that it is easy to interface with the rest of the world. I hate spending eons to get unit tests to find that last path theough the code in a meaningful way. I hate waiting for pipelines to run unit tests and coverage to find a mistake. Modern C++ built with static analysis (including modernization checks) takes as long as a rust builds, but still doesn't catch as much stuff. I'm not using rust yet, but I'm planning to start. I'm tired of this stuff, it's been almost 30 years.
- selfmodruntime 4y agoYou‘re also missing that there is a chance of UB in your code base for which your compiler just assumes your original intention and will then go on to correct it
- davidatbu 4y agoQuestion: The author of the 9P server module "created an executor that can run async code in a kernel workqueue"; is this made a lot easier by the fact that Rust doesn't include an async executor as part of the core/stdlib?
- JoshTriplett 4y agoThe work on Rust in the Linux kernel doesn't include the standard library, and even if it did, having an executor around doesn't prevent you from creating and using a different one.
- davidatbu 4y agoThank you!
- estebank 4y agoI think they are pointing out that because the Rust design has decoupled the functionality from the implementation, these kind of replacements are even possible in the first place. Of course, if you use an alternative executor there are large swaths of the (non-std) async ecosystem that are not available to you (thinking of anything like relying on a tokio::timer::Delay as an example). But as you point out Rust in the Linux kernel already doesn't rely on the std, so they wouldn't even begin to imagine to use an off the shelf crate and expect it to work unmodified for their use case. I just think it is worth pointing out that this is a constraint we currently have.
- davidatbu 4y agoYou're better at expressing my thoughts than I am :)
- littlestymaar 4y agoestebank reads other people's mind, that's why he's so good at writing helpful compiler errors.
- Kukumber 4y ago
- galangalalgol 4y agoYou forgot to insult vi or emacs and denegrate some brace style.
- iExploder 4y agoTypical spaces over tabs enjoyer...
- azernik 4y agoReligions generally don't phrase their actions as experiments, as this developer is doing. Engineers test new techniques to see what works. Not all of them play out. The Linux kernel community is very good at sorting out the junk, and this is part of that process.
- sophacles 4y agoWhat junk am I subscribing to by thinking compiler checking of my memory access is neat, and by thinking it has uses in the kernel (which manages memory as one of its jobs IIRC)?
- Kukumber 4y agoRepeating what i said elsewhere: My issue is people who will "trust" security only because of a language used, wich is a flawed and dangerous mindset I don't want to depend on that kind of project
- sophacles 4y agoIm confused. You are going to change an OS because you think "trusting something based on use of a particular language" is a problem that might come up. Isn't that just you basing your trust level on the use of a particular language?
- 4y ago
- ainar-g 4y agoI did not expect Plan 9, the Linux kernel, and Rust to intersect like that! Would adding this module mean that I could finally mount a 9P filesystem on my Linux machine without the need for third-party software and FUSE?
- remexre 4y agoYou already could, actually! zgrep CONFIG_9P_FS /proc/config.gz to see if your kernel's configured for it, modprobe 9p if it's built as a module. The article's talking about doing an in-kernel 9P _server_ instead; I think this is just so that it can act as an abstraction of an SMB server, because you ought to be able to do a 9P server in userspace with no loss of capabilities...
- ainar-g 4y ago/proc/config.gz seems to be unavailable to Debian 11, at least by default, but "modinfo 9p" does show the module. Thanks for the clarification!
- rascul 4y ago> /proc/config.gz seems to be unavailable to Debian 11 For clarification: > /proc/config.gz isn't available in Debian, because the config is provided in /boot/config-*, no need for the in-memory variant https://wiki.debian.org/KernelFAQ#line-46 https://wiki.debian.org/KernelFAQ#line-46
- MisterTea 4y ago> I think this is just so that it can act as an abstraction of an SMB server, because you ought to be able to do a 9P server in userspace with no loss of capabilities... Correct, 9p file servers can live in user-space. At some point you have to mount the fs which on lunix/unix is done in-kernel. In plan 9 the kernel is just a VFS that lets you build arbitrary file trees from collections of mounts and binds. On-disk file systems like FAT and EXT are served via user-space programs which open the disk partition (served as a file by sd(3)) and translates the 9p messages to EXT, FAT, etc operations. File servers can also provide synthetic files, e.g. an arduino can serve its io bits and/or internal variables over 9p (ninepea on github written by echoine.) Since its a VFS all the way down and data moves via two way pipes mapped to fd's you can boot and pull root from ANY 9p server. Don't care if its CWFS, HJFS, FAT, EXT, SMB, SSHFS, NFS, etc. Kernel don't care as long as it sees files through a 9p connection over whatever 2-way medium you are using including rs232. You cant touch that level of flexibility with other operating systems.
- zppln 4y agoThis was a pretty weak article? All it basically says is that it can be done and adds some speculation that it might make things better. But not much testing have been carried out and the comparisons to existing code that have been made show the Rust implementation to perform worse.
- kibwen 4y ago> show the Rust implementation to perform worse Here in the Linux Plumbers Conference talk regarding the NVMe driver (the same one discussed in the OP), the benchmarks show nearly identical performance to the existing implementation. At the end of the talk the co-author of the NVMe spec (Matthew Wilcox) made a point to stand up and comment with how unexpectedly impressed he is with the performance: "I was not expecting to see these performance numbers, they are amazing." (at this timestamp: https://youtu.be/Xw9pKeJ-4Bw?t=9486 https://youtu.be/Xw9pKeJ-4Bw?t=9486 ). EDIT: here's the timestamp of the benchmarks themselves: https://youtu.be/Xw9pKeJ-4Bw?t=8626 https://youtu.be/Xw9pKeJ-4Bw?t=8626
- Karellen 4y agoLinux Weekly News focusses heavily on work going on in the Linux kernel, and often looks at features and patch sets which aren't quite ready for mainline yet, but could be an interesting shift in direction for the kernel in future. The inclusion of Rust in the kernel is one of these ongoing bits of work which isn't ready for mainline yet, but has still generated a lot of interesting discussion in the past on whether Rust is suitable at all, or whether it's suitable in it's default form, or if it is suitable then what is the best way to start introducing it, or, or, etc... And this is another step along that (long, convuluted) path. The proposed 9P implementation specifically - to me at least. If you mostly care about the Linux kernel insofar as what features it can offer you right now then, sure, this might not seem that interesting. And that's entirely fair - there are plenty of other technologies out there that I have that level of interest in. But for the people who take a deep interest in the upcoming work and future direction of Linux, I think it's quite interesting. I'd agree that the article probably is a bit too niche, or forward-looking, for a generic tech website/magazine, and doesn't have a huge target audience. But for a focussed site like LWN, I think it fits in pretty well, and matches the audience that LWN targets. It's definitely the kind of article that I keep my subscription for.
- nu11ptr 4y agoI'm a huge fan of Rust (I write in it for a living), but I still can't help wonder how they are going to tolerate the compile times. A couple of drivers...no problem, but what happens once they get a large number of drivers in Rust? Compile times will definitely start going up considerably. I wonder how much this has been considered, or possibly they have no plans to do anything this extensive?
- CraigJPerry 4y agoIt might not be a big hurdle. OS developers aren’t as sensitive to compile times as app devs. An hour of futzing to bring up on a hardware dev board isn’t unheard of (although 5-10 mins would be more typical). There was a big regression in compile times in the linux 2.2 to 2.4 time frame (the kernel grew a lot of functionality) but i don’t recall it getting much more than a little grumbling. That was in the time of the Linux from Scratch book and Gentoo stage 0 builds. Some people don’t mind overnight compiles if it means they can specify just exactly the right combo of compiler flags to eeek out the max performance of their cpu.
- guenthert 4y agoI worked on a (legacy now) larg-ish code base (mostly C++ '03 with some self-imposed restrictions, but extensive usage of templates) for an 'embedded' system (enterprise NAS). A full build took north of 6h on the reasonable beefy desk-side workstations available to us (we considered distributed builds, but never got to test it afair). Every once in a while there was some grumbling about the build times, but it never really improved (every three years or so we got better build machines, but by that time the code base had grown as well), partly, I suspect, due to https://xkcd.com/303/ https://xkcd.com/303/ and partly because we learned to work-around that by finding the Makefile in the tree which (likely) gave us a proper build in the shortest possible time for the files we were actually working on (not being helped by the most complex Makefile tree I've ever seen). I very much doubt that such large build times would go over well in the Linux kernel community.
- yencabulator 4y agoKernel compiles are a classic benchmark by which Linus has judged performance-related patches. Kernel developers do care(, mostly about incremental builds).
- pipeline_peak 4y ago> and unlike in C++, we can write it and actually trust that it's correct Some of these people are so obnoxious...
- playdmc2 4y agoobnoxious about what? c++ is so brittle that a gentle breeze could cause a cacophony of soundness issues
- pipeline_peak 4y ago> obnoxious about what? Vaguely descriptive attacks that serve no purpose other than religious wars. When I see statements like that I see another person who thinks it’s simply fashionable to hate popular languages because they heard someone like Torvalds, Thompson, or Stallman say it. If you have a fundamental explanation like they do, then it's at least reasonable, disregarding the irrelevant context of a Linux kernel forum thread. But I rarely see that, instead I find mutter like "cpp is bloated, unstable, overcomplicated crap with ugly syntax, that's why I use rust, or nim, or any other pretty new thing that was conceived 35+ years later". For instance: > c++ is so brittle that a gentle breeze could cause a cacophony of soundness issues Okay, how so, anything about C++ or just C in general?
- tialaramex 4y agoThere's a design principle here, rather than just some sort of hubris. In languages as powerful as Rust and C++ you can express programs which fall into three categories, two of these aren't very interesting, the programs which are obviously valid, and the programs which are obviously nonsense. We know what to do with these programs, the former should result in emitting correct machine code, the latter should cause a diagnostic (error message). The problem is the last group, programs whose validity is difficult to discern. The bigger and more complicated your software the more likely it may end up in this last group. In C++ the third category are treated as valid. In the ISO standard this is achieved by having clauses which declare that in certain cases the program is "Ill-formed, no diagnostic required" which means the standard can't tell you what happens, but you won't necessarily get an error message, your compiler may spit out a program that does... something. Maybe it does what you expected, and maybe it doesn't, the standard asks nothing more. In Rust these are treated as invalid. If you try hard enough (or cheat and Google for one) you can write Rust programs which you can reason through why they should work but the compiler says no. You get an error message explaining why the compiler won't accept this program. Now, if Rust's compiler doesn't like your program, you can rewrite it so that it's valid. Once you do that, which is often very easy - you're definitely in that first category of correct programs, hooray. The program might well do something you didn't intend, the compiler isn't a mind reader and has no idea that you meant to write "Fnord" in that text output and "Ford" is a typo, but it's definitely a valid Rust program. In the C++ case we can't tell. Maybe our program is the ravings of a lunatic, the compiler isn't obliged to mention that and we are in blissful ignorance. This also provides little impetus for the standard's authors or compiler vendors to reduce the size of the third category of programs, after all they seem to compile just fine. Obviously it'd be excellent in theory to completely eliminate the third category. Unfortunately Rice's Theorem says we cannot do that.
- opless 4y agoThe 9P file protocol, he said, comes from the Plan 9 operating system. The kernel has a 9P client, but no 9P server. There is a 9P server in QEMU that can be used to export host filesystems into a guest. The protocol is simple, Almeida said, defining a set of only ten operations. His 9P server implementation works now, in a read-only mode, and required just over *1,000* lines of code. ... I wonder how many lines of code the Plan9 (C) server takes? (much less I suspect)
- deleted 4y ago[deleted]
- tialaramex 4y agoFrom previous example drivers, I'd expect that it's not dissimilar overall and mostly the cause will be style preferences e.g. maybe the C programmers loves one-line for loops and the Rust programmer chooses default Rust style which doesn't do that, or contrariwise the C programmer finds it helpful to write out complicated variable types across multiple lines and the Rust programmer was comfortable letting them be inferred instead. If the C ends up hand rolling something Rust just has built-in, that adds up pretty quickly. For example Rust's arrays, strings etc. know how big they are, C's do not and so you need extra code to re-calculate or to explicitly store and retrieve those sizes. On the other hand it may be tempting to express a Rust type as having methods defined over it, rather than as you must in C using only free functions throughout, there's also a Rust discipline of always deriving common traits e.g. Clone, Debug and Eq, when they're appropriate, as a convenience, that's one line extra in typical style.
- yencabulator 4y agoThis Linux 9P server is a 9P-to-Linux-VFS translator. No such software component exists in Plan9. You can think of the Plan9 kernel as a 9P multiplexer. 9P in, 9P out. The closest matching thing would be e.g. a 9P-to-ext4-in-a-file translator -- but it's always "9P server talking to $FOO" (or "9P server that makes files up on the fly"). Most of the complexity of this Linux 9P server would be in translating between the unrelated worlds of 9P and Linux VFS. A more apt comparison is the size of this 9P server vs an in-kernel NFS server -- they both translate something unrelated to Linux VFS.
- ArrayBoundCheck 4y agoSo one person said creating the rust interface is very difficult to do correctly and the other said rust bare metal isn't complete enough for his use case I called both of these (and compile times) when people first tried to get rust on the linux kernel. Maybe it's better to acknowledge these problems before you start instead of beating your head? I'm no psychic and I know I wasn't the only one who called these
- dogleash 4y ago> Maybe it's better to acknowledge these problems before you start instead of beating your head? What's unacknowledged? They're doing exploratory work. That often involves developing under known and expected limitations to understand the problem space and have something to test with when progress is made towards addressing limitations.
- ArrayBoundCheck 4y agoThe fact that writing unsafe rust code is harder than unsafe C due to more rules? The fact that Rust has little support of nostdlib which is what the kernel is using? There's also the fact that most of the time in drivers (in my limited experience) you almost always want unsafe things but I assumed noone would think or admit that one