10 ms·
The Linux Kernel Prepares for Rust 1.77 Upgrade
- SushiHippie 3y agoDo I understand it correctly that upgrading to a new rust version is mostly implementing new best practices and new features, instead of needing to "fix" your code, as rust is backwards compatible? I've only used rust nightly for my own projects and didn't give too much thought about rust versions
- codetrotter 3y agoFor normal projects yes. And you can use clippy to tell you about changes you should make. For example, in my projects I run this in the CI pipeline: cargo clippy --all-targets --all-features and cargo fmt --all --check In addition to the regular test and build steps. This both means that I follow clippy recommendations and cargo fmt in the first place, and also that my CI tells me about any clippy changes if I didn’t notice them myself as well as any formatting I’m not following. In my main IDE I auto format the code of course. But sometimes I make small changes in vim and don’t run the format step myself so it’s nice to have for that reason as well. For the integration of Rust into the Linux kernel I imagine it’s a bit more convoluted.
- estebank 3y agoFor Rust on Linux it's a bit more involved because they use nightly features, which can change from day to day. In practice there's an implicit tiered strata, with features that rarely change and features that frequently change in bursts. I would like it if we formalized that distinction a bit. We already mark when a feature is very likely to change (unstable_features), but not when they are very close to being stabilized.
- CUViper 3y agoRust is backwards compatible when you stick to stable features, but the kernel uses unstable features that can and do incur breaking changes. https://github.com/Rust-for-Linux/linux/issues/2 https://github.com/Rust-for-Linux/linux/issues/2
- surajrmal 3y agoIt seems prudent to limit rust usage in the kernel until that list can be burned down to zero. It makes sense that you need to at least get rust in the kernel to find out what missing features you need to have implemented and stabilized, but excessive use will make folks lives painful as they try to track upstream rust releases.
- estebank 3y agoWhat you propose is exactly what's been done by the kernel. They are integrating the language in a non-mandatory way, to both exercise the kernel side and the language itself. The unstable features haven't been stabilized because either they have open questions on their implementation (and having a customer using them helps define them) or no-one has cared enough to complete them (and having a customer using them gives them the extra push). Either way what's happening now is exactly the process you are proposing. The article is about updating the Rust version the kernel targets where a feature they use (offset_of) was stabilized.
- fl0ki 3y agoPlease bear in mind that Linux has used non-standard GCC extensions to C for decades as well. The tradeoffs here are their call to make. Besides, at this stage, it makes perfect sense for Linux to use unstable Rust features. It was one thing to say Rust should be great for writing kernels, it's another to actually get feedback on how it needs to be better, and that's only possible if the potential improvements are motivated by those who need them and incubated without the constraints of backwards compatibility nor the risks of locking in permanent tech debt. Rust's unstable feature concept was designed for exactly this kind of freeform evolution and it's working exactly as intended. As for the specific tradeoffs being made in Linux, its contributors are in a much better position to weigh those than we are.
- pornel 3y agoRust-for-Linux made it more complicated for themselves, because they chose to enable unstable/experimental features of the compiler without waiting until they’re released, so they don’t get the stability and compatibility guarantees that normal Rust projects get.
- bonzini 3y agoThe alternative was to do the same and also not merge what they had. Stuff like custom allocator support is not optional.
- saghm 3y agoI was confused when reading this because I was pretty sure that using other allocators had been supported for a while in Rust. From refreshing myself on the details, it seems that replacing the default allocator is stable (https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html https://doc.rust-lang.org/std/alloc/trait.GlobalAlloc.html), but the API for arbitrary allocators (which includes stuff like being able to do "zero-size" allocations) is not yet stable (https://doc.rust-lang.org/std/alloc/trait.Allocator.html https://doc.rust-lang.org/std/alloc/trait.Allocator.html). I guess if there were ever a project that needed fine-grained control over how allocators be work, it would be the kernel.
- charcircuit 3y agoMore importantly allocator_api adds try_new which means you can handle allocation errors.
- saghm 3y agoAh, that is important. I didn't even notice that wasn't possible in the GlobalAlloc API, but you're definitely right that it's not.
- LegionMammal978 3y ago
- steveklabnik 3y agoYes, generally. For example, recently David Tolnay shared the amount of burden Meta has when upgrading the compiler: https://old.reddit.com/r/rust/comments/19dtz5b/freebsd_discusses_use_of_rust_in_the_base_system/kj8qfso/ https://old.reddit.com/r/rust/comments/19dtz5b/freebsd_discu... > I estimate it's about ½ hour per 1 million lines, on average. That being said, Rust for Linux isn't using stable Rust, so they have a higher burden than projects that do.
- almatabata 3y agoIf you ignore dependencies and stick to stable features yes. If you include dependencies then it can happen that a dependency relies on unstable features. In which case you might have to upgrade the library version (if they support the new compiler version). The library might have changed the API by then which would force you to change your code. Except for the above use case, upgrades to the latest version of the compiler have been painless for me.
- paavohtl 3y agoDependencies are not allowed to use unstable features either with the stable compiler. The only exception is the standard library, which uses numerous unstable features even with a stable distribution of Rust.
- tialaramex 3y agoWorth briefly explaining the rationale for this (stdlib gets to use unstable features) Rust's stdlib is maintained with the rest of the language and by the same broad team, so, if you're tweaking unstable feature X, you are also responsible for ensuring the stdlib people using feature X sort that out. I'm not sure if Rust's internal policies mean you shouldn't land a change to the main tree without accompanying stdlib patches, or whether you're only required to give them adequate notice, but either way it's not going out the door in a stable release being incompatible with its own implementation. This couldn't really work with 3rd party libraries.
- jcranmer 3y agoThere's a second category of unstable features to mention here. Some of the features are essentially perma-unstable, because they're exposing some compiler intrinsics for the library to be able to use. This is the equivalent of things like __builtin_* for C compilers.
- kramerger 3y agoThe LKM post mentions binary size improvements. One issues I have had with Rust applications is the huge binary size (yes, I know this has improved a bit lately). Is there a good comparison between kernel C and kernel Rust code in this regard?
- makapuf 3y agoFrom (1) I guess most of the binsize comes from stdlib that the kernel does not use, so I guess that's two different problems. (1) https://github.com/johnthagen/min-sized-rust https://github.com/johnthagen/min-sized-rust
- vlovich123 3y agoThat wouldn’t explain what the size improvements are from the upgrade as the kernel always built nostd.
- surajrmal 3y agoMost size issues come from not using release builds, using too many dependencies, or overuse of generics. The rust std lib being linked in statically also contributes.The kernel shouldn't suffer from any of these problems. Plenty of embedded use is able to use rust in highly constrained environments without size issues compared to C.
- zadokshi 3y agoSaying part of the problem is “using too many dependencies” is not an overly helpful thing if the ecosystem keeps on trying to download 3Gb of build dependencies because you tried to use some simple little library. The problem is obvious, it’s the solution that is much more difficult.
- Aurornis 3y ago> if the ecosystem keeps on trying to download 3Gb of build dependencies because you tried to use some simple little library. Downloading 3GB of dependencies is not a thing that happens in the Rust ecosystem. Reality is orders of magnitude smaller than that. Why are you exaggerating so much? Some people bristle at the thought of external dependencies, but if you want to do common tasks it makes sense to pull in common dependencies. That’s life.
- bsd_source 3y agoRust is going to kill LFS
- horeszko 3y agoLFS is Linux from Scratch right? (https://www.linuxfromscratch.org/index.html https://www.linuxfromscratch.org/index.html) What are the implications of using Rust on building Linux?
- rekado 3y agoThe Rust bootstrapping story is ugly. https://guix.gnu.org/blog/2018/bootstrapping-rust/ https://guix.gnu.org/blog/2018/bootstrapping-rust/
- charcircuit 3y agoLFS does not need to do all of that. LFS already uses the host computer's C compiler, so it seems just as reasonable to also use the host computer's rust compiler.
- MuffinFlavored 3y agoDoesn't part of the Rust compliation chain end up using `cc` anyway eventually for like, linking or something? That might not apply at a "system's" level but I'm guessing in the massive Linux compilation job with module support you're making a bunch of object files with exported symbols?
- charcircuit 3y agoThe Linux kernel's build system uses rustc only for making object files.
- legobmw99 3y agoThat honestly doesn’t seem that horrible - particularly compared to some of the nigh-on-impossible tasks like bootstrapping GHC (For the curious) https://elephly.net/posts/2017-01-09-bootstrapping-haskell-part-1.html https://elephly.net/posts/2017-01-09-bootstrapping-haskell-p...
- bobse 3y ago[dead]
- amelius 3y agoHow many % of the kernel is Rust now, in terms of LoC?
- rwmj 3y agoRust: 20,887 lines C: 33,351,596 lines (That's just doing 'wc -l' rather than using any proper code metrics tool)
- amelius 3y agoOk. Does that include device drivers?
- izacus 3y agoYes.
- geertj 3y agoHow much of those 20K lines are Rust infrastructure, versus drivers written in Rust?
- bonzini 3y agoThe latter are basically a rounding error, though there is a rewrite of the Android binder driver.
- SushiHippie 3y agoI've run tokei [0] on the Linux git repository on the latest commit [1] and this was the output: =============================================================================== Language Files Lines Code Comments Blanks =============================================================================== C 33553 23772322 17694564 2662642 3415116 C Header 24554 9562920 7395591 1436546 730783 Device Tree 5041 1512839 1240129 76384 196326 ReStructuredText 3473 711669 539971 0 171698 JSON 788 443098 443096 0 2 YAML 3905 433860 352107 16626 65127 GNU Style Assembly 1317 372131 271873 55613 44645 Shell 894 172036 120033 21590 30413 Plain Text 1739 151033 0 123992 27041 Makefile 2946 76889 52985 12355 11549 Python 203 68545 54627 4452 9466 SVG 74 49420 48159 1171 90 Perl 59 43992 34124 4074 5794 Happy 10 6069 5359 0 710 Assembly 5 3319 3065 0 254 C++ 5 2138 1860 61 217 BASH 59 1943 1318 335 290 Unreal Script 5 707 445 158 104 ASN.1 16 660 445 87 128 Autoconf 5 429 373 26 30 LD Script 8 376 288 29 59 CSS 3 295 172 69 54 Gherkin (Cucumber) 1 291 199 58 34 TeX 1 236 156 74 6 XSL 10 200 122 52 26 HEX 2 173 173 0 0 Module-Definition 2 128 113 0 15 C++ Header 2 125 59 55 11 RPM Specfile 1 108 93 1 14 Objective-C 1 89 72 0 17 Vim script 1 42 33 6 3 Markdown 1 36 0 27 9 Automake 3 31 23 3 5 Ruby 1 29 25 0 4 INI 2 13 6 5 2 TOML 1 12 2 9 1 Apache Velocity 1 12 12 0 0 CMake 2 8 8 0 0 ------------------------------------------------------------------------------- Rust 64 12637 9489 1612 1536 |- Markdown 55 8243 808 5557 1878 (Total) 20880 10297 7169 3414 ------------------------------------------------------------------------------- HTML 2 28 22 3 3 |- JavaScript 1 7 7 0 0 (Total) 35 29 3 3 =============================================================================== Total 78760 37400888 28271191 4418115 4711582 =============================================================================== So If we would only count code and not comments, it is only 9489 LoC Rust. Which would be about 0.03% and if we take all lines and not only LoC it would be around 0.05% [0] https://github.com/XAMPPRocky/tokei https://github.com/XAMPPRocky/tokei [1] https://github.com/torvalds/linux/commit/b401b621758e46812da61fa58a67c3fd8d91de0d https://github.com/torvalds/linux/commit/b401b621758e46812da...
- loeg 3y agoSo this is a huge tangent but couldn't most of the uses of the non_null!() macro in this diff just be (safe!) pointer comparisons or subtractions, without the unsafe{} logic to convert a pointer value to a reference just for the purposes of comparison or subtraction? https://lore.kernel.org/lkml/20240217002717.57507-1-ojeda@kernel.org/ https://lore.kernel.org/lkml/20240217002717.57507-1-ojeda@ke...
- saghm 3y agoAlthough I think I recall that the kernel doesn't use Rust's standard library (which is consistent with the diff you linked), it's possible that the standard library's documentation on the pointer subtraction might reference a concern they could share (https://doc.rust-lang.org/std/primitive.pointer.html#method.sub https://doc.rust-lang.org/std/primitive.pointer.html#method....): > If any of the following conditions are violated, the result is Undefined Behavior: > * Both the starting and resulting pointer must be either in bounds or one byte past the end of the same allocated object. > * The computed offset cannot exceed isize::MAX bytes. > * The offset being in bounds cannot rely on “wrapping around” the address space. That is, the infinite-precision sum must fit in a usize. > Most platforms fundamentally can’t even construct such an allocation. For instance, no known 64-bit platform can ever serve a request for 263 bytes due to page-table limitations or splitting the address space. However, some 32-bit and 16-bit platforms may successfully serve a request for more than isize::MAX bytes with things like Physical Address Extension. As such, memory acquired directly from allocators or memory mapped files may be too large to handle with this function. > Consider using wrapping_sub instead if these constraints are difficult to satisfy. The only advantage of this method is that it enables more aggressive compiler optimizations. If their pointer subtraction uses similar semantics, there might be issues if they want to compare pointers from different allocation objects, are worried about 32-bit or 16-bit platforms, or maybe even consider the performance concerns too worrisome. The Rust I write tends to be a bit higher-level and doesn't require unsafe, so my instinctual reaction to "using unsafe for performance" generally errs on the same of abject terror, but it's a fundamental part of what makes the safe side of abstractions I use possible, and the kernel is probably one of those places that needs to do that sometimes, so I'd reluctantly have to admit I'm probably not qualified to evaluate whether these cases would merit unsafety for performance alone, but the first two concerns sound like legitimate things that the kernel would need to handle.
- 1letterunixname 3y agoMy wishlist would include gradually refactoring core in Rust and formal verification a-la seL4 to prove correctness. There's no point to refactor churn from one language religion to another for low entropy, core code without improvements in assurance that it's also provably bug-free, race-free, and secure while also being as fast or possibly faster.
- deleted 3y ago[deleted]
- sroussey 3y agoCurious, why Rust instead of Zig?