23 ms·

I'm opposed to this, and it's not because of some idiologic kernel-should-be-pure-c thing. I'm opposed to it because the rust compiler is slow. And the rust com

I'm opposed to this, and it's not because of some idiologic kernel-should-be-pure-c thing. I'm opposed to it because the rust compiler is slow. And the rust compiler is written in rust. Compiling rust is a nightmare if you don't have a high-end PC. I want to be able to actually compile my software if I wish so. This is becoming increasingly difficult. Rust is adding to the problem.

_ph_ 4y ago
While I lack personal experience with Rust and I really appreciate fast compilers (that is why I am a Go user), over all features and characteristics, Rust seems to be the best choice for safe kernel development. Other posters have described well how urgent it is, to improve the security of kernel code. So just not doing anything about this, doesn't seem to be a good option. It seems, there is a wide group of developers which thinks that Rust is the best candidate as a kernel development language. If you see issues with that choice, now would be the time to propose an alternative and try to find momentum in the developer community supporting that alternative. While I also lack practical experience there, by all what I heard, ADA could be one. But I don't know how it exactly compares to Rust and what the trade offs are. But so far, no one has pushed for ADA as a possible kernel implementation language.
cube00 4y ago
I'm not sure it's realistic to expect a safety focused compiler to compete with one that doesn't offer those checks. We should aspire to make it as fast but in the short term a slower compiler in exchange for less CVEs and random buffer overrun crashes seems like a reasonable trade off to me. Distributions such as Fedora offer build infrastructure[1] that you can use to compile packages to use in your system for testing if you feel your local hardware isn't powerful enough. [1]: https://copr.fedorainfracloud.org https://copr.fedorainfracloud.org
- raverbashing 4y ago
  I think it's more related to the grammatical complexity of the language than to safety itself (Same reason why C with typedefs is slower to compile than plain C, why C++ is slower, etc) (that and cargo dependencies, etc - also C compilers have some +30yrs of optimizations)
  - akira2501 4y ago
    I've seen some Rust programs that look nice, but in the kernel, the code ends up making perl'ls propensity for line noise look competitive. I'm sure some of that is the need to shovel between languages, but it does make you wonder if the entire effort is a little misguided. A full Rust kernel might be neat. A compromised Rust driver inside of a C kernel is never going to be the default choice.
    - bonzini 4y ago
      Most of the code committed to the kernel is infrastructure for C<->Rust interoperability. It's the least readable of it all, and as fun to work on as watching paint dry. The actual driver code is pretty readable, see https://lwn.net/Articles/863459/ https://lwn.net/Articles/863459/.
    - coldacid 4y ago
      >A full Rust kernel might be neat. May I interest you in Redox? https://www.redox-os.org/ https://www.redox-os.org/
    - steveklabnik 4y ago
      If "embedded OS" counts as "full Rust kernel", you may want to check out https://hubris.oxide.computer/ https://hubris.oxide.computer/ Obviously very different than Linux.
  - WastingMyTime89 4y ago
    More complexe languages are faster to compile. The issue is mostly the implementation. It could, should and will be better but it takes time.
  - api 4y ago
    As others have pointed out most of the performance hit is in the optimization of the relatively verbose IR code the Rust compiler sends to LLVM, not the Rust compiler itself. There's been multiple passes at making the Rust compiler faster. It'll happen. There's a GCC-based Rust implementation in the works too.
- goodpoint 4y ago
  Nim is also memory safe and the compilation is really fast.
  - codewithcheese 4y ago
    > No, Nim doesn't have stronger or weaker safety guarantees than Rust [1]. Rust's memory safety is nothing new, either. It is mostly that some older languages like C/C++ are the exceptions in not being memory-safe. There is nothing new or magical about memory safety. LISP was already memory-safe when it was invented in 1958. The only question is how much performance you need to trade away for it (the value is never zero for non-trivial programs, but can vary greatly, depending on whether the language was designed with it in mind or not). > The main difference between Rust and other languages is that it does some more (but not all [2]) safety checks at compile time rather than at runtime. It also allows you to avoid GC, but does not provide you any memory safety over GC. Rust's borrow checker allows you to statically prove that references are live [3]; a GC simply avoids deallocating any memory that has a live reference to it (on the other hand, a GC can ensure that references remain live even where this is hard or impossible to prove statically). The end result is the same with respect to memory safety (the reason some people want to avoid GC is for performance reasons, not memory safety). https://forum.nim-lang.org/t/1961 https://forum.nim-lang.org/t/1961
    - goodpoint 4y ago
      That's outdated information from 2016. Now Nim uses ARC/ORC. ARC implements memory management at compile time like Rust and it's still very fast at compiling.
      - codewithcheese 4y ago
        Cool, I wasn't aware, thanks. https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc-in-nim.html https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc... > The main difference between ARC and Nim GCs is that ARC is fully deterministic - the compiler automatically injects destructors when it deems that some variable (a string, sequence, reference, or something else) is no longer needed. In this sense, it’s similar to C++ with its destructors (RAII). To illustrate, we can use Nim’s expandArc introspection (will be available in Nim 1.4). > This shows one of the main ARC features: scope-based memory management. A scope is a separate region of code in the program. Scope-based MM means that the compiler will automatically insert destructor calls for any variables which need a destructor after the scope ends. Many Nim constructs introduce new scopes: procs, funcs, converters, methods, block statements and expressions, for and while loops, etc. > ARC also has so-called hooks - special procedures that can be defined for types to override the default compiler behaviour when destroying/moving/copying the variable. These are particularly useful when you want to make custom semantics for your types, deal with low-level operations involving pointers, or do FFI.
- rob74 4y ago
  Actually C/C++ compilers are shooting themselves in the foot performance-wise with the header mechanism, so writing a compiler that's faster than C/C++ is not that hard. Not sure what the reasons of the Rust compiler being (even more?) slow are, maybe the ability to easily interface with C code has something to do with it?
  - nine_k 4y ago
    Last time I checked [1], the Rust part of the Rust compiler is not the slowest. LLVM spends a lot of time processing relatively poor IR code which the Rust compiler emits. Rust compilation is not intrinsically slow; better IR and more incremental compilation can and will improve it. Incremental compilation (only changed files, not crates) is 7 years in making though. [1]: https://prev.rust-lang.org/en-US/faq.html#why-is-rustc-slow https://prev.rust-lang.org/en-US/faq.html#why-is-rustc-slow
    - rob74 4y ago
      Thanks for that link! That sounds like the really major factor is the decision to recompile the entire crate when a file in the crate changes. I'm sure that made sense at the time ;) And, if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision - incremental compilation might have its edge cases and pitfalls, but other compilers are doing it just fine, so it shouldn't be rocket science? Other reasons: - suboptimal LLVM IR code, as you mentioned, and other tech debt - the preferred strategy of monomorphising generics is fast at runtime, but slow to compile, using trait objects is faster - the complex type system of course also plays a part - LLVM itself is of course not optimized for Rust
      - rbanffy 4y ago
        > if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision Maybe we are giving Rust developers computers that are too fast. ;-) OTOH, it's always wise to design for the future, so it also makes sense to give Rust developers beefy server grade hardware so they can play with SIMD pipelines and all those extra cores and threads, because that's what workstations will be a couple years from now.
- pjmlp 4y ago
  There are languages with similar type safety, which are faster, because the authors have placed focus on having several ways available to compile the code. Ada, Delphi, OCaml, C#/F# (.NET Native / Native AOT), D, Nim,...
  - 0dayz 4y ago
    I will admit I never understood why Ada never took off (is it only because of the tooling being propertiary for a long time?), but c# is not as memory safe as rust.
    - pjmlp 4y ago
      The only memory safety that C# lacks in comparisaion with Rust is one special case, data races for in-process data. For everything else regarding concurrent access to shared data out of process, they are on the same foot. Ada didn't took off, because of several reasons, price of compilers, on UNIX it was an additional SKU on top of the respective developers SDK that already offered C and C++ in the box, 1980's mainstream hardware wasn't able to cope with it, most OS vendors outside UNIX decide to migrate from their toolchains into C and C++, so again additional money on top of the OS SDK.
    - thesuperbigfrog 4y ago
      >> I will admit I never understood why Ada never took off (is it only because of the tooling being propertiary for a long time?) Early Ada compilers had expensive licensing and required expensive hardware to run. By the time GNAT was added to GCC, C++ had already taken over most of the spaces that were not Ada exclusive (meaning safety critical / defense / aerospace niches where either Ada was once mandated or has thrived in despite the original high costs).
  - mejutoco 4y ago
    Does C# not have a GC? I thought that would immediately disqualify it for kernel work.
    - theta_d 4y ago
- VikingCoder 4y ago
  What's the old adage? It's easier to make a correct thing fast than it is to make a fast thing correct.
Xylakant 4y ago
I remember the times when installing Gentoo was a matter of days. Compiling the kernel was a matter of hours. Sure, rustc is slower than gcc, but if you're not constantly compiling your software over and over and over again, then the time spent in the compiler is not your primary concern. Start the compile, go to bed, let it run.
- akira2501 4y ago
  Gentoo.. the only reason I'd ever know or need to know that 'distcc' is a thing that exists.
  - srazzaque 4y ago
    Ha! I was a wimp and did a "stage 3" install back in the day. Or was it stage 2? (shrug). Now I'm even wimpier and just use Fedora. Came across this page, on the motivations of Gentoo, a few years later: > Installing a working Linux box used to require over 550 man hours, learning a Nordic language, sacrificing a goat, wading through hundreds of pages of (purposely) inscrutable help files...Old-school Linux users were desperate to find a new way to feel superior. https://en.uncyclopedia.co/wiki/Gentoo https://en.uncyclopedia.co/wiki/Gentoo
    - speed_spread 4y ago
      It honestly wasn't bad at all. Daniel Robbins documentation for Gentoo is still one of the best doc I've ever read and I'm sure he inspired many others. IMO It's actually the greatest thing about Gentoo (along with Portage).
- karolist 4y ago
  Running Gentoo on 9900KS and 980 Pro SSD with 64GB of memory, kernel compiles in about 8 minutes, the longest project to build I've seen is Chromium and QT Web Engine at ~120m. It's OK but I'm tempted by the new Ryzen CPUs.
  - Xylakant 4y ago
jdub 4y ago
Points from the flip side: - the Linux project is very likely to stick to simple, fast elements of Rust (based on the excellent approach of the Linux/Rust devs thus far) - the more Rust is used, the more work will be done to improve its performance - you can still build a kernel on a low-powered device... i've built kernels that took > 12 hours on, for example, PA-RISC boxes that were once regarded as beefy :-) - most people don't (and shouldn't) compile their kernel, and by most I mean more than 99%
- unsafecast 4y ago
  All that is fair. It's just, maybe a bit early? I don't usually say this, but I would've appreciated if it came a little later. All I really want is to be able to compile my stuff without waiting overnight (or more).
  - jdub 4y ago
    It almost certainly won't be relevant to you, as a Linux kernel user, for a while anyway. So, it will be a little later.
- craigmart 4y ago
  Could you expand on why people shouldn't compile their kernel? I think it's fairly useful to compile their own to get a better understanding of what the kernel does and to better suit everyone's needs. For example, if I have little free space on my boot partition and I have my disk encrypted, I want my kernel to be as small as possibile, so I will deselect every driver I don't need. Or maybe the driver for my new device is not included in the kernel builds of my distribution. Not only I woundn't say that most people shouldn't compile their kernel, I would say that most linux users* should do it at least once, so they can understand the power they have compared to closed-source operaring systems. *with linux users I mean users that use linux as their main operating system, not people that do ssh once in a while or rarely boots their linux partition
  - froh 4y ago
    using the same binaries shares the verification for said binary. a modular kernel with a custom inird (generated by the distro) is small enough for most. so if you are into adventures or you are in the business of kernel development yes roll your own. anybody else is better served standing on the shoulders of a maintained binary distribution.
EugeneOZ 4y ago
Rust compiler does all the Rust’s magic (where the borrow checker is the biggest part). This magic is really important and helpful, and it's better to do it during compilation, not in runtime (for huge performance benefits).
- unsafecast 4y ago
  I'm aware. I know how rust works and why people want it. I haven't checked the times, but if the borrow checker is really the slowest part, maybe making rust skip it is a valid approach for end users. Sounds like an interesting experiment.
  - nevi-me 4y ago
    There was a PR merged yesterday which should give us 5-10% improvements in compile times [0]. As long as there are people monitoring and working on compile times, one could optimistically hope that as more companies become willing to throw money at Rust, we'll see improvements over time. Regardless of how fast one's machine is, I think having some compile caching infra like sscache should help improve time to compile. [0] https://perf.rust-lang.org/compare.html?start=9be2f35a4c1ed1b04aa4a6945b64763f599259ff&end=1ca6777c014813e3bdb98d155562fc3d111d86dd&stat=instructions:u https://perf.rust-lang.org/compare.html?start=9be2f35a4c1ed1...
  - EugeneOZ 4y ago
    By “biggest” I meant “giving the biggest impact”. I don't know if it's the slowest part.
  - a4a4a4a4 4y ago
    > but if the borrow checker is really the slowest part, maybe making rust skip it is a valid approach for end users. Relevant username?...
    - unsafecast 4y ago
      I mean, if the people that write the code don't skip it, I don't see the issue. The code in the source tree is supposed to be safe, so you might as well not do borrow checking and whatever other safety stuff you have.
bayindirh 4y ago
gccrs is entering the scene with GCC-13. We'll see how it turns out. It'll become part of the GCC toolchain, so everything can use the same pipe.
- choeger 4y ago
  From a bootstrapping perspective, GCC is not perfect, either, because of the C++ addition in 5.0 (at least I think that's the version.) So to bootstrap rust this way, you'd need to go GCC 4.2 -> GCC 13+ -> Rust
  - bayindirh 4y ago
    I meant directly compiling Rust with gccrs, which will be included in gcc toolchain starting gcc13.
keepquestioning 4y ago
Whatever happened to CraneLift?
- nevi-me 4y ago
  It's coming along well, here's an update from 2 weeks ago https://bjorn3.github.io/2022/10/12/progress-report-okt-2022.html https://bjorn3.github.io/2022/10/12/progress-report-okt-2022...
- nextaccountic 4y ago
  I'm using it right now for faster compile times, following instructions from Perseus repo (Perseus is a web framework) https://framesurge.sh/perseus/en-US/docs/next/reference/compilation-times https://framesurge.sh/perseus/en-US/docs/next/reference/comp...
nousermane 4y ago
I'm not a fan of rust either, not by a long mile. But current kernel approach to memory safety is a complete, utter, demonstrated failure. Look at last week's CVE-2022-41674: https://seclists.org/oss-sec/2022/q4/23 https://seclists.org/oss-sec/2022/q4/23 This is a catastrophic bug, that (after some work on developing an actual RCE) lets anybody within wifi range to get root on your laptop (or phone, or access point). And all it took is this one line: https://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless.git/commit/?id=aebe9f4639 https://git.kernel.org/pub/scm/linux/kernel/git/wireless/wir... We've all been repeating the "1000 eyes - all bugs are shallow" mantra for far too long. This one was in the mainline for more than 3 years, and nobody noticed. How many more are lurking there?
- JoeAltmaier 4y ago
  Confused by that bug. Adding 2 to a U8 causing overflow makes it smaller, not larger. It will copy too little? not too much.
  - planede 4y ago
    "The subsequent memcpy" is this: memcpy(pos, mbssid + cpy_len, ((ie + ielen) - (mbssid + cpy_len)));
  - abainbridge 4y ago
    How would Rust fix this? Here's a Rust program that overflows a u8, with no compile error or warning. https://godbolt.org/z/ME3e9KeMe https://godbolt.org/z/ME3e9KeMe You can turn on runtime overflow checking with the Rust compiler, but you can do that with gcc compiling C too. I guess the answer is that you wouldn't do the memcpy like that in idiomatic Rust - you'd use some higher level construct that gives the compiler more chance to catch your errors. Could anyone comment on how this works in a case like this? edit: Can't reply to repliers. Defined overflow doesn't help here. A checked_add() function could be used in the Linux kernel in C, just as easily as in Rust. Forcing checked_add() to be used in the Rust compiler would help, but would also have performance and readability impact, which is presumably why it is still being debated.
gnull 4y ago
It's a fair remark, but your requirement is very niche. There's very few people who will value compilation time over runtime speed, safety, good abstractions, expressiveness.
- pclmulqdq 4y ago
  This ethos played out in the C++ community 5-ish years ago. It turned into a compiler with quadratic behavior, and eventually many people who didn't care at all about compiler performance cared a lot. It turns out that compiler speed makes development faster, keeps people interested in the language, and ultimately allows more iterations before release (which can be better for speed and safety than throwing in a bunch of extra compile steps).
  - gnull 4y ago
    You misread my comment. I didn't say it was useless, I said there are things with higher priority. And well… unlike Rust, C++ sucked at some of those things too, this I think makes them different. Rust offers a better trade-off between compilation time and other parameters, than C++.
    - pclmulqdq 4y ago
      They didn't say it was useless either; they also said it wasn't a high priority. Developer velocity and ergonomics should always be a high priority for compiler writers.
      - gnull 4y ago
        "High" as in "higher than runtime speed, safety, abstractions, plugin system"? Where would you put it if you had to rank compilation speed with other criteria? And let's admit even though Rust compilation speed could use some improvement, it's not terrible.
      - gnull 4y ago
        I re-read it. Yes, they're not saying it's useless, but then what are they objecting to?
sylware 4y ago
It is expected to compile rust with a rust written-compiler (rust fanboys will improve it, probably). Now, I wonder what is the most reasonable option: writting a naive and simple 'c11' compiler or a naive and simple rust compiler. I wonder if somebody has done a "syntax complexity diff" between 'c11' and rust. I know that linux is written in "gcc C", not 'c11'... so... On the other end of the software stack, we have servo, mozilla web engine written in rust. What's up there? Still a drop of rust in a ocean of c++? (SDK included). Because after years, if it is still impossible to run servo without c++ code, this is bad omens for kernel rust.
strangescript 4y ago
Yeah we should probably optimize for the 0.5% niche of people that care. Who needs builtin in memory and thread safety on core server systems so long as I can compile the kernel on my 486. Oh never mind, they are dropping 486 support too soon. Why can't it stay 1996 forever.
jeffparsons 4y ago
I wonder if a solution to your use case could look something like mrustc: https://github.com/thepowersgang/mrustc https://github.com/thepowersgang/mrustc The idea here would be to compile in a way that just assumes everything is correct and either crashes catastrophically or produces invalid output otherwise. But in doing so, it should allow at least slightly faster compilation. You could even remove the need for this "fast and loose" compiler to do any type inference by shipping pre-processed source with all types resolved. However I don't know if this addition would fit your needs if, e.g., your goal is to be able to compile from any given commit rather than only official releases. At very least it could be an interesting experiment to discover what tradeoffs are possible.
Gigachad 4y ago
Seeing this as the top issue for Rust on linux in this thread, I can see the future is bright for Rust and Linux.
- boppo1 4y ago
  Is this sarcasm?
  - Gigachad 4y ago
    No. If the most pressing issue found is that it compiles slower on old computers, then there is an absence of real issues.
pornel 4y ago
BTW, people assume it's slow because of the safety checks, but that's not the case. `cargo check` runs just the checks, and is pretty fast. Majority of the time is spent in LLVM, because rustc throws a ton of code at it to clean up. This is being addressed by MIR optimizations (rustc's built-in optimizer working on higher-level code) to remove costly abstractions before they become a pile of low-level code to eliminate.
oxplot 4y ago
> I'm opposed to this. I want to be able to actually compile my software if I wish so. So what you're saying is that you're opposed to millions of people having more secure software, and perhaps millions of dollars spared from breaches, because it makes your own occasional singular personal experience of compiling the software faster? Or did you mean something else?
- qu4z-2 4y ago
  I mean, slow compile times affect everyone who compiles the kernel. And being able to compile things yourself (possibly with patches) is one of the key features of open source...
ghoward 4y ago
Absolutely, and my own experience is that it's possible to create a language with Rust's safety guarantees while making it quick to compile. I know this because I've managed to add a form of RAII and borrow checking to portable C11, and C is known for being faster than Rust to compile. Imagine what happened if we made a language with that stuff built-in. The funny thing is that C is also slower to compile than it could be because of headers.
llogiq 4y ago
I used to work on the Rust compiler itself on a Chromebook with a 1.1GHz dual core, 4G RAM and 32G of disk. That's about as far from a high-end PC you can get. Most mid-range phones nowadays have more processing power and memory than that. And the Rust compiler has been sped up considerably since then. Even with a 4 year old mid-tier PC you can get a complete Rust compile in half an hour. Roughly half of that is building LLVM. So you can of course compile your Rust compiler. If you are used to compile clang or gcc, it's not that much of a hassle. And the benefits have already been shown. If you only want to compile Rust code, and not develop it, mrustc might also be a good choice for you (it doesn't implement borrowck, just what's needed for codegen). Finally, if you don't want to use Rust drivers, you can simply configure them out and don't need to build Rust. It'll be quite a long while until Rust will arrive in the kernel outside of drivers (which tend to benefit most from Rust anyway).
deleted 4y ago
[deleted]
npigrounet 4y ago
shakow 4y ago
Optimizing language choice for something as fundamental as the kernel for compile speed seems very naive and short-sighted to me. If I had a choice between 50% CVEs and 2h per compile or 5' compiles, I'd take the former in a blink.
sliken 4y ago
Just as a data point, my desktop is a Xeon e3-1230 v5, which is the same silicon as the i7 from 2015. The CPU cost $250 new, the entire desktop was $1100 or so, not including the monitors. I followed the https://rustc-dev-guide.rust-lang.org/building/how-to-build-and-run.html https://rustc-dev-guide.rust-lang.org/building/how-to-build-... Then ran the build step "time ./x.py build -j 8" ... Build completed successfully in 0:36:53 real 36m53.398s user 254m52.720s sys 12m48.289s Seems pretty reasonable considering it's not a particularly high end desktop from 2015. Seems like a cheap price to pay for increased reliability and security.
itripn 4y ago
Considering only compile time is a shallow approach to the idea of using Rust more widely. I would encourage you to think about the aggregate amount of time our industry spends finding, and then fixing, and then repairing the damage done by, classes of bugs which idiomatic Rust completely prevents. It does all this without impacting runtime (like GC language often do). Fast compute at this point is quite literally the least expensive part of the equation. Machines will get faster. Compilers will get optimized. We've spent decades optimizing the developer experience (compile times) at the expense of the rigor, robustness, and quality of our resulting product. I've been doing this for 30 years, and I can categorically say that I've spent FAR more time chasing NPE, OBO, and race condition bugs than I would have ever added to my build time with a slightly slower compiler. itripn&

_ph_ 4y ago

While I lack personal experience with Rust and I really appreciate fast compilers (that is why I am a Go user), over all features and characteristics, Rust seems to be the best choice for safe kernel development. Other posters have described well how urgent it is, to improve the security of kernel code. So just not doing anything about this, doesn't seem to be a good option. It seems, there is a wide group of developers which thinks that Rust is the best candidate as a kernel development language. If you see issues with that choice, now would be the time to propose an alternative and try to find momentum in the developer community supporting that alternative. While I also lack practical experience there, by all what I heard, ADA could be one. But I don't know how it exactly compares to Rust and what the trade offs are. But so far, no one has pushed for ADA as a possible kernel implementation language.

cube00 4y ago

I'm not sure it's realistic to expect a safety focused compiler to compete with one that doesn't offer those checks. We should aspire to make it as fast but in the short term a slower compiler in exchange for less CVEs and random buffer overrun crashes seems like a reasonable trade off to me. Distributions such as Fedora offer build infrastructure[1] that you can use to compile packages to use in your system for testing if you feel your local hardware isn't powerful enough. [1]: https://copr.fedorainfracloud.org https://copr.fedorainfracloud.org

raverbashing 4y ago
I think it's more related to the grammatical complexity of the language than to safety itself (Same reason why C with typedefs is slower to compile than plain C, why C++ is slower, etc) (that and cargo dependencies, etc - also C compilers have some +30yrs of optimizations)
- akira2501 4y ago
  I've seen some Rust programs that look nice, but in the kernel, the code ends up making perl'ls propensity for line noise look competitive. I'm sure some of that is the need to shovel between languages, but it does make you wonder if the entire effort is a little misguided. A full Rust kernel might be neat. A compromised Rust driver inside of a C kernel is never going to be the default choice.
  - bonzini 4y ago
    Most of the code committed to the kernel is infrastructure for C<->Rust interoperability. It's the least readable of it all, and as fun to work on as watching paint dry. The actual driver code is pretty readable, see https://lwn.net/Articles/863459/ https://lwn.net/Articles/863459/.
  - coldacid 4y ago
    >A full Rust kernel might be neat. May I interest you in Redox? https://www.redox-os.org/ https://www.redox-os.org/
  - steveklabnik 4y ago
    If "embedded OS" counts as "full Rust kernel", you may want to check out https://hubris.oxide.computer/ https://hubris.oxide.computer/ Obviously very different than Linux.
- WastingMyTime89 4y ago
  More complexe languages are faster to compile. The issue is mostly the implementation. It could, should and will be better but it takes time.
- api 4y ago
  As others have pointed out most of the performance hit is in the optimization of the relatively verbose IR code the Rust compiler sends to LLVM, not the Rust compiler itself. There's been multiple passes at making the Rust compiler faster. It'll happen. There's a GCC-based Rust implementation in the works too.
goodpoint 4y ago
Nim is also memory safe and the compilation is really fast.
- codewithcheese 4y ago
  > No, Nim doesn't have stronger or weaker safety guarantees than Rust [1]. Rust's memory safety is nothing new, either. It is mostly that some older languages like C/C++ are the exceptions in not being memory-safe. There is nothing new or magical about memory safety. LISP was already memory-safe when it was invented in 1958. The only question is how much performance you need to trade away for it (the value is never zero for non-trivial programs, but can vary greatly, depending on whether the language was designed with it in mind or not). > The main difference between Rust and other languages is that it does some more (but not all [2]) safety checks at compile time rather than at runtime. It also allows you to avoid GC, but does not provide you any memory safety over GC. Rust's borrow checker allows you to statically prove that references are live [3]; a GC simply avoids deallocating any memory that has a live reference to it (on the other hand, a GC can ensure that references remain live even where this is hard or impossible to prove statically). The end result is the same with respect to memory safety (the reason some people want to avoid GC is for performance reasons, not memory safety). https://forum.nim-lang.org/t/1961 https://forum.nim-lang.org/t/1961
  - goodpoint 4y ago
    That's outdated information from 2016. Now Nim uses ARC/ORC. ARC implements memory management at compile time like Rust and it's still very fast at compiling.
    - codewithcheese 4y ago
      Cool, I wasn't aware, thanks. https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc-in-nim.html https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc... > The main difference between ARC and Nim GCs is that ARC is fully deterministic - the compiler automatically injects destructors when it deems that some variable (a string, sequence, reference, or something else) is no longer needed. In this sense, it’s similar to C++ with its destructors (RAII). To illustrate, we can use Nim’s expandArc introspection (will be available in Nim 1.4). > This shows one of the main ARC features: scope-based memory management. A scope is a separate region of code in the program. Scope-based MM means that the compiler will automatically insert destructor calls for any variables which need a destructor after the scope ends. Many Nim constructs introduce new scopes: procs, funcs, converters, methods, block statements and expressions, for and while loops, etc. > ARC also has so-called hooks - special procedures that can be defined for types to override the default compiler behaviour when destroying/moving/copying the variable. These are particularly useful when you want to make custom semantics for your types, deal with low-level operations involving pointers, or do FFI.
rob74 4y ago
Actually C/C++ compilers are shooting themselves in the foot performance-wise with the header mechanism, so writing a compiler that's faster than C/C++ is not that hard. Not sure what the reasons of the Rust compiler being (even more?) slow are, maybe the ability to easily interface with C code has something to do with it?
- nine_k 4y ago
  Last time I checked [1], the Rust part of the Rust compiler is not the slowest. LLVM spends a lot of time processing relatively poor IR code which the Rust compiler emits. Rust compilation is not intrinsically slow; better IR and more incremental compilation can and will improve it. Incremental compilation (only changed files, not crates) is 7 years in making though. [1]: https://prev.rust-lang.org/en-US/faq.html#why-is-rustc-slow https://prev.rust-lang.org/en-US/faq.html#why-is-rustc-slow
  - rob74 4y ago
    Thanks for that link! That sounds like the really major factor is the decision to recompile the entire crate when a file in the crate changes. I'm sure that made sense at the time ;) And, if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision - incremental compilation might have its edge cases and pitfalls, but other compilers are doing it just fine, so it shouldn't be rocket science? Other reasons: - suboptimal LLVM IR code, as you mentioned, and other tech debt - the preferred strategy of monomorphising generics is fast at runtime, but slow to compile, using trait objects is faster - the complex type system of course also plays a part - LLVM itself is of course not optimized for Rust
    - rbanffy 4y ago
      > if slow compilation is such a pain point, I wonder why it takes seven years and counting to fix this bad decision Maybe we are giving Rust developers computers that are too fast. ;-) OTOH, it's always wise to design for the future, so it also makes sense to give Rust developers beefy server grade hardware so they can play with SIMD pipelines and all those extra cores and threads, because that's what workstations will be a couple years from now.
pjmlp 4y ago
There are languages with similar type safety, which are faster, because the authors have placed focus on having several ways available to compile the code. Ada, Delphi, OCaml, C#/F# (.NET Native / Native AOT), D, Nim,...
- 0dayz 4y ago
  I will admit I never understood why Ada never took off (is it only because of the tooling being propertiary for a long time?), but c# is not as memory safe as rust.
  - pjmlp 4y ago
    The only memory safety that C# lacks in comparisaion with Rust is one special case, data races for in-process data. For everything else regarding concurrent access to shared data out of process, they are on the same foot. Ada didn't took off, because of several reasons, price of compilers, on UNIX it was an additional SKU on top of the respective developers SDK that already offered C and C++ in the box, 1980's mainstream hardware wasn't able to cope with it, most OS vendors outside UNIX decide to migrate from their toolchains into C and C++, so again additional money on top of the OS SDK.
  - thesuperbigfrog 4y ago
    >> I will admit I never understood why Ada never took off (is it only because of the tooling being propertiary for a long time?) Early Ada compilers had expensive licensing and required expensive hardware to run. By the time GNAT was added to GCC, C++ had already taken over most of the spaces that were not Ada exclusive (meaning safety critical / defense / aerospace niches where either Ada was once mandated or has thrived in despite the original high costs).
- mejutoco 4y ago
  Does C# not have a GC? I thought that would immediately disqualify it for kernel work.
  - theta_d 4y ago
VikingCoder 4y ago
What's the old adage? It's easier to make a correct thing fast than it is to make a fast thing correct.

Xylakant 4y ago

I remember the times when installing Gentoo was a matter of days. Compiling the kernel was a matter of hours. Sure, rustc is slower than gcc, but if you're not constantly compiling your software over and over and over again, then the time spent in the compiler is not your primary concern. Start the compile, go to bed, let it run.

akira2501 4y ago
Gentoo.. the only reason I'd ever know or need to know that 'distcc' is a thing that exists.
- srazzaque 4y ago
  Ha! I was a wimp and did a "stage 3" install back in the day. Or was it stage 2? (shrug). Now I'm even wimpier and just use Fedora. Came across this page, on the motivations of Gentoo, a few years later: > Installing a working Linux box used to require over 550 man hours, learning a Nordic language, sacrificing a goat, wading through hundreds of pages of (purposely) inscrutable help files...Old-school Linux users were desperate to find a new way to feel superior. https://en.uncyclopedia.co/wiki/Gentoo https://en.uncyclopedia.co/wiki/Gentoo
  - speed_spread 4y ago
    It honestly wasn't bad at all. Daniel Robbins documentation for Gentoo is still one of the best doc I've ever read and I'm sure he inspired many others. IMO It's actually the greatest thing about Gentoo (along with Portage).
karolist 4y ago
Running Gentoo on 9900KS and 980 Pro SSD with 64GB of memory, kernel compiles in about 8 minutes, the longest project to build I've seen is Chromium and QT Web Engine at ~120m. It's OK but I'm tempted by the new Ryzen CPUs.
- Xylakant 4y ago

jdub 4y ago

Points from the flip side: - the Linux project is very likely to stick to simple, fast elements of Rust (based on the excellent approach of the Linux/Rust devs thus far) - the more Rust is used, the more work will be done to improve its performance - you can still build a kernel on a low-powered device... i've built kernels that took > 12 hours on, for example, PA-RISC boxes that were once regarded as beefy :-) - most people don't (and shouldn't) compile their kernel, and by most I mean more than 99%

unsafecast 4y ago
All that is fair. It's just, maybe a bit early? I don't usually say this, but I would've appreciated if it came a little later. All I really want is to be able to compile my stuff without waiting overnight (or more).
- jdub 4y ago
  It almost certainly won't be relevant to you, as a Linux kernel user, for a while anyway. So, it will be a little later.
craigmart 4y ago
Could you expand on why people shouldn't compile their kernel? I think it's fairly useful to compile their own to get a better understanding of what the kernel does and to better suit everyone's needs. For example, if I have little free space on my boot partition and I have my disk encrypted, I want my kernel to be as small as possibile, so I will deselect every driver I don't need. Or maybe the driver for my new device is not included in the kernel builds of my distribution. Not only I woundn't say that most people shouldn't compile their kernel, I would say that most linux users* should do it at least once, so they can understand the power they have compared to closed-source operaring systems. *with linux users I mean users that use linux as their main operating system, not people that do ssh once in a while or rarely boots their linux partition
- froh 4y ago
  using the same binaries shares the verification for said binary. a modular kernel with a custom inird (generated by the distro) is small enough for most. so if you are into adventures or you are in the business of kernel development yes roll your own. anybody else is better served standing on the shoulders of a maintained binary distribution.

EugeneOZ 4y ago

Rust compiler does all the Rust’s magic (where the borrow checker is the biggest part). This magic is really important and helpful, and it's better to do it during compilation, not in runtime (for huge performance benefits).

unsafecast 4y ago
I'm aware. I know how rust works and why people want it. I haven't checked the times, but if the borrow checker is really the slowest part, maybe making rust skip it is a valid approach for end users. Sounds like an interesting experiment.
- nevi-me 4y ago
  There was a PR merged yesterday which should give us 5-10% improvements in compile times [0]. As long as there are people monitoring and working on compile times, one could optimistically hope that as more companies become willing to throw money at Rust, we'll see improvements over time. Regardless of how fast one's machine is, I think having some compile caching infra like sscache should help improve time to compile. [0] https://perf.rust-lang.org/compare.html?start=9be2f35a4c1ed1b04aa4a6945b64763f599259ff&end=1ca6777c014813e3bdb98d155562fc3d111d86dd&stat=instructions:u https://perf.rust-lang.org/compare.html?start=9be2f35a4c1ed1...
- EugeneOZ 4y ago
  By “biggest” I meant “giving the biggest impact”. I don't know if it's the slowest part.
- a4a4a4a4 4y ago
  > but if the borrow checker is really the slowest part, maybe making rust skip it is a valid approach for end users. Relevant username?...
  - unsafecast 4y ago
    I mean, if the people that write the code don't skip it, I don't see the issue. The code in the source tree is supposed to be safe, so you might as well not do borrow checking and whatever other safety stuff you have.

bayindirh 4y ago

gccrs is entering the scene with GCC-13. We'll see how it turns out. It'll become part of the GCC toolchain, so everything can use the same pipe.

choeger 4y ago
From a bootstrapping perspective, GCC is not perfect, either, because of the C++ addition in 5.0 (at least I think that's the version.) So to bootstrap rust this way, you'd need to go GCC 4.2 -> GCC 13+ -> Rust
- bayindirh 4y ago
  I meant directly compiling Rust with gccrs, which will be included in gcc toolchain starting gcc13.

keepquestioning 4y ago

Whatever happened to CraneLift?

nevi-me 4y ago
It's coming along well, here's an update from 2 weeks ago https://bjorn3.github.io/2022/10/12/progress-report-okt-2022.html https://bjorn3.github.io/2022/10/12/progress-report-okt-2022...
nextaccountic 4y ago
I'm using it right now for faster compile times, following instructions from Perseus repo (Perseus is a web framework) https://framesurge.sh/perseus/en-US/docs/next/reference/compilation-times https://framesurge.sh/perseus/en-US/docs/next/reference/comp...

nousermane 4y ago

I'm not a fan of rust either, not by a long mile. But current kernel approach to memory safety is a complete, utter, demonstrated failure. Look at last week's CVE-2022-41674: https://seclists.org/oss-sec/2022/q4/23 https://seclists.org/oss-sec/2022/q4/23 This is a catastrophic bug, that (after some work on developing an actual RCE) lets anybody within wifi range to get root on your laptop (or phone, or access point). And all it took is this one line: https://git.kernel.org/pub/scm/linux/kernel/git/wireless/wireless.git/commit/?id=aebe9f4639 https://git.kernel.org/pub/scm/linux/kernel/git/wireless/wir... We've all been repeating the "1000 eyes - all bugs are shallow" mantra for far too long. This one was in the mainline for more than 3 years, and nobody noticed. How many more are lurking there?

JoeAltmaier 4y ago
Confused by that bug. Adding 2 to a U8 causing overflow makes it smaller, not larger. It will copy too little? not too much.
- planede 4y ago
  "The subsequent memcpy" is this: memcpy(pos, mbssid + cpy_len, ((ie + ielen) - (mbssid + cpy_len)));
- abainbridge 4y ago
  How would Rust fix this? Here's a Rust program that overflows a u8, with no compile error or warning. https://godbolt.org/z/ME3e9KeMe https://godbolt.org/z/ME3e9KeMe You can turn on runtime overflow checking with the Rust compiler, but you can do that with gcc compiling C too. I guess the answer is that you wouldn't do the memcpy like that in idiomatic Rust - you'd use some higher level construct that gives the compiler more chance to catch your errors. Could anyone comment on how this works in a case like this? edit: Can't reply to repliers. Defined overflow doesn't help here. A checked_add() function could be used in the Linux kernel in C, just as easily as in Rust. Forcing checked_add() to be used in the Rust compiler would help, but would also have performance and readability impact, which is presumably why it is still being debated.

gnull 4y ago

It's a fair remark, but your requirement is very niche. There's very few people who will value compilation time over runtime speed, safety, good abstractions, expressiveness.

pclmulqdq 4y ago
This ethos played out in the C++ community 5-ish years ago. It turned into a compiler with quadratic behavior, and eventually many people who didn't care at all about compiler performance cared a lot. It turns out that compiler speed makes development faster, keeps people interested in the language, and ultimately allows more iterations before release (which can be better for speed and safety than throwing in a bunch of extra compile steps).
- gnull 4y ago
  You misread my comment. I didn't say it was useless, I said there are things with higher priority. And well… unlike Rust, C++ sucked at some of those things too, this I think makes them different. Rust offers a better trade-off between compilation time and other parameters, than C++.
  - pclmulqdq 4y ago
    They didn't say it was useless either; they also said it wasn't a high priority. Developer velocity and ergonomics should always be a high priority for compiler writers.
    - gnull 4y ago
      "High" as in "higher than runtime speed, safety, abstractions, plugin system"? Where would you put it if you had to rank compilation speed with other criteria? And let's admit even though Rust compilation speed could use some improvement, it's not terrible.
    - gnull 4y ago
      I re-read it. Yes, they're not saying it's useless, but then what are they objecting to?

sylware 4y ago

It is expected to compile rust with a rust written-compiler (rust fanboys will improve it, probably). Now, I wonder what is the most reasonable option: writting a naive and simple 'c11' compiler or a naive and simple rust compiler. I wonder if somebody has done a "syntax complexity diff" between 'c11' and rust. I know that linux is written in "gcc C", not 'c11'... so... On the other end of the software stack, we have servo, mozilla web engine written in rust. What's up there? Still a drop of rust in a ocean of c++? (SDK included). Because after years, if it is still impossible to run servo without c++ code, this is bad omens for kernel rust.

strangescript 4y ago

Yeah we should probably optimize for the 0.5% niche of people that care. Who needs builtin in memory and thread safety on core server systems so long as I can compile the kernel on my 486. Oh never mind, they are dropping 486 support too soon. Why can't it stay 1996 forever.

jeffparsons 4y ago

I wonder if a solution to your use case could look something like mrustc: https://github.com/thepowersgang/mrustc https://github.com/thepowersgang/mrustc The idea here would be to compile in a way that just assumes everything is correct and either crashes catastrophically or produces invalid output otherwise. But in doing so, it should allow at least slightly faster compilation. You could even remove the need for this "fast and loose" compiler to do any type inference by shipping pre-processed source with all types resolved. However I don't know if this addition would fit your needs if, e.g., your goal is to be able to compile from any given commit rather than only official releases. At very least it could be an interesting experiment to discover what tradeoffs are possible.

Gigachad 4y ago

Seeing this as the top issue for Rust on linux in this thread, I can see the future is bright for Rust and Linux.

boppo1 4y ago
Is this sarcasm?
- Gigachad 4y ago
  No. If the most pressing issue found is that it compiles slower on old computers, then there is an absence of real issues.

pornel 4y ago

BTW, people assume it's slow because of the safety checks, but that's not the case. `cargo check` runs just the checks, and is pretty fast. Majority of the time is spent in LLVM, because rustc throws a ton of code at it to clean up. This is being addressed by MIR optimizations (rustc's built-in optimizer working on higher-level code) to remove costly abstractions before they become a pile of low-level code to eliminate.

oxplot 4y ago

> I'm opposed to this. I want to be able to actually compile my software if I wish so. So what you're saying is that you're opposed to millions of people having more secure software, and perhaps millions of dollars spared from breaches, because it makes your own occasional singular personal experience of compiling the software faster? Or did you mean something else?

qu4z-2 4y ago
I mean, slow compile times affect everyone who compiles the kernel. And being able to compile things yourself (possibly with patches) is one of the key features of open source...

ghoward 4y ago

Absolutely, and my own experience is that it's possible to create a language with Rust's safety guarantees while making it quick to compile. I know this because I've managed to add a form of RAII and borrow checking to portable C11, and C is known for being faster than Rust to compile. Imagine what happened if we made a language with that stuff built-in. The funny thing is that C is also slower to compile than it could be because of headers.

llogiq 4y ago

I used to work on the Rust compiler itself on a Chromebook with a 1.1GHz dual core, 4G RAM and 32G of disk. That's about as far from a high-end PC you can get. Most mid-range phones nowadays have more processing power and memory than that. And the Rust compiler has been sped up considerably since then. Even with a 4 year old mid-tier PC you can get a complete Rust compile in half an hour. Roughly half of that is building LLVM. So you can of course compile your Rust compiler. If you are used to compile clang or gcc, it's not that much of a hassle. And the benefits have already been shown. If you only want to compile Rust code, and not develop it, mrustc might also be a good choice for you (it doesn't implement borrowck, just what's needed for codegen). Finally, if you don't want to use Rust drivers, you can simply configure them out and don't need to build Rust. It'll be quite a long while until Rust will arrive in the kernel outside of drivers (which tend to benefit most from Rust anyway).

deleted 4y ago

[deleted]

npigrounet 4y ago

shakow 4y ago

Optimizing language choice for something as fundamental as the kernel for compile speed seems very naive and short-sighted to me. If I had a choice between 50% CVEs and 2h per compile or 5' compiles, I'd take the former in a blink.

sliken 4y ago

Just as a data point, my desktop is a Xeon e3-1230 v5, which is the same silicon as the i7 from 2015. The CPU cost $250 new, the entire desktop was $1100 or so, not including the monitors. I followed the https://rustc-dev-guide.rust-lang.org/building/how-to-build-and-run.html https://rustc-dev-guide.rust-lang.org/building/how-to-build-... Then ran the build step "time ./x.py build -j 8" ... Build completed successfully in 0:36:53 real 36m53.398s user 254m52.720s sys 12m48.289s Seems pretty reasonable considering it's not a particularly high end desktop from 2015. Seems like a cheap price to pay for increased reliability and security.

itripn 4y ago

Considering only compile time is a shallow approach to the idea of using Rust more widely. I would encourage you to think about the aggregate amount of time our industry spends finding, and then fixing, and then repairing the damage done by, classes of bugs which idiomatic Rust completely prevents. It does all this without impacting runtime (like GC language often do). Fast compute at this point is quite literally the least expensive part of the equation. Machines will get faster. Compilers will get optimized. We've spent decades optimizing the developer experience (compile times) at the expense of the rigor, robustness, and quality of our resulting product. I've been doing this for 30 years, and I can categorically say that I've spent FAR more time chasing NPE, OBO, and race condition bugs than I would have ever added to my build time with a slightly slower compiler. itripn&