27 ms·

Supporting Linux kernel development in Rust

648 points by dochtman 6y ago

boogies 6y ago
Probably a crazy unpopular opinion, but I kind of agree with the Hyperbola (GNU / formerly Linux-libre, in future using a fork of the OpenBSD kernel https://itsfoss.com/hyperbola-linux-bsd/ https://itsfoss.com/hyperbola-linux-bsd/) devs that Rust in Linux is not necessarily a good thing, for the reasons they mentioned when they announced they were switching kernels (https://www.hyperbola.info/news/announcing-hyperbolabsd-roadmap/ https://www.hyperbola.info/news/announcing-hyperbolabsd-road...) (Edit: and the LLVM dependency).
- TylerE 6y ago
  I don’t see how that conplaint is at all meaningful to kernel development. It’s typical GNU/Pendantry. The kernel will not be shipping a rust compiler, patched or otherwise.
  - boogies 6y ago
    The article mentions possible ABI compatibility challenges and I also think it’s not a good thing for a language in such critical applications to only have a single major implementation.
    - eptcyka 6y ago
      The linux kernel still can only be compiled with GCC. Of course, it's a single patch away from clang compatibility, but the kernel has never been compilable with much else but GCC.
      - pjmlp 6y ago
        Android and ChromeOS beg to differ, as Google has been using clang with them for at least around three years now.
        eptcyka 6y ago
        Yes, clang compatibility is a single patch away. Google, unlike the vast majority of companies out there, is capable of maintaining it's own forks of the kernel.
- jcranmer 6y ago
  If I'm understanding correctly, the only reason for not using Rust is... Rust is trademarked?
  - dijit 6y ago
    I'm generally in support of rust in the kernel, but in the interest of playing devils advocate: Would adding more and more dependence on rust mean that it ties the kernel to a single set of compilers? I can see why people would be opposed to that. I'm thinking about LLVM (and, obviously, rustc)
    - Silhouette 6y ago
      Would adding more and more dependence on rust mean that it ties the kernel to a single set of compilers? Like GCC? Your point is certainly valid, but I'm not sure it's a new problem.
    - JoshTriplett 6y ago
      Linux currently only compiles on GCC and Clang, and the latter is a recent development. rustc uses LLVM as a backend, just like Clang, and people are working on other backends. Linux maintainers didn't express any concerns about this being an issue.
    - danudey 6y ago
      The Linux kernel would only compile with GCC for... decades? I think? So it's not really a new thing. It would be really nice if the kernel did have first-class support for LLVM and Clang directly (rather than having to set a bunch of environment variables to set the compiler, the linker to use with that compiler, various variables, etc. That said, it would certainly be uncomfortable to move from "it compiles with Clang, though it's a pain, or GCC, which works fine" to "it compiles with GCC, if you don't use any of this functionality, or LLVM only, if you want any of these features or anything that depends on them".
      - JoshTriplett 6y ago
        Originally, we thought that Linux maintainers were asking us to only support kernels compiled with Clang, because they didn't want to support code built with two different compilers. However, in the session, Greg Kroah-Hartman specifically said that if it works to build Rust code with the LLVM-based rustc and C code with GCC and link the two together, and there aren't any problems in practice, then that's perfectly fine. So, we're currently expecting that supporting Rust will not place any requirements on the C compiler you use, unless you want to use cross-language LTO (which we'll eventually want to do, but it won't be a hard requirement for a working kernel).
- badrabbit 6y ago
  Maybe a GNU Rust compiler without the trademark is in order? The state of KSPP is not nice as well. I wish Ada was more popular. From what I know, it has much of the security guarantees rust has and it has been battle tested.
  - guerrilla 6y ago
    I'm not sure, but aren't they saying that they wouldn't be able to call it "GNU Rust" [1] in the same way they can't call IceCat "GNU Firefox"? 1. https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedom_flaws https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedo...
    - homarp 6y ago
      well, plenty of names to play with:) GNR: Gnr Not Rust TRust: Treated Rust ...
      - gpm 6y ago
        How do you pronounce GNR? It think it needs to be a vowel, or an R. Some attempts: GRR: Gnu Renames Rust, GAR: Gnu assimilates rust, GER: Gnu non Est Rust (gnu is not rust, but in latin to get an E), GIR: Gnu Isn't Rust, GOR: Gnu Oxidizes Rust, GUR: Gnu Unseats Rust.
        homarp 6y ago
        pronounces it Gnar or Gener, depending if you are a gif or a jif person. You can also go for GunsNRoses, but they won't like it and complain. Which might be good for visibility at the beginning.
      - el_oni 6y ago
        Or something like "oxide", ferrite, maGNUtite. Play with iron ores or compounds
        6y ago
- JoshTriplett 6y ago
  The vast majority of the text in that post is inaccurate and hyperbolic. Linux is not "forcing adoption of HDCP" (it's just something Linux has a driver for), Rust does not require internet access to use (and the trademark comments are not particularly accurate either), the Kernel Self Protection Project is alive and well and making new improvements in every new version of the kernel (see Kees Cook's regular blog posts for details), systemd is not spelled "SystemD", gettext has no dependency on Java (and the link in that post explains that). And in general, decisions about what technologies to use are made by those who show up and do the work to build viable solutions, not by people who snark. It's a rant by an obscure one-developer distribution that formerly used the Linux kernel. Some people are never happy no matter what you do, and if you make the mistake of treating their rants as useful feedback, you get a list of 30 more demands before they'd consider gracing your little project with the honor of their all-important usage again.
  - ColanR 6y ago
    I think you were being "inaccurate and hyperbolic". > Linux is not "forcing adoption of HDCP" (it's just something Linux has a driver for) What they actually said in the article: > Historically, some features began as optional ones until they reached total functionality. Then they became forced and difficult to patch out. Even if this does not happen in the case of HDCP, we remain cautious about such implementations.
    - JoshTriplett 6y ago
      My comment was entirely based on the second link. Direct quote from that: > Linux kernel forcing adaption of DRM, including HDCP. (Which linked to a patch adding driver support for handling HDCP, with absolutely no possibility of it somehow being forced.) Regarding the quote from that interview: sure, and there's also no guarantee that the Linux kernel won't drop support for every architecture except SPARC, except of course for all the people involved not putting up with it. In what possible universe would Linux developers decide it was a good idea to mandate HDCP? It's completely unfounded and unsupported hyperbole and fearmongering, without even a hint of potential truth. It's also roughly consistent with what I'd expect from someone who thinks that gettext daring to have support for Java and C# format strings is a horrific problem that should be ripped out by the roots.
- Macha 6y ago
  The licensing complaints about Rust apply equally to Python, which they package without objection: https://www.hyperbola.info/packages/extra/x86_64/python/ https://www.hyperbola.info/packages/extra/x86_64/python/
  - cylon13 6y ago
    Not exactly: [0] > Some users have correctly mentioned that many other software packages have trademarks, do we plan to remove them all? No. We are not against all trademarks, only those which explicitly prohibit normal use, patching, and modification. > As an example, neither Python PSF nor Perl Trademarks currently prohibit patching the code without prior approval. They do prohibit abuse of their trademarks, e.g. you cannot create a company called “Python”, but this does not affect your ability to modify their free software and/or apply patches. > Due to the anti-modification clause, Rust is a non-permissive trademark that violates user freedom. [0] https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedom_flaws https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedo...
    - Macha 6y ago
      Well, good luck calling your modified Python release Python: https://github.com/naftaliharris/tauthon/issues/47 https://github.com/naftaliharris/tauthon/issues/47
    - jcranmer 6y ago
      I suspect whomever wrote that hasn't actually read the trademark policies in detail, or conferred with lawyers about it. Neither Python's nor Perl's agreements explicitly cover the case in detail, but Perl does mention this: > Licensed redistributors of Perl code are permitted by their licenses to use the Perl logo in connection with their distribution services, on product packaging, and in promotional materials. IANAL, but that text would especially suggest to me that you would have no right to use the trademark even to distribute an unmodified binary of Perl without approval. I suspect the real reason is Debian and Mozilla had a very public, well-known spat over licensing, and Python and Perl have not had very public, well-known spats with anybody.
- ArchOversight 6y ago
  Once again the GPL camp takes and relicenses a project so that further improvements they make can't be used by the original authors. Shame really.
  - boogies 6y ago
    They can be used by the original authors, if the original authors stop intentionally licensing their code so weakly that anyone can make changes that no one else can even see.
  - JetSetWilly 6y ago
    If they didn't want people to be able to make changes that cannot be used upstream, there's a license for that. It's called the GPL.
- bitwize 6y ago
  I'd be more concerned about the fact that the Rust Evangelism Strike Force is hellbent on getting everybody to depend on their exhausts-all-32-bit-address-space-to-build compiler and toolchain -- in addition to whatever toolchain the project currently uses, at least as long as the strangler pattern is being applied to migrate the code base to 100% Rust. GNU software in particular has always been such that the core stuff is bootstrappable from just C. But it seems, from an RESF standpoint, that the fact that any C code is out there running at all is a crisis-level problem.
  - JoshTriplett 6y ago
    Standing reminder: the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. We discourage that kind of thing wherever we see it. We're much more in favor of a measured, cautious approach.
Thaxll 6y ago
Looks like there is a lot of work involved to have anything done, remind me the issues that the Chrome team said 2 weeks ago.
- stmw 6y ago
  Can you elaborate re: issues and lots of work required? I think it's generally to be expected that non-C language support in the kernel would be hard...
  - muricula 6y ago
    Here's the article the parent mentioned in passing, it may answer your question: https://www.chromium.org/Home/chromium-security/memory-safety/rust-and-c-interoperability https://www.chromium.org/Home/chromium-security/memory-safet... It's very Chrome and C++ specific, but the problems will likely be similar in practice.
    - stmw 6y ago
      Ah, right, I'd read that - but it's for C++, so it's a bit different than the kernel situation, and I actually had a much more positive reaction to it ("wow, seems like C++<->Rust is actually pretty far along"). Guess both are reasonable takeaways.
- gpm 6y ago
  There is no doubt a lot of work here, but C <-> Rust interoperability is a lot cleaner than C++ <-> Rust. And for what it's worth, for almost all langauges C <-> X interoperability is a lot cleaner than C++ <-> X. Since almost all languages (including rust) "speak C", both in terms of compiler support and and being able to map every important C concept to a concept in their language. The same isn't true with C++. One thing to worry about with adding any language (including rust) to the kernel, is that the next good looking language will probably interoperate well with C and not interoperate well with Rust. So there is a much higher cost the second time you try to add a language.
  - 6y ago
puzzledobserver 6y ago
One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers and the features the existing kernel exports to userland in the form of syscalls. Would it be possible for a hypothetical new kernel, presumably written in Rust, to run an existing kernel such as Linux in a VM, just to tap into its drivers and emulate its syscalls? As more drivers are ported to the base kernel, the reliance on the donor kernel would shrink over time. Edit: This was an aside. Of course I realize that the conference was about adding Rust code to the Linux kernel, and not about rewriting any large, important, and functional codebase in language-of-the-day™. Hence my speculative language: "hypothetical" new kernel, "presumably" written in Rust.
- cmckn 6y ago
  This conference talk considered a re-write out-of-scope; they were only discussing how new code could be written in Rust. Interesting idea, though!
- ColanR 6y ago
  I was thinking about something similar recently - if the hardware drivers could somehow be abstracted from the rest of the linux kernel (as if...), then suddenly a ton of experimental kernels could have widespread hardware availability. It would probably shake up the OS ecosystem to no small degree, but I imagine the fresh ideas that could be experimented with by non-experts would be hugely beneficial. Isn't that the big argument against so many new projects? "It looks cool, but the hardware support isn't there." FreeBSD might be a better target though, since that's specifically designed to be compatible with everything.
  - alex7o 6y ago
    This exists and is called rumpkerbels if I remember correctly.
    - puzzledobserver 6y ago
      rumpkerbels? Mind sharing a link? Google Search is curiously empty.
ghostwriter 6y ago
I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects, that it isn't. The safety guarantees that Rust provides are neither unique nor complete, and if we discuss the amount of effort necessary for bringing new interfaces like the one this article mentions ("one can define a kmalloc_for_rust() symbol containing an un-inlined version"), we should compare it to other existing solutions, like ATS [1], that are designed to be seamlessly interoperable with C codebases without giving up on the safety side of the argument. Just a week ago there was a link in ATS reddit channel that advertised ATS Linux [2][3] initiative, that may be of great interest for the same people who are interested in bringing more memory sefety to Kernel development, without giving up on existing C interfaces and the toolchain[4]. [1] http://www.ats-lang.org/Documents.html#INT2PROGINATS http://www.ats-lang.org/Documents.html#INT2PROGINATS [2] https://www.reddit.com/r/ATS/comments/ibyczp/ats_linux/ https://www.reddit.com/r/ATS/comments/ibyczp/ats_linux/ [3] http://git.bejocama.org http://git.bejocama.org [4] http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/c2016.html http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...
- gpm 6y ago
  Your evidence (that there are potentially other alternatives) doesn't support your claim (that Rust isn't a great C replacement). There can be multiple great C replacements. Not being "perfect" in terms of the guarantees you provide doesn't mean you aren't a great improvement. Nor would not providing any guarantees at all mean that a language is necessarily not a great improvement. That said, I would be very interested in seeing a comparison between C, Rust, and ATS in practical code. This is the first I'm hearing of ATS and it sounds interesting.
  - ghostwriter 6y ago
    > Your evidence (that there are potentially other alternatives) doesn't support your claim (that Rust isn't a great C replacement). I didn't claim that Rust isn't a great replacement overall, I was specifically mentioning that it's not great for existing C codebases, by the fact that it requires significant work to replace internal interfaces that may be important for performance-, backward-compatibility- and conventional tooling reasons. It doesn't interoperate with C as well as other alternatives are capable of, whilst not being objectively better at safety guarantees either.
The_rationalist 6y ago
It's not about safety, if that was the real argument we should start with obvious improvements like checked C https://github.com/microsoft/checkedc https://github.com/microsoft/checkedc The C++ lifetime checker or Cyclone
- geofft 6y ago
  Checked C addresses a small number of memory safety problems - it addresses bounds checking / buffer overflows, but not use-after-frees, double frees, data races / insufficient locking, and several other things that fall into the general category of "memory safety," nor more general "safety" things like type confusion. The kernel is not written in C++ and is unlikely to use C++, for many reasons - Rust is a more natural C-but-better for the kernel than C is. Cyclone hasn't been maintained for over a decade; it was a research language, and it largely served its purpose as a research language in inspiring production-supported languages like Rust.
chromedev 6y ago
Pretty soon they'll be arguing for inclusion of JavaScript in the Linux kernel
cbmuser 6y ago
If we get the Rust backend for gcc finalized and merged first, it would be much easier to get Rust code into the kernel: > https://github.com/philberty/gccrs/ https://github.com/philberty/gccrs/
- gok 6y ago
  That's a front end?
  - Spivak 6y ago
    Compile C to Rust! Easiest language migration ever.
    - fanf2 6y ago
      https://c2rust.com/ https://c2rust.com/
    - zelly 6y ago
      It shares the code generator with gcc. It doesn't transpile C to Rust or vice versa. The goal is to produce machine code directly.
    - fractalb 6y ago
      That would eliminate the need for Rust. Just imagine, if your C code can be converted into Rust, would there ever be a case to start a fresh project in Rust? It will permanently become an Intermediate language.
      - adrianN 6y ago
        Converting C to Rust is not that hard. Converting C to safe Rust is the difficult part.
        fractalb 6y ago
        Rust without safety isn't Rust at all!
ogoffart 6y ago
> The ubiquitous kmalloc() function, for instance, is defined as __always_inline, meaning that it is inlined into all of its callers and no kmalloc() symbol exists in the kernel symbol table for Rust to link against. This problem can be easily worked around — one can define a kmalloc_for_rust() symbol containing an un-inlined version but performing these workarounds by hand would result in a large amount of manual work and duplicated code. That's why there exists crates like https://docs.rs/cpp/ https://docs.rs/cpp/ which allow to embed C++ (and thus C) directly into the rust source code, simplifying the manual work and reducing the duplicated code.
- geofft 6y ago
  Yup - that got a very brief mention in the article as "Google is working on automated C++ conversions"; there's also https://github.com/google/autocxx https://github.com/google/autocxx which builds on top of it. C++ is a richer language than C and has more information about ownership (e.g., if you return a std::string you know how ownership works; if you have two arguments that are a char * and a length it's much less clear), but additional annotations in the kernel's C headers to convey this level of information in a machine-parseable way would be awesome.
  - ogoffart 6y ago
    The cpp and cxx crates are two different crates with different goal. The cpp crate is about convenience, but require the use of unsafe within the rust code. The cxx crate do extra checks making the rust code safer but needs more boilerplate. In the case of C however, these type checking are not really possible, as you say.
  - pjmlp 6y ago
    That is one of the learnings Microsoft had with XP SP2, hence SAL macros everywhere on Windows APIs.
msla 6y ago
What kind of performance hit is rewriting part of the kernel in Rust going to entail when it comes to compiling the kernel? How many platforms will cease to be able to compile their own kernels? The article says there will be some, but doesn't give numbers. More worryingly, the article also says that only some platforms will be "Rusted" in this manner, splitting the codebase into "mainline Rust" and "obscure platform C" as far as these sections are concerned; this seems like it will fracture the development community between people who know Rust and those who don't, and those who can work on mainstream platforms and those who can't. Fewer eyes on this code seems like the wrong way to go.
- sanxiyn 6y ago
  Architectures supported by Linux kernel but not supported by Rust are tracked at https://github.com/fishinabarrel/linux-kernel-module-rust/issues/112 https://github.com/fishinabarrel/linux-kernel-module-rust/is.... It currently lists 14 architecutres (as opposed to 10 supported architectures), so it's less than half.
- mlindner 6y ago
  The point is that if you implement a driver for a device that only exists on hardware that supports LLVM then there is no problem.
  - msla 6y ago
    No, the point is that increasing the number of languages in the kernel imposes restrictions. (The further point is that it's impossible to have this discussion here.)
mrtweetyhack 6y ago
I'd rather see a complete rewrite instead
egberts1 6y ago
So glad that Rust at 1.38 ditched the Ada model of compiling all dependent modules firstly.
- ifmpx 6y ago
  Can you please expand further on what you mean by this? Are you saying that rustc is compiling a crate before its dependencies?
  - pornel 6y ago
    Compilation in Cargo has been split into metadata pass and code generation pass. Metadata part has interdependencies, but is relatively fast (it's roughly like a C header generation). This allowed more codegen to be done in parallel. To be honest, it was a modest improvement, because compilation has been decently parallelized already (compiler uses incremental builds, parallel codegen, and ThinLTO even within a single crate)
jasonhansel 6y ago
I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products. People underestimate the role of copyleft licenses in preserving long-running FOSS products like GCC, Linux, etc.
- dnautics 6y ago
  Can you give an example of a non-copyleft open source product where the major improvement is proprietary?
  - rickbutton 6y ago
    nginx
    - dnautics 6y ago
      Sorry, I wasn't aware. What's the major improvement?
      - kevincox 6y ago
        https://www.nginx.com/products/nginx/ https://www.nginx.com/products/nginx/ Although this is probably not the best example because this is sold by the company that largely funds the open source project. Although that is a big conflict of interest.
      - tick_tock_tick 6y ago
        Everything in the Plus category. https://www.nginx.com/products/nginx/#compare-versions https://www.nginx.com/products/nginx/#compare-versions
      - pornel 6y ago
        Cloudflare-nginx has much improved HTTP/2 support: https://blog.cloudflare.com/nginx-structural-enhancements-for-http-2-performance/ https://blog.cloudflare.com/nginx-structural-enhancements-fo...
nielsbot 6y ago
Sounds like they have the same problem Apple's Swift does for calling into C/Obj-C. Two examples: My understanding is that there's a ton of code and added complexity in Swift itself to support Obj-C interoperation. And while you can call C and Obj-C APIs, as-is, there was/is basically a company-wide effort to write API wrappers (aka. overlays) for existing system APIs. For the second part, maybe if Rust for Linux kernel programming gets to the level of popularity as, say, TypeScript in the JavaScript community, the Linux community will end up creating the needed API wrappers as an organic, group effort.
- rapsey 6y ago
  Not even remotely as difficult as Swift. Rust can call into C at any time. It is ABI compatible and has no VM
  - Lukasa 6y ago
    The same is true of Swift, and Swift can also call into C whenever it likes. The difference is that Swift can do the same for Objective-C at full fidelity, including support the full Objective-C feature set (ARC, arbitrary selectors, interfaces, properties, you name it).
- why_only_15 6y ago
  This is mostly just a random tidbit but one of the fascinating things to me about Swift <-> ObjC interop is that it's not just that Swift was modified for ObjC, but ObjC was modified for Swift as well. In general it works pretty well and I wonder how much of that is because Apple can modify ObjC whenever it wants.
  - pjmlp 6y ago
    .NET <-> C++/COM also went through a similar process.
    - snuxoll 6y ago
      .Net is a much simpler integration, all of the support is in the CLR because COM defines a limited ABI that can be called from any language that can walk a few pointers in a C++ vtable given an interface definition. One can even completely ignore the RCW/CCW support in .Net entirely and just hack together some struct types that match the vtable layout.
bigbossgalasso 6y ago
Methinks machine code and assembly is the way to go not high level pussification of the kernel.
aey 6y ago
We built a whole BPF tool chain for rust :), although not for kernel development. https://github.com/solana-labs/rust https://github.com/solana-labs/rust If this kind of seems interesting please send us a CV.
- manvendrasingh 6y ago
  It does, problem is i am in IST Timezone, Will that work for the team ?
- RMPR 6y ago
  Do you have some kind of official blog for this? I'm interested in BPF and currently learning Rust, would love to learn more about this.
m00dy 6y ago
I believe someone will show up and rewrite entire kernel in Rust.
- pjmlp 6y ago
  Better start fresh with proper OS architectures, https://www.redox-os.org/ https://www.redox-os.org/

648 points by dochtman 6y ago

Probably a crazy unpopular opinion, but I kind of agree with the Hyperbola (GNU / formerly Linux-libre, in future using a fork of the OpenBSD kernel https://itsfoss.com/hyperbola-linux-bsd/ https://itsfoss.com/hyperbola-linux-bsd/) devs that Rust in Linux is not necessarily a good thing, for the reasons they mentioned when they announced they were switching kernels (https://www.hyperbola.info/news/announcing-hyperbolabsd-roadmap/ https://www.hyperbola.info/news/announcing-hyperbolabsd-road...) (Edit: and the LLVM dependency).

TylerE 6y ago
I don’t see how that conplaint is at all meaningful to kernel development. It’s typical GNU/Pendantry. The kernel will not be shipping a rust compiler, patched or otherwise.
- boogies 6y ago
  The article mentions possible ABI compatibility challenges and I also think it’s not a good thing for a language in such critical applications to only have a single major implementation.
  - eptcyka 6y ago
    The linux kernel still can only be compiled with GCC. Of course, it's a single patch away from clang compatibility, but the kernel has never been compilable with much else but GCC.
    - pjmlp 6y ago
      Android and ChromeOS beg to differ, as Google has been using clang with them for at least around three years now.
      - eptcyka 6y ago
        Yes, clang compatibility is a single patch away. Google, unlike the vast majority of companies out there, is capable of maintaining it's own forks of the kernel.
jcranmer 6y ago
If I'm understanding correctly, the only reason for not using Rust is... Rust is trademarked?
- dijit 6y ago
  I'm generally in support of rust in the kernel, but in the interest of playing devils advocate: Would adding more and more dependence on rust mean that it ties the kernel to a single set of compilers? I can see why people would be opposed to that. I'm thinking about LLVM (and, obviously, rustc)
  - Silhouette 6y ago
    Would adding more and more dependence on rust mean that it ties the kernel to a single set of compilers? Like GCC? Your point is certainly valid, but I'm not sure it's a new problem.
  - JoshTriplett 6y ago
    Linux currently only compiles on GCC and Clang, and the latter is a recent development. rustc uses LLVM as a backend, just like Clang, and people are working on other backends. Linux maintainers didn't express any concerns about this being an issue.
  - danudey 6y ago
    The Linux kernel would only compile with GCC for... decades? I think? So it's not really a new thing. It would be really nice if the kernel did have first-class support for LLVM and Clang directly (rather than having to set a bunch of environment variables to set the compiler, the linker to use with that compiler, various variables, etc. That said, it would certainly be uncomfortable to move from "it compiles with Clang, though it's a pain, or GCC, which works fine" to "it compiles with GCC, if you don't use any of this functionality, or LLVM only, if you want any of these features or anything that depends on them".
    - JoshTriplett 6y ago
      Originally, we thought that Linux maintainers were asking us to only support kernels compiled with Clang, because they didn't want to support code built with two different compilers. However, in the session, Greg Kroah-Hartman specifically said that if it works to build Rust code with the LLVM-based rustc and C code with GCC and link the two together, and there aren't any problems in practice, then that's perfectly fine. So, we're currently expecting that supporting Rust will not place any requirements on the C compiler you use, unless you want to use cross-language LTO (which we'll eventually want to do, but it won't be a hard requirement for a working kernel).
badrabbit 6y ago
Maybe a GNU Rust compiler without the trademark is in order? The state of KSPP is not nice as well. I wish Ada was more popular. From what I know, it has much of the security guarantees rust has and it has been battle tested.
- guerrilla 6y ago
  I'm not sure, but aren't they saying that they wouldn't be able to call it "GNU Rust" [1] in the same way they can't call IceCat "GNU Firefox"? 1. https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedom_flaws https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedo...
  - homarp 6y ago
    well, plenty of names to play with:) GNR: Gnr Not Rust TRust: Treated Rust ...
    - gpm 6y ago
      How do you pronounce GNR? It think it needs to be a vowel, or an R. Some attempts: GRR: Gnu Renames Rust, GAR: Gnu assimilates rust, GER: Gnu non Est Rust (gnu is not rust, but in latin to get an E), GIR: Gnu Isn't Rust, GOR: Gnu Oxidizes Rust, GUR: Gnu Unseats Rust.
      - homarp 6y ago
        pronounces it Gnar or Gener, depending if you are a gif or a jif person. You can also go for GunsNRoses, but they won't like it and complain. Which might be good for visibility at the beginning.
    - el_oni 6y ago
      Or something like "oxide", ferrite, maGNUtite. Play with iron ores or compounds
      - 6y ago
JoshTriplett 6y ago
The vast majority of the text in that post is inaccurate and hyperbolic. Linux is not "forcing adoption of HDCP" (it's just something Linux has a driver for), Rust does not require internet access to use (and the trademark comments are not particularly accurate either), the Kernel Self Protection Project is alive and well and making new improvements in every new version of the kernel (see Kees Cook's regular blog posts for details), systemd is not spelled "SystemD", gettext has no dependency on Java (and the link in that post explains that). And in general, decisions about what technologies to use are made by those who show up and do the work to build viable solutions, not by people who snark. It's a rant by an obscure one-developer distribution that formerly used the Linux kernel. Some people are never happy no matter what you do, and if you make the mistake of treating their rants as useful feedback, you get a list of 30 more demands before they'd consider gracing your little project with the honor of their all-important usage again.
- ColanR 6y ago
  I think you were being "inaccurate and hyperbolic". > Linux is not "forcing adoption of HDCP" (it's just something Linux has a driver for) What they actually said in the article: > Historically, some features began as optional ones until they reached total functionality. Then they became forced and difficult to patch out. Even if this does not happen in the case of HDCP, we remain cautious about such implementations.
  - JoshTriplett 6y ago
    My comment was entirely based on the second link. Direct quote from that: > Linux kernel forcing adaption of DRM, including HDCP. (Which linked to a patch adding driver support for handling HDCP, with absolutely no possibility of it somehow being forced.) Regarding the quote from that interview: sure, and there's also no guarantee that the Linux kernel won't drop support for every architecture except SPARC, except of course for all the people involved not putting up with it. In what possible universe would Linux developers decide it was a good idea to mandate HDCP? It's completely unfounded and unsupported hyperbole and fearmongering, without even a hint of potential truth. It's also roughly consistent with what I'd expect from someone who thinks that gettext daring to have support for Java and C# format strings is a horrific problem that should be ripped out by the roots.
Macha 6y ago
The licensing complaints about Rust apply equally to Python, which they package without objection: https://www.hyperbola.info/packages/extra/x86_64/python/ https://www.hyperbola.info/packages/extra/x86_64/python/
- cylon13 6y ago
  Not exactly: [0] > Some users have correctly mentioned that many other software packages have trademarks, do we plan to remove them all? No. We are not against all trademarks, only those which explicitly prohibit normal use, patching, and modification. > As an example, neither Python PSF nor Perl Trademarks currently prohibit patching the code without prior approval. They do prohibit abuse of their trademarks, e.g. you cannot create a company called “Python”, but this does not affect your ability to modify their free software and/or apply patches. > Due to the anti-modification clause, Rust is a non-permissive trademark that violates user freedom. [0] https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedom_flaws https://wiki.hyperbola.info/doku.php?id=en:main:rusts_freedo...
  - Macha 6y ago
    Well, good luck calling your modified Python release Python: https://github.com/naftaliharris/tauthon/issues/47 https://github.com/naftaliharris/tauthon/issues/47
  - jcranmer 6y ago
    I suspect whomever wrote that hasn't actually read the trademark policies in detail, or conferred with lawyers about it. Neither Python's nor Perl's agreements explicitly cover the case in detail, but Perl does mention this: > Licensed redistributors of Perl code are permitted by their licenses to use the Perl logo in connection with their distribution services, on product packaging, and in promotional materials. IANAL, but that text would especially suggest to me that you would have no right to use the trademark even to distribute an unmodified binary of Perl without approval. I suspect the real reason is Debian and Mozilla had a very public, well-known spat over licensing, and Python and Perl have not had very public, well-known spats with anybody.
ArchOversight 6y ago
Once again the GPL camp takes and relicenses a project so that further improvements they make can't be used by the original authors. Shame really.
- boogies 6y ago
  They can be used by the original authors, if the original authors stop intentionally licensing their code so weakly that anyone can make changes that no one else can even see.
- JetSetWilly 6y ago
  If they didn't want people to be able to make changes that cannot be used upstream, there's a license for that. It's called the GPL.
bitwize 6y ago
I'd be more concerned about the fact that the Rust Evangelism Strike Force is hellbent on getting everybody to depend on their exhausts-all-32-bit-address-space-to-build compiler and toolchain -- in addition to whatever toolchain the project currently uses, at least as long as the strangler pattern is being applied to migrate the code base to 100% Rust. GNU software in particular has always been such that the core stuff is bootstrappable from just C. But it seems, from an RESF standpoint, that the fact that any C code is out there running at all is a crisis-level problem.
- JoshTriplett 6y ago
  Standing reminder: the Rust project is not a fan of rabid Rust over-evangelism (e.g. "Rewrite It In Rust", "Rust Evangelism Strike Force"), and sees it as damaging and unhelpful. We discourage that kind of thing wherever we see it. We're much more in favor of a measured, cautious approach.

Thaxll 6y ago

Looks like there is a lot of work involved to have anything done, remind me the issues that the Chrome team said 2 weeks ago.

stmw 6y ago
Can you elaborate re: issues and lots of work required? I think it's generally to be expected that non-C language support in the kernel would be hard...
- muricula 6y ago
  Here's the article the parent mentioned in passing, it may answer your question: https://www.chromium.org/Home/chromium-security/memory-safety/rust-and-c-interoperability https://www.chromium.org/Home/chromium-security/memory-safet... It's very Chrome and C++ specific, but the problems will likely be similar in practice.
  - stmw 6y ago
    Ah, right, I'd read that - but it's for C++, so it's a bit different than the kernel situation, and I actually had a much more positive reaction to it ("wow, seems like C++<->Rust is actually pretty far along"). Guess both are reasonable takeaways.
gpm 6y ago
There is no doubt a lot of work here, but C <-> Rust interoperability is a lot cleaner than C++ <-> Rust. And for what it's worth, for almost all langauges C <-> X interoperability is a lot cleaner than C++ <-> X. Since almost all languages (including rust) "speak C", both in terms of compiler support and and being able to map every important C concept to a concept in their language. The same isn't true with C++. One thing to worry about with adding any language (including rust) to the kernel, is that the next good looking language will probably interoperate well with C and not interoperate well with Rust. So there is a much higher cost the second time you try to add a language.
- 6y ago

puzzledobserver 6y ago

One of the challenges to building an entirely new kernel is the vast amount of hardware support in the form of drivers and the features the existing kernel exports to userland in the form of syscalls. Would it be possible for a hypothetical new kernel, presumably written in Rust, to run an existing kernel such as Linux in a VM, just to tap into its drivers and emulate its syscalls? As more drivers are ported to the base kernel, the reliance on the donor kernel would shrink over time. Edit: This was an aside. Of course I realize that the conference was about adding Rust code to the Linux kernel, and not about rewriting any large, important, and functional codebase in language-of-the-day™. Hence my speculative language: "hypothetical" new kernel, "presumably" written in Rust.

cmckn 6y ago
This conference talk considered a re-write out-of-scope; they were only discussing how new code could be written in Rust. Interesting idea, though!
ColanR 6y ago
I was thinking about something similar recently - if the hardware drivers could somehow be abstracted from the rest of the linux kernel (as if...), then suddenly a ton of experimental kernels could have widespread hardware availability. It would probably shake up the OS ecosystem to no small degree, but I imagine the fresh ideas that could be experimented with by non-experts would be hugely beneficial. Isn't that the big argument against so many new projects? "It looks cool, but the hardware support isn't there." FreeBSD might be a better target though, since that's specifically designed to be compatible with everything.
- alex7o 6y ago
  This exists and is called rumpkerbels if I remember correctly.
  - puzzledobserver 6y ago
    rumpkerbels? Mind sharing a link? Google Search is curiously empty.

ghostwriter 6y ago

I won't get tired of repeating the same comment in every topic that suggests Rust to be a great replacement of C in existing projects, that it isn't. The safety guarantees that Rust provides are neither unique nor complete, and if we discuss the amount of effort necessary for bringing new interfaces like the one this article mentions ("one can define a kmalloc_for_rust() symbol containing an un-inlined version"), we should compare it to other existing solutions, like ATS [1], that are designed to be seamlessly interoperable with C codebases without giving up on the safety side of the argument. Just a week ago there was a link in ATS reddit channel that advertised ATS Linux [2][3] initiative, that may be of great interest for the same people who are interested in bringing more memory sefety to Kernel development, without giving up on existing C interfaces and the toolchain[4]. [1] http://www.ats-lang.org/Documents.html#INT2PROGINATS http://www.ats-lang.org/Documents.html#INT2PROGINATS [2] https://www.reddit.com/r/ATS/comments/ibyczp/ats_linux/ https://www.reddit.com/r/ATS/comments/ibyczp/ats_linux/ [3] http://git.bejocama.org http://git.bejocama.org [4] http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/c2016.html http://ats-lang.sourceforge.net/DOCUMENT/INT2PROGINATS/HTML/...

gpm 6y ago
Your evidence (that there are potentially other alternatives) doesn't support your claim (that Rust isn't a great C replacement). There can be multiple great C replacements. Not being "perfect" in terms of the guarantees you provide doesn't mean you aren't a great improvement. Nor would not providing any guarantees at all mean that a language is necessarily not a great improvement. That said, I would be very interested in seeing a comparison between C, Rust, and ATS in practical code. This is the first I'm hearing of ATS and it sounds interesting.
- ghostwriter 6y ago
  > Your evidence (that there are potentially other alternatives) doesn't support your claim (that Rust isn't a great C replacement). I didn't claim that Rust isn't a great replacement overall, I was specifically mentioning that it's not great for existing C codebases, by the fact that it requires significant work to replace internal interfaces that may be important for performance-, backward-compatibility- and conventional tooling reasons. It doesn't interoperate with C as well as other alternatives are capable of, whilst not being objectively better at safety guarantees either.

The_rationalist 6y ago

It's not about safety, if that was the real argument we should start with obvious improvements like checked C https://github.com/microsoft/checkedc https://github.com/microsoft/checkedc The C++ lifetime checker or Cyclone

geofft 6y ago
Checked C addresses a small number of memory safety problems - it addresses bounds checking / buffer overflows, but not use-after-frees, double frees, data races / insufficient locking, and several other things that fall into the general category of "memory safety," nor more general "safety" things like type confusion. The kernel is not written in C++ and is unlikely to use C++, for many reasons - Rust is a more natural C-but-better for the kernel than C is. Cyclone hasn't been maintained for over a decade; it was a research language, and it largely served its purpose as a research language in inspiring production-supported languages like Rust.

chromedev 6y ago

Pretty soon they'll be arguing for inclusion of JavaScript in the Linux kernel

cbmuser 6y ago

If we get the Rust backend for gcc finalized and merged first, it would be much easier to get Rust code into the kernel: > https://github.com/philberty/gccrs/ https://github.com/philberty/gccrs/

gok 6y ago
That's a front end?
- Spivak 6y ago
  Compile C to Rust! Easiest language migration ever.
  - fanf2 6y ago
    https://c2rust.com/ https://c2rust.com/
  - zelly 6y ago
    It shares the code generator with gcc. It doesn't transpile C to Rust or vice versa. The goal is to produce machine code directly.
  - fractalb 6y ago
    That would eliminate the need for Rust. Just imagine, if your C code can be converted into Rust, would there ever be a case to start a fresh project in Rust? It will permanently become an Intermediate language.
    - adrianN 6y ago
      Converting C to Rust is not that hard. Converting C to safe Rust is the difficult part.
      - fractalb 6y ago
        Rust without safety isn't Rust at all!

ogoffart 6y ago

> The ubiquitous kmalloc() function, for instance, is defined as __always_inline, meaning that it is inlined into all of its callers and no kmalloc() symbol exists in the kernel symbol table for Rust to link against. This problem can be easily worked around — one can define a kmalloc_for_rust() symbol containing an un-inlined version but performing these workarounds by hand would result in a large amount of manual work and duplicated code. That's why there exists crates like https://docs.rs/cpp/ https://docs.rs/cpp/ which allow to embed C++ (and thus C) directly into the rust source code, simplifying the manual work and reducing the duplicated code.

geofft 6y ago
Yup - that got a very brief mention in the article as "Google is working on automated C++ conversions"; there's also https://github.com/google/autocxx https://github.com/google/autocxx which builds on top of it. C++ is a richer language than C and has more information about ownership (e.g., if you return a std::string you know how ownership works; if you have two arguments that are a char * and a length it's much less clear), but additional annotations in the kernel's C headers to convey this level of information in a machine-parseable way would be awesome.
- ogoffart 6y ago
  The cpp and cxx crates are two different crates with different goal. The cpp crate is about convenience, but require the use of unsafe within the rust code. The cxx crate do extra checks making the rust code safer but needs more boilerplate. In the case of C however, these type checking are not really possible, as you say.
- pjmlp 6y ago
  That is one of the learnings Microsoft had with XP SP2, hence SAL macros everywhere on Windows APIs.

msla 6y ago

What kind of performance hit is rewriting part of the kernel in Rust going to entail when it comes to compiling the kernel? How many platforms will cease to be able to compile their own kernels? The article says there will be some, but doesn't give numbers. More worryingly, the article also says that only some platforms will be "Rusted" in this manner, splitting the codebase into "mainline Rust" and "obscure platform C" as far as these sections are concerned; this seems like it will fracture the development community between people who know Rust and those who don't, and those who can work on mainstream platforms and those who can't. Fewer eyes on this code seems like the wrong way to go.

sanxiyn 6y ago
Architectures supported by Linux kernel but not supported by Rust are tracked at https://github.com/fishinabarrel/linux-kernel-module-rust/issues/112 https://github.com/fishinabarrel/linux-kernel-module-rust/is.... It currently lists 14 architecutres (as opposed to 10 supported architectures), so it's less than half.
mlindner 6y ago
The point is that if you implement a driver for a device that only exists on hardware that supports LLVM then there is no problem.
- msla 6y ago
  No, the point is that increasing the number of languages in the kernel imposes restrictions. (The further point is that it's impossible to have this discussion here.)

mrtweetyhack 6y ago

I'd rather see a complete rewrite instead

egberts1 6y ago

So glad that Rust at 1.38 ditched the Ada model of compiling all dependent modules firstly.

ifmpx 6y ago
Can you please expand further on what you mean by this? Are you saying that rustc is compiling a crate before its dependencies?
- pornel 6y ago
  Compilation in Cargo has been split into metadata pass and code generation pass. Metadata part has interdependencies, but is relatively fast (it's roughly like a C header generation). This allowed more codegen to be done in parallel. To be honest, it was a modest improvement, because compilation has been decently parallelized already (compiler uses incremental builds, parallel codegen, and ThinLTO even within a single crate)

jasonhansel 6y ago

I'm concerned about the gradual move from GCC to LLVM. The lack of copyleft protections on LLVM means that it's much more dependent on corporate sponsorship, and means that there's a risk that major improvements to LLVM compilers will only become available as proprietary products. People underestimate the role of copyleft licenses in preserving long-running FOSS products like GCC, Linux, etc.

dnautics 6y ago
Can you give an example of a non-copyleft open source product where the major improvement is proprietary?
- rickbutton 6y ago
  nginx
  - dnautics 6y ago
    Sorry, I wasn't aware. What's the major improvement?
    - kevincox 6y ago
      https://www.nginx.com/products/nginx/ https://www.nginx.com/products/nginx/ Although this is probably not the best example because this is sold by the company that largely funds the open source project. Although that is a big conflict of interest.
    - tick_tock_tick 6y ago
      Everything in the Plus category. https://www.nginx.com/products/nginx/#compare-versions https://www.nginx.com/products/nginx/#compare-versions
    - pornel 6y ago
      Cloudflare-nginx has much improved HTTP/2 support: https://blog.cloudflare.com/nginx-structural-enhancements-for-http-2-performance/ https://blog.cloudflare.com/nginx-structural-enhancements-fo...

nielsbot 6y ago

Sounds like they have the same problem Apple's Swift does for calling into C/Obj-C. Two examples: My understanding is that there's a ton of code and added complexity in Swift itself to support Obj-C interoperation. And while you can call C and Obj-C APIs, as-is, there was/is basically a company-wide effort to write API wrappers (aka. overlays) for existing system APIs. For the second part, maybe if Rust for Linux kernel programming gets to the level of popularity as, say, TypeScript in the JavaScript community, the Linux community will end up creating the needed API wrappers as an organic, group effort.

rapsey 6y ago
Not even remotely as difficult as Swift. Rust can call into C at any time. It is ABI compatible and has no VM
- Lukasa 6y ago
  The same is true of Swift, and Swift can also call into C whenever it likes. The difference is that Swift can do the same for Objective-C at full fidelity, including support the full Objective-C feature set (ARC, arbitrary selectors, interfaces, properties, you name it).
why_only_15 6y ago
This is mostly just a random tidbit but one of the fascinating things to me about Swift <-> ObjC interop is that it's not just that Swift was modified for ObjC, but ObjC was modified for Swift as well. In general it works pretty well and I wonder how much of that is because Apple can modify ObjC whenever it wants.
- pjmlp 6y ago
  .NET <-> C++/COM also went through a similar process.
  - snuxoll 6y ago
    .Net is a much simpler integration, all of the support is in the CLR because COM defines a limited ABI that can be called from any language that can walk a few pointers in a C++ vtable given an interface definition. One can even completely ignore the RCW/CCW support in .Net entirely and just hack together some struct types that match the vtable layout.

bigbossgalasso 6y ago

Methinks machine code and assembly is the way to go not high level pussification of the kernel.

aey 6y ago

We built a whole BPF tool chain for rust :), although not for kernel development. https://github.com/solana-labs/rust https://github.com/solana-labs/rust If this kind of seems interesting please send us a CV.

manvendrasingh 6y ago
It does, problem is i am in IST Timezone, Will that work for the team ?
RMPR 6y ago
Do you have some kind of official blog for this? I'm interested in BPF and currently learning Rust, would love to learn more about this.

m00dy 6y ago

I believe someone will show up and rewrite entire kernel in Rust.

pjmlp 6y ago
Better start fresh with proper OS architectures, https://www.redox-os.org/ https://www.redox-os.org/