6 ms·

These seem like the first features that Rust in Linux bring to the Rust language that are not almost exclusively useful to the kernel. In my perception the focu

These seem like the first features that Rust in Linux bring to the Rust language that are not almost exclusively useful to the kernel. In my perception the focus on bringing features for the kernel has held up development in other parts of the language and the standard library.

oersted 1y ago
As I understand it systems programming is the priority application area for Rust, and there are plenty of projects working on OSs, embedded or other bare-metal cases, as well as interoperability with complex C codebases. At first glance, these features look quite general to me and not particularly tied to the kernel, they are important utilities for doing this kind of programming in the real world.
- heavyset_go 1y ago
  What's the story with C interop now with these and related changes? I'm out of the loop.
  - deleted 1y ago
    [deleted]
  - muvlon 1y ago
    C interop is excellent and has been for years. The one piece that still needs unstable is defining/exposing varargs functions (support for calling them was stabilized many years ago). You can write almost anything you can write in C in (partly unsafe) Rust, in fact there are projects like c2rust that automate this translation. These new features are all about making things that the kernel devs need possible in safe Rust. This often requires support for some quite fancy abstractions, some of which cannot be expressed in current stable Rust.
    - Foxboron 1y ago
      > C interop is excellent and has been for years. Only if you primarily work with `cargo` and want to interact with C from Rust. The other way around has far less support and `rustc` does not standardize the object generation. This is actively preventing projects like `systemd` to adopt Rust into their project as an example. https://github.com/systemd/systemd/pull/19598 https://github.com/systemd/systemd/pull/19598
      - aw1621107 1y ago
        > Only if you primarily work with `cargo` and want to interact with C from Rust. In what way(s) does Rust's C interop depend on cargo? > The other way around has far less support and `rustc` does not standardize the object generation. I believe in this context the understanding is that you're going to be using `extern "C"` and/or `#[repr(C)]` in your Rust code, which gives you a plain C interface. I think attempting to use "raw" Rust code from other languages is a rare phenomenon, if it's even attempted at all. > This is actively preventing projects like `systemd` to adopt Rust into their project as an example. Could you point out specific instances from that thread? From a quick glance I didn't see any obvious instances of someone saying that using Rust from C is problematic.
        humanrebar 1y ago
        > In what way(s) does Rust's C interop depend on cargo? Do rust and cargo allow for multiple interpretations of the same C header file across different objects in the same program? That's how C libraries are often implemented in practice due to preprocessor tricks, though I wish it wasn't normal to do this sort of thing.
        nicoburns 1y ago
        I would expect so. Rust and Cargo don't consume C header files directly at all. They consume bindings generated by bindgen (or hand written if you prefer). So you could probably generate mulitple bindings if you needed multiple interpretations of a C header. If the header files are consumed by C code that is then consumed by Rust then you'll have full support for what C supports because it will be compiled by a C compiler.
    - namr2000 1y ago
      I have to disagree here a little bit. Calling C functions from Rust is a very pleasant experience, but the other way around is not so nice. You usually have to manually create types that will unpack rust collections into C compatible structures (think decomposing `Vec` into ptr, len, capacity) & then ensure that memory passed between the two sides is free'd with the appropriate allocator. Even with `cbindgen` taking care of the mindless conversions for you, you still have to put a lot of thought into the API between the two languages. I am currently working on a fairly involved C & Rust embedded systems project and getting the inter-language interface stable and memory-leak free took a good amount of effort. It probably didn't help that I don't have access to valgrind or gdb on this platform.
      - aw1621107 1y ago
        I feel this might come down to the scope one has in mind when thinking of the word "interop". I think one can reasonably simultaneously claim that the interop "mechanics" are excellent in that it's generally possible to create a Rust library that quacks like a C library and that basically any C library is usable by Rust code, but the interop "ergonomics" are suboptimal in that (as you say) actually writing the glue code can be a bit of an adventure.
        namr2000 1y ago
        I think that's a fair assessment. To your point `cbindgen` makes the mechanics of the whole thing painless & linking was trivial. That's worth a lot especially when compared to other languages.
    - heavyset_go 1y ago
      Cool, thanks for the clarification
  - 3836293648 1y ago
    It's C++ that is problematic, C has been easy for years
    - heavyset_go 1y ago
      Are the kernel-related changes applicable to C++ interop? Honest question, I don't know.
      - gpm 1y ago
        The kernel doesn't use any C++, so no more than incidentally. There is some C++/rust interop in the past that I've worked on that would have enjoyed the arbitrary self types feature, but not particularly because of the C++ part of the equation. In fact I think if it had been a pure rust project it would also have enjoyed that feature just as much so... eh... take it for what little it's worth I guess.
  - cyphar 1y ago
    Calling C code from Rust? Pretty nice. Writing Rust code to be called from C (but within the same application)? Doable but somewhat painful. Writing Rust code to act like a C shared library? Quite painful and some pretty important features are missing (proper symbol versioning support being the most obvious one). Theoretically doable if you're willing to compromise. There's also some aspects of FFI-safety that are very subtle and easy to mess up: * #[repr(C)] enums still have the same requirements as Rust enums and so C callers can easily trigger UB, so you need to use something like open_enum. Thankfully cbindgen is too dumb to know that #[open_enum] is a proc macro and produces a non-enum type. * Before io_safety in Rust 1.63, dealing with file descriptors from C without accidentally closing them was horrific (though this was a wider problem in Rust). BorrowedFd is quite nice -- though Rustix will panic if you use negative fds and so you need to add validation and your own type in practice. However, #[repr(transparent)] is very nice for this. * Lots of reading about unsafe Rust is necessary when doing most non-trivial things with C FFI. * You need to make use of a lot of compiler internals, build scripts, and other magic to get the output you want. * Tools like cargo-c and cbindgen are nice and probably work great for 80% of projects, but the 20% really suffer from no useful tooling. I haven't tried to use rustc directly to work around some of the remaining issues, but I suspect it'd be even more painful. I would say that the C interop with Rust is pretty good but it has lots of room for improvement and it feels like very few resources have been spent on it after they got the core stuff working. Source: I've been writing a Rust library intended to be used primarily via C FFI and run into a lot of issues...
    - heavyset_go 1y ago
      Thanks for the info, I've been wanting to use Rust in a project with a lot of FFI going both Rust <-> C ways. Still sounds like it's a bit hairy.
pjmlp 1y ago
I see Rust's place on low level systems programming, for everything else on userspace compiled managed languages are a much better option, systems following an architecture like Self, Inferno or Android, so I don't see a big deal with these efforts focusing on low level C like capabilities.
- timschmidt 1y ago
  > for everything else on userspace compiled managed languages are a much better option As someone who's written a number of userspace applications in many languages as well as embedded firmwares running on bare metal, Rust is a rare gem that excels at both.
  - pjmlp 1y ago
    Only if those userspace applications are headless, Rust exceling at GUIs is a bit of a strech.
    - timschmidt 1y ago
      Well https://github.com/timschmidt/egui-rad-builder https://github.com/timschmidt/egui-rad-builder has come together rather well in the last week of hacking, if I say so myself. I think building a similar app with QT, for example, would have been significantly more challenging. I'm particularly fond of how easy it was to make all the controls live in the editor, and editable with changes appearing immediately. imgui would probably provide a similar experience, but I find C++ much more of a pain to work with than Rust.
      - mdhb 1y ago
        But that’s no longer the choice you need to make. Ubuntu themselves have said for a couple of years now that every new GUI app they make natively for Linux is going to be Flutter and dedicated a bunch of engineers to the project to make sure it’s a first class citizen. Beyond that, Dart / Futter are truly an absolute pleasure to use for that use case.
junon 1y ago
There are a lot of features being added for kernel/firmware development, they're just not on everyone's radar. Philopp has been a particular force for change in this area.

oersted 1y ago

As I understand it systems programming is the priority application area for Rust, and there are plenty of projects working on OSs, embedded or other bare-metal cases, as well as interoperability with complex C codebases. At first glance, these features look quite general to me and not particularly tied to the kernel, they are important utilities for doing this kind of programming in the real world.

heavyset_go 1y ago
What's the story with C interop now with these and related changes? I'm out of the loop.
- deleted 1y ago
  [deleted]
- muvlon 1y ago
  C interop is excellent and has been for years. The one piece that still needs unstable is defining/exposing varargs functions (support for calling them was stabilized many years ago). You can write almost anything you can write in C in (partly unsafe) Rust, in fact there are projects like c2rust that automate this translation. These new features are all about making things that the kernel devs need possible in safe Rust. This often requires support for some quite fancy abstractions, some of which cannot be expressed in current stable Rust.
  - Foxboron 1y ago
    > C interop is excellent and has been for years. Only if you primarily work with `cargo` and want to interact with C from Rust. The other way around has far less support and `rustc` does not standardize the object generation. This is actively preventing projects like `systemd` to adopt Rust into their project as an example. https://github.com/systemd/systemd/pull/19598 https://github.com/systemd/systemd/pull/19598
    - aw1621107 1y ago
      > Only if you primarily work with `cargo` and want to interact with C from Rust. In what way(s) does Rust's C interop depend on cargo? > The other way around has far less support and `rustc` does not standardize the object generation. I believe in this context the understanding is that you're going to be using `extern "C"` and/or `#[repr(C)]` in your Rust code, which gives you a plain C interface. I think attempting to use "raw" Rust code from other languages is a rare phenomenon, if it's even attempted at all. > This is actively preventing projects like `systemd` to adopt Rust into their project as an example. Could you point out specific instances from that thread? From a quick glance I didn't see any obvious instances of someone saying that using Rust from C is problematic.
      - humanrebar 1y ago
        > In what way(s) does Rust's C interop depend on cargo? Do rust and cargo allow for multiple interpretations of the same C header file across different objects in the same program? That's how C libraries are often implemented in practice due to preprocessor tricks, though I wish it wasn't normal to do this sort of thing.
        nicoburns 1y ago
        I would expect so. Rust and Cargo don't consume C header files directly at all. They consume bindings generated by bindgen (or hand written if you prefer). So you could probably generate mulitple bindings if you needed multiple interpretations of a C header. If the header files are consumed by C code that is then consumed by Rust then you'll have full support for what C supports because it will be compiled by a C compiler.
  - namr2000 1y ago
    I have to disagree here a little bit. Calling C functions from Rust is a very pleasant experience, but the other way around is not so nice. You usually have to manually create types that will unpack rust collections into C compatible structures (think decomposing `Vec` into ptr, len, capacity) & then ensure that memory passed between the two sides is free'd with the appropriate allocator. Even with `cbindgen` taking care of the mindless conversions for you, you still have to put a lot of thought into the API between the two languages. I am currently working on a fairly involved C & Rust embedded systems project and getting the inter-language interface stable and memory-leak free took a good amount of effort. It probably didn't help that I don't have access to valgrind or gdb on this platform.
    - aw1621107 1y ago
      I feel this might come down to the scope one has in mind when thinking of the word "interop". I think one can reasonably simultaneously claim that the interop "mechanics" are excellent in that it's generally possible to create a Rust library that quacks like a C library and that basically any C library is usable by Rust code, but the interop "ergonomics" are suboptimal in that (as you say) actually writing the glue code can be a bit of an adventure.
      - namr2000 1y ago
        I think that's a fair assessment. To your point `cbindgen` makes the mechanics of the whole thing painless & linking was trivial. That's worth a lot especially when compared to other languages.
  - heavyset_go 1y ago
    Cool, thanks for the clarification
- 3836293648 1y ago
  It's C++ that is problematic, C has been easy for years
  - heavyset_go 1y ago
    Are the kernel-related changes applicable to C++ interop? Honest question, I don't know.
    - gpm 1y ago
      The kernel doesn't use any C++, so no more than incidentally. There is some C++/rust interop in the past that I've worked on that would have enjoyed the arbitrary self types feature, but not particularly because of the C++ part of the equation. In fact I think if it had been a pure rust project it would also have enjoyed that feature just as much so... eh... take it for what little it's worth I guess.
- cyphar 1y ago
  Calling C code from Rust? Pretty nice. Writing Rust code to be called from C (but within the same application)? Doable but somewhat painful. Writing Rust code to act like a C shared library? Quite painful and some pretty important features are missing (proper symbol versioning support being the most obvious one). Theoretically doable if you're willing to compromise. There's also some aspects of FFI-safety that are very subtle and easy to mess up: * #[repr(C)] enums still have the same requirements as Rust enums and so C callers can easily trigger UB, so you need to use something like open_enum. Thankfully cbindgen is too dumb to know that #[open_enum] is a proc macro and produces a non-enum type. * Before io_safety in Rust 1.63, dealing with file descriptors from C without accidentally closing them was horrific (though this was a wider problem in Rust). BorrowedFd is quite nice -- though Rustix will panic if you use negative fds and so you need to add validation and your own type in practice. However, #[repr(transparent)] is very nice for this. * Lots of reading about unsafe Rust is necessary when doing most non-trivial things with C FFI. * You need to make use of a lot of compiler internals, build scripts, and other magic to get the output you want. * Tools like cargo-c and cbindgen are nice and probably work great for 80% of projects, but the 20% really suffer from no useful tooling. I haven't tried to use rustc directly to work around some of the remaining issues, but I suspect it'd be even more painful. I would say that the C interop with Rust is pretty good but it has lots of room for improvement and it feels like very few resources have been spent on it after they got the core stuff working. Source: I've been writing a Rust library intended to be used primarily via C FFI and run into a lot of issues...
  - heavyset_go 1y ago
    Thanks for the info, I've been wanting to use Rust in a project with a lot of FFI going both Rust <-> C ways. Still sounds like it's a bit hairy.

pjmlp 1y ago

I see Rust's place on low level systems programming, for everything else on userspace compiled managed languages are a much better option, systems following an architecture like Self, Inferno or Android, so I don't see a big deal with these efforts focusing on low level C like capabilities.

timschmidt 1y ago
> for everything else on userspace compiled managed languages are a much better option As someone who's written a number of userspace applications in many languages as well as embedded firmwares running on bare metal, Rust is a rare gem that excels at both.
- pjmlp 1y ago
  Only if those userspace applications are headless, Rust exceling at GUIs is a bit of a strech.
  - timschmidt 1y ago
    Well https://github.com/timschmidt/egui-rad-builder https://github.com/timschmidt/egui-rad-builder has come together rather well in the last week of hacking, if I say so myself. I think building a similar app with QT, for example, would have been significantly more challenging. I'm particularly fond of how easy it was to make all the controls live in the editor, and editable with changes appearing immediately. imgui would probably provide a similar experience, but I find C++ much more of a pain to work with than Rust.
    - mdhb 1y ago
      But that’s no longer the choice you need to make. Ubuntu themselves have said for a couple of years now that every new GUI app they make natively for Linux is going to be Flutter and dedicated a bunch of engineers to the project to make sure it’s a first class citizen. Beyond that, Dart / Futter are truly an absolute pleasure to use for that use case.

junon 1y ago

There are a lot of features being added for kernel/firmware development, they're just not on everyone's radar. Philopp has been a particular force for change in this area.