9 ms·
Rust in QEMU Roadmap
- bonzini 2y agoThanks for posting this to HN! Author here, happy to answer any questions. (By the way, I did not originally start the project, though I've worked quite a bit on the safe abstractions that are mentioned in the roadmap).
- ndesaulniers 2y agoBullish or bearish on rust in the kernel?
- bonzini 2y agoHello! Small world. :) For drivers, it's already happening, especially for graphics but not limited to that. 6.13 has some very important changes. A lot of Linux is drivers so that's already a reason to be bullish. Answering for QEMU instead: it depends on the community being willing to share the burden of writing the FFI code. Despite Rust being low level, there is still a substantial amount of work to do. Replies to the roadmap pointed out tracepoints as an area where I know nothing and therefore I would like someone else to do the work (I am working mostly on the object and threading models, which is also where a lot of the impedance mismatch between Rust and C lies).
- aapoalas 2y agoHello. I assume tracepoints mean kprobes/uprobes or something along those lines? I've just this weekend worked on implementing/adapting a crate for DTrace USDTs aka DTrace probes to also work on Linux and generate SystemTap SDTs (aka USDTs aka dtrace probes). This is probably a little different from tracepoints in the kernel space but I'm somewhat interested in going deeper and into the kernel side of things. Let me know if you have any pointers as to where I might be of concrete assistance to you!
- bonzini 2y agoQEMU has several tracepoint providers, the main ones are (a slightly fancy version of) printf and USDT. There is a Python program that generates the C code for the chosen backend(s), so the thing to do would be to adjust the script to produce either an FFI bridge or the equivalent Rust code.
- aapoalas 2y agoCan you link the script? This sounds vaguely like something that would be no skin off my back, so I'd be quite happy to help with this.
- bonzini 2y agoYes it's https://github.com/qemu/qemu/tree/master/scripts/tracetool https://github.com/qemu/qemu/tree/master/scripts/tracetool. I also found https://github.com/cuviper/probe-rs/tree/master/src/platform https://github.com/cuviper/probe-rs/tree/master/src/platform which seems interesting.
- aapoalas 2y agoHere's my WIP addition to Oxide Computer's usdt crate: https://github.com/oxidecomputer/usdt/pull/340 https://github.com/oxidecomputer/usdt/pull/340 This crate gives you a nifty macro that translates Rust probe definitions into the correct ELF sections and other related assembly. Other ways to declare probes are also supported, though. Their support is currently only for illumos and OS X but I managed to get Linux SystemTap SDTs working as well, and a PR for FreeBSD exists as well but has been languishing in review purgatory for two years: I imagine QEMU might enjoy this wide support (if they land).
- JoshTriplett 2y agoWhat does your wishlist for Rust look like? (Besides "simpler C/Rust interoperability", of course.) Has QEMU run into things that Rust-for-Linux hasn't, that feel missing from the Rust language?
- bonzini 2y agoRight now the only language-level thing I would like is const operator overloading. Even supporting MSRV as old as 1.63 was not a big deal, the worst thing was dependencies using let...else which we will vendor and patch. Pin is what it is, but it is mostly okay since I haven't needed projection so far. Initialization using Linux's "impl PinInit<Self>" approach seems to be working very well in my early experiments, I contributed changes to use the crate without unstable features. In the FFI area: Bindgen support for toml configuration (https://github.com/rust-lang/rust-bindgen/pull/2917 https://github.com/rust-lang/rust-bindgen/pull/2917 but it could also be response files on the command line), and easier passing of closures from Rust to C (though I found a very nice way to do it for ZSTs that implement Fn, which is by far the common case). The "data structure interoperability" part of the roadmap is something I should present to someone in the Rust community for impressions. Some kind of standardization of core FFI traits would be nice. Outside the Rust core proper, Meson support needs to mature a bit for ease of use, but it is getting there. Linux obviously doesn't need that. BTW, saw your comment in the dead thread, you're too nice. I have been curious about Rust for some time and with Linux maturing, and Linaro doing the first contribution of build system integration + sample device code, it was time to give it a try.
- JoshTriplett 2y ago> Right now the only language-level thing I would like is const operator overloading. As far as I know, const traits are still on track. > easier passing of closures from Rust to C As in, turning a Rust closure into a C function-pointer-plus-context-parameter? > The "data structure interoperability" part of the roadmap is something I should present to someone in the Rust community for impressions. Some kind of standardization of core FFI traits would be nice. Would be happy to work with you to get something on the calendar.
- o11c 2y ago> Related to this, the IsA trait allows typesafe compile-time checked casts. Unlike in C code, casting to a superclass can be written in such a way that the compiler will complain if the destination type is not a superclass, and with zero runtime cost. This is unfair to C. With a little work when defining all classes (or non-leaf classes if you're willing for a hairier implementation), you can do this there too. There are probably other ways, but the way that seems most "obvious" to me is to define the base via union of all bases in the chain, rather than only the immediate base class. So: /* class Foo {int f;}; class Bar extends Foo {int b;}; class Qux extends Bar {int q;}; class Leaf extends Qux {int l;}; */ struct Foo { int f; }; struct Bar { union { struct Foo base, base_Foo}; int b; }; struct Qux { union { struct Bar base, base_Bar; struct Foo base_Foo; }; int q; }; struct Leaf { struct Qux base; int l; } Then your "compile-time-safe-cast to base class" macro just checks 3 things in order: 1. check if we're already the right class. 2. check if `base` is the right class. 3. unconditionally try to use the appropriately-named `base_Foo` member as calculated from the type. (runtime-safe-checked downcasts just have to check that the reverse is possible)
- bonzini 2y ago"Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should." :D Though I admit that Rust also has to include the full list of base classes (https://github.com/rust-lang/rfcs/pull/1268 https://github.com/rust-lang/rfcs/pull/1268 would fix it).
- o11c 2y agoThe main reason the C version must hard-code is because C macros can't be recursive. What's Rust's excuse? (that said, the C approach is also nice for manually accessing fields of distant base classes)
- bonzini 2y agoIn Rust's case it is that when you say "you have IsA<Device> for all types that implement IsA<PCIDevice>" you are actually saying "you have IsA<Device> for all types, but only if they implement IsA<PCIDevice>". When you later say "you have IsA<Device> for all types, but only if they implement IsA<I2CDevice>" the two conditions conflict. It might be possible to avoid this issue with a different implementation but this is the simplest one that works and QEMU's hierarchy is generally shallow. If a better implementation came along, it would be a matter of search and repeat.
- greatgib 2y ago[flagged]
- raggi 2y agojust wait until you see the real conspiracy
- JoshTriplett 2y agoBy "the Rust community" here, you mean the author of the mail, one of the biggest long-standing developers of QEMU since before Rust was a thing? You're complaining that the developers of QEMU themselves are interested in adopting Rust because they think it'll work well for them?
- rcxdude 2y agoThis kind of thing reminds me of the backlash against systemd, where detractors felt like it was being imposed on the distributions that were for the most part adopting it because they liked what it offered.
- akira2501 2y agosystemd has tried to alter the kernel in breaking ways for it's own ends plenty of times. it's easy enough to not use and plenty of distributions do so with some effort but it's not as if systemd has had zero consequences on the rest of the ecosystem.
- Neywiny 2y agoI've said this before on here and I'll say it again. The QEMU code base is a nightmare. The amount of fake C++ is mind numbing. Every time I come across a variable or strict declaration or method with the word "class" in it, I'm reminded of how much easier the whole thing would've been with C++. You can't even compile C++ into QEMU because of how the headers use keywords. That's not even touching their macro abuse template functions. You know what has templates and classes? C++. And constructors. There's just so much. All this to say: if Rust can make it in, that's great, because I'm tired of dealing with C's simplicity for more complicated tasks. I'm not a rust user but if it lets me use classes and templates, I'll switch over
- masklinn 2y ago> I'm not a rust user but if it lets me use classes and templates, I'll switch over Yer not switching any time soon then. Rust does have methods but not classes (QOM’s inheritance is specifically called as an issue in TFA) and it uses Haskell-style generics rather than C++-style templates.
- Neywiny 2y ago:( the QOM inheritance is where I've had my wrist bugs. Recently I merged from master (upgrading me from version 8.something to 9.2.?) and they dramatically changed the resets. I tried the new way and had segfaults in bad places that went away when I reordered code that shouldn't have ordering requirements. That was too scary so I switched to their legacy reset function and it was all fine again. All this blind casting of opaques makes me nervous.
- bonzini 2y agoThe legacy reset way is fine, and yeah this is where being more strict and doing more stuff within the language should help.
- ThePhysicist 2y agoI mean it has polymorphism via v-tables and composition via traits, that's enough object-orientation for me. Inheritance is a core principle of OOP but in practice most C++ class hierarchies are relatively flat (there are exceptions like Qt, which I think uses inheritance in a good way), in practice most of that can be mimicked with embedding and composition well enough to work. Not sure if you want to call that object-oriented but operating on polymorphous structures that encapsulate their data is pretty close to object orientation (the good parts of OOP, at least). And I'm not a big expert on Rust but I think you can do most of the things we do in C++ with templates using macros.
- jdefr89 2y ago[flagged]
- School-Cotton 2y agoThe fact that rust makes it harder to write memory corruption bugs is only one of its many advantages. It’s genuinely a lot nicer and easier to use than comparable languages like C++, and I’d prefer it to that even if I didn’t care about memory corruption at all. Also, what’s wrong with the syntax? I hear a lot that people find the syntax ugly but I never understood what’s so fundamentally different about it compared to any other mainstream Algol-based language
- jdefr89 2y agoWhat are the other advantages exactly besides tool chain? A lot of people using Rust where the could have just used Go or another language that is memory safe and actually productive. Maybe I am just being a hater because I grew up on C/C++ but I know for a fact "Rustaceans" are getting out of control. Approaching zealot territory for sure. The time people spend fighting the Rust compiler for a project, they could have just written secure C.. That is just my personal opinion. I am not saying Rust shouldn't exist. I am just saying it isn't some big universal answer to all security and systems programming issues..
- hmry 2y ago> The time people spend fighting the Rust compiler for a project, they could have just written secure C I simply don't believe this anymore, based on the number of buffer overflow and memory corruption CVEs coming out of even mature C codebases every year
- jdefr89 2y agoCVEs are being assigned recklessly these days. Most of them aren't even security bugs and or cannot be exploited. Be wary of CVEs and their practical utility. Also this is a good read from AFL creator lcamtuf: https://lcamtuf.substack.com/p/a-reactionary-take-on-memory-safety https://lcamtuf.substack.com/p/a-reactionary-take-on-memory-...
- vlovich123 2y agoAny ideas why people are relying on distro packaged Rust for development instead of rustup? For Rust it feels weird making development choices around several year old versions of the language.
- deleted 2y ago[deleted]
- tangent128 2y agoRustup downloads toolchains from third-party (to the distro) repositories; distros do not want to be in a position where they can no longer build packages because of an external service going down. So, if you are developing something you want to see packaged in distros, it needs to be buildable with the tool versions in the distro's repositories. (Not just rustup- Debian requires repackaging Cargo dependencies so that the build can be conducted offline entirely from source packages.)
- vlovich123 2y agoYou’re answering a slightly different question but to me that’s a Debian packaging problem to solve. It’s weird to me that QEMU devs take this problem seriously enough to be putting in all sorts of workarounds to support old versions of the toolchain in the tip of tree just to privilege Debian support. This feels more like a CI thing for the QEMU project and I’m sure solvable by using rustup or a trusted deb repo that makes the latest tool chain available on older Debian platforms. As for Debian itself, for toolchains it really should do a better job back porting more recent versions of toolchains (not just Rust) or at least making them available to be installed. The current policy Debian is holding is really difficult to work with and causes downstream projects to do all sorts of workarounds to make Debian builds work (not just for Rust by the way - this applies to C++ as well). And it’s not like this is something it’s unfamiliar with - you can install multiple JVM versions in parallel and choose a different default.
- ronsor 2y ago> As for Debian itself, for toolchains it really should do a better job back porting more recent versions of toolchains (not just Rust) or at least making them available to be installed. Most toolchains don't have as much churn as Rust.