17 ms·
Giving Rust a chance for in-kernel codecs
- jeffbee 2y ago> raw pointer arithmetic and problematic memcpy() calls can be eliminated, array accesses can be checked at run time, and error paths can be greatly simplified. Complicated algorithms can be expressed more succinctly through the use of more modern abstractions such as iterators, ranges, generics, and the like. I know why people choose Rust today, but we could have had all of those benefits decades ago if not for the recalcitrance of reflexive C++ haters.
- pas 2y agoOf course, but ergonomics, vibe, or look-and-feel are important. C++ doesn't have a package manager, the .hpp thing is a mess (modules took ages), SFINAE, preprocessor macros vs proc-macros, and so on. That said, Linux is a very conservative and idiosyncratic project. It's really the bazaar. (No issue tracker, firehose of emails, etc.)
- vlovich123 2y agois the kernel using cargo? I don’t think the package manager is relevant.
- steveklabnik 2y agoIt is not using Cargo.
- pengaru 2y agoabsolutely not
- jsheard 2y ago> (modules took ages) They're arguably still not there yet, sure the standard was ratified years ago but the implementations are still a mess and uptake is almost nonexistent.
- flohofwoe 2y agoNot to mention: what C++ subset to use, and how to deal with the endless discussion-circlejerks around that one topic alone. For instance most of the C++ stdlib is useless for kernel (or embedded) development, so you end up with a C++ that's not much more than a "C with namespaces". At least Rust brings a couple of actual language improvements to the table (not a big fan of Rust, but if the only other option is C++, then Rust is the clear winner).
- jeffbee 2y agoNone of the C stdlib can be used in the kernel, either, and yet the kernel is written in C. Zircon is an existence proof that you can use the C++ std library in a kernel. Do they use every damned thing in std? No, but they do use array, pair, unique_ptr, iterators, and more.
- SAI_Peregrinus 2y ago> None of the C stdlib can be used in the kernel, either, Except the bits allowed in a freestanding environment and (stretching the definition of "used in") the "nolibc" partial C standard library the kernel includes for environments that don't have any other libc available (`tools/include/nolibc`, mostly used for kernel tests where there's no userspace at all).
- dale_glass 2y agoC++ even today has plenty footguns that will make code unsafe, and some of the safety comes at performance costs, or uses features that are ill-suited for the kernel.
- b20000 2y agothere is nothing problematic about memcpy, the problem is that you need to know what you are doing and rust won’t solve that problem.
- deleted 2y ago[deleted]
- jeroenhd 2y agoIt took C++ decades to get good, in my opinion. If the kernel team had switched to C++ decades ago, we probably would've ended up with a worse kernel code base. For me, C++11 and C++14 created the modern C++ that's actually usable, but compilers took a while to implement all those features (efficiently and correctly). I find the C++ equivalent of the Rust iterators and such to be even harder to read (almost an accomplishment, given the density of Rust code); I don't think features like ranges would be expressed more succinctly using std::range the same way it can be done in Rust, for instance. I also find C++'s iterators' API design rather verbose, and I don't think there's much good to be said of implementing generics through C++ templates. There are good reasons for why the language was designed this way, but I get the impression succinctness didn't seem to be a primary objective designing them. Rather, I get the feeling that the language designers prioritised making the features accessible to people who already knew C++ and were used to the complexer side of C++.
- akira2501 2y ago> the recalcitrance of reflexive C++ haters. I don't reflexively hate C++, just all the implementations of it.
- tialaramex 2y agoCrucially, the Rust for Linux people proposed to do the work to make Rust for Linux. Whenever we see this complaint from C++ people their idea is basically "LOL, Linus should rewrite kernel to be suitable for C++" and the answer is No. That's not going to happen. Once you refuse their fantasy "Don't lift a finger" way to add C++, they lose interest.
- codedokode 2y agoI don't understand why video codecs must be in kernel and run with supervisor privileges. Why can't they run in a userspace?
- ben-schaaf 2y agoThis is for hardware encoding/decoding.
- ec109685 2y agoI believe the reason is that it’s not safe to send arbitrary bitstreams directly to hardware decoders and given the “stateless” nature of them, you need something trusted to run the full video encoder / decoder logic.
- fulafel 2y agoWhat happens if you send bad bitstreams to the hardware?
- adastra22 2y agoYou can have it overwrite kernel memory.
- screcth 2y agoIsn't it possible to restrict what memory regions a device may have access to?
- adastra22 2y agoNo, it's on the PCIe bus. It sends data straight to RAM, circumventing the memory controller on the CPU. SOME systems have write protection logic on the RAM controllers themselves, but this is not universal.
- 2y ago
- b20000 2y agoRust must be removed from the kernel. It is a huge time sink for smaller companies to deal with another language and frameworks. Effort should be focused on getting more hardware support mainlined such that device manufacturers have less work which will increase adoption. Memory safety can be addressed via kernel tools or frameworks and should not be the job of a language IMHO.
- Faaak 2y agoSorry for this non-hn reply, but "lol"
- phi-go 2y agoIsn't Rust only used for kernel modules? So no one needs to depend on Rust code if not needed?
- steveklabnik 2y agoThat is correct, yes.
- b20000 2y agoi don’t want to learn rust to write a driver.
- yjftsjthsd-h 2y agoThen write your driver in C? The claim is that Rust makes it easier to produce high-quality drivers, but nobody's mandating it.
- hamandcheese 2y agoAs a user I am not particularly sympathetic to the economic concerns of some small company if that comes at the expense of my security. And Rust does seem to be increasing the rate and level of support small outfits can offer - see Asahi Linux for an example of that.
- 2y ago
- jeroenhd 2y agoI think these types of drivers, taking care of parsing in a place where I would personally say parsing should be avoided where possible, is an excellent place to move towards Rust. The language may not be great at doing things like low-level memory management, but parsing data and forwarding it to hardware seems like an excellent use case here. And yes, you can parse safely in C. Unfortunately, correctness in C has been proven to be rather difficult to achieve.
- andrepd 2y agoWhy is rust not good at low-level memory management? Can you give an example?
- matu3ba 2y agoHard mode Rust provides the trickyness of separating memory handling from the (pure) code logic https://matklad.github.io/2022/10/06/hard-mode-rust.html https://matklad.github.io/2022/10/06/hard-mode-rust.html which makes batching inputs unnecessary hard. The other main tradeoff is that explicit lifetime annotations make (big) refactorings or explorative coding harder, unless one uses a lot copies or shared pointers (unacceptable for Kernel code). However, I am not familiar how those two tradeoffs ought to be solved in Kernel. I would expect that Rust would do rarely changing high level abstractions expected to never change fundamentally in design and/or being limited in scope.
- deleted 2y ago[deleted]
- jmull 2y agoThis seems like a case of two classic mistakes to me… #1 starting with a solution, and then applying it to problems (imperfectly); #2 solving the problem at the wrong level (related to problem #1) - in this case at too low a level, which can work, but is a lot more work than a solution at the right level. That is: why not sandboxing rather than a rewrite?
- saagarjha 2y agoSandboxing an in-kernel codec?
- blipvert 2y agoeBPF?
- st_goliath 2y agoThe kernel has API in place to run code in userspace from within a kernel module ("usermode driver"). That would in theory be one way something complex in the kernel could be isolated. The obvious downside being the additional context switches and round trips for the data adding latency. Here's a blog post that demonstrates embedding an entire Go program as a blob into a kernel module and running it in userspace from the module: https://www.sigma-star.at/blog/2023/07/embedded-go-prog/ https://www.sigma-star.at/blog/2023/07/embedded-go-prog/
- saagarjha 2y agoWhile this is definitely an interesting idea (if with a lot of sharp edges, some of which you’ve discussed) some of these things are difficult to actually sandbox like this. Like, how do you meaningfully sandbox something that sets up page tables? Yes, it’s running in userspace–but if I exploit it there then I have extremely access to the system, largely equivalent to it running with kernel privileges in the first place. Likewise code that manages where the processor comes out of reset effectively has privileges regardless of where it runs, because the processor comes out of reset in a privileged state and influences on its execution are largely equivalent to having that privilege.