4 ms·

It's true that for bare metal programming Rust is quite far from 1.0. You can't even do inline assembly in stable rust... Arbitrary sized integers do sound ver

by simias 7y ago

It's true that for bare metal programming Rust is quite far from 1.0. You can't even do inline assembly in stable rust... Arbitrary sized integers do sound very convenient, although I guess the counterargument would be that they simply can't map to machine types so you'll have to have magic behind the scenes which may not make complete sense in low level languages. Still, C has bitfields in structs which are sort of the same thing so why not. Pointer handling however I'm not sold on. I'd argue that the very cumbersome pointer handling in Rust is a feature, not a bug. It's explicitly discouraged, and for good reasons. Map your pointer into a clean and safe reference, slice or wrapper object ASAP and leave raw pointers to the messy details of C FFIs and ultra low-level code. If you end up having to mess with raw pointers everywhere in your codebase you're doing it wrong, as far as Rust is concerned. I think this is reasonable even for kernel code. After all typically you wouldn't dereference register pointers directly in C code because of volatility issues and making it obvious that you're accessing hardware and not RAM (readl/writel or similar), so having a safe wrapper adds no overhead in my experience.

jackhalford 7y ago
Ok maybe I was user pointers wrong in Rust. And I must confess that I never really understood lifetimes either, however * I don't think that low level code needs to be "messy", and * I don't think that adding an abstraction on top solves anything complexity wise. For memory paging I used to have a whole library in Rust that handled hierarchy of page tables with very abstract pagetable classes and interfaces [0]. Now I just have 80 lines of zig that handles everything [1]. I much prefer the latter. > I think this is reasonable even for kernel code. It's not, for some of the reasons that Linus doesn't want C++ code in the kernel [2] [0] https://github.com/gz/rust-x86/blob/28d5839933973e0b639ef354cb82023031cb9535/src/bits64/paging.rs https://github.com/gz/rust-x86/blob/28d5839933973e0b639ef354... [1] https://git.sr.ht/~jzck/kernel/tree/master/src/arch/x86/paging.zig https://git.sr.ht/~jzck/kernel/tree/master/src/arch/x86/pagi... [2] https://yarchive.net/comp/linux/c++.html https://yarchive.net/comp/linux/c++.html
- CDSlice 7y ago
  The Rust code you wrote seems to be extremely heavy on boilerplate. A couple of macros could have drastically reduced the line count. I don't think what Linus wrote in 2004 on C++ has much relevance for 2020 Rust, especially for a new kernel that doesn't have a ton of contributors. Most of his complaints seem to be on how C++ abstractions can make it hard to review unknown code and how 2004 C++ code bases often had extremely messy code. Rust IMO doesn't suffer from those problems near as much as 2004 C++ did. Also, Google is using Rust for part of their new Fuchsia kernel so at least they think Rust has something good to offer kernel dev.
  - roblabla 7y ago
    Small nitpick: Google isn't using Rust for the Fuchsia kernel. The kernel, Zircon, is written entirely in C and is based on Little Kernel. Google is using Rust for some userspace drivers (Fuchsia being a micro-kernel).
    - loeg 7y ago
      > The kernel, Zircon, is written entirely in C No, it's written in C++[1]. They're very different languages. That said, Google seems to think C++ has something to offer to all kinds of development. It seems to work for them, but they are a heavily C++ shop. I wouldn't read too much out of their use of C++ anywhere; it'd be more surprising and interesting to see them use anything else. [1]: https://fuchsia.googlesource.com/fuchsia/+/master/zircon/kernel/kernel/scheduler.cc https://fuchsia.googlesource.com/fuchsia/+/master/zircon/ker...
      - roblabla 7y ago
        Oh that's interesting. Zircon is based on LK[0] which is in C, which is why I thought it was still written in C. I wonder how much is left of the original LK code then. [0]: https://fuchsia.dev/fuchsia-src/concepts/kernel/zx_and_lk https://fuchsia.dev/fuchsia-src/concepts/kernel/zx_and_lk
zozbot234 7y ago
I don't think bitfields are even that portable in C? You end up with a weird mix of bit-endianness and byte-endianness issues. The best approach AFAICT, if you need to model a binary format with arbitrary-sized integers, is to keep the binary structure opaque and have a function to "unpack" it to a record of machine-native types. Modern compilers can easily optimize the resulting code into shifts+bit operations, and it is a reasonably foolproof approach.
pcwalton 7y ago
The problem with C-style bitfield values/arbitrary-sized integers is that they don't have addresses, so you can't take references to them. This means that you need special cases everywhere, and it increases the complexity of the language a lot. In Rust, integers often have methods that take self by reference, which demands an actual address. For these reasons, in Rust I think it's better that arbitrarily-sized integers aren't first class. They're never really first class anyway.
- AndyKelley 7y ago
  In Zig they do; here is an example: https://clbin.com/HlJ0X https://clbin.com/HlJ0X Output: Test [1/1] test "bit field access"... *align(:3:1) u3 All 1 tests passed. Zig's pointer type has optional metadata to describe alignment, sub-byte offset, and SIMD vector element index. This generally makes things just work, and you'll get a compile error if you expected an unadorned aligned pointer. If I were to swap: - fn deref(ptr: var) u3 { + fn deref(ptr: *u3) u3 { Then I'd get: test.zig:13:23: error: expected type '*u3', found '*align(:3:1) u3' assert(deref(&data.b) == 2); ^
  - simias 7y ago
    How do you deal with atomicity though? A big deal in Rust is that you can only have one mutable reference to an object at any given time which prevents all sorts of aliasing issues, but here if you have two mutable fat pointers, one to and and one to b, and you attempt to write them without synchronization you're going to have a big problem, won't you? This is the main risk with bitfields in C, it makes reads and writes that appear to be independent actually access the same memory cell behind the scenes. I'm not convinced by the idea of adding even more magic on top of it to be honest, it is true that packing and unpacking bitfields is annoying and requires some boilerplate but in the end that is how it the hardware does it anyway.
    - AndyKelley 7y ago
      The builtin functions for atomic operations provided by the language do not allow unaligned pointers, so it would be a compile error.

by simias 7y ago

jackhalford 7y ago

Ok maybe I was user pointers wrong in Rust. And I must confess that I never really understood lifetimes either, however * I don't think that low level code needs to be "messy", and * I don't think that adding an abstraction on top solves anything complexity wise. For memory paging I used to have a whole library in Rust that handled hierarchy of page tables with very abstract pagetable classes and interfaces [0]. Now I just have 80 lines of zig that handles everything [1]. I much prefer the latter. > I think this is reasonable even for kernel code. It's not, for some of the reasons that Linus doesn't want C++ code in the kernel [2] [0] https://github.com/gz/rust-x86/blob/28d5839933973e0b639ef354cb82023031cb9535/src/bits64/paging.rs https://github.com/gz/rust-x86/blob/28d5839933973e0b639ef354... [1] https://git.sr.ht/~jzck/kernel/tree/master/src/arch/x86/paging.zig https://git.sr.ht/~jzck/kernel/tree/master/src/arch/x86/pagi... [2] https://yarchive.net/comp/linux/c++.html https://yarchive.net/comp/linux/c++.html

CDSlice 7y ago
The Rust code you wrote seems to be extremely heavy on boilerplate. A couple of macros could have drastically reduced the line count. I don't think what Linus wrote in 2004 on C++ has much relevance for 2020 Rust, especially for a new kernel that doesn't have a ton of contributors. Most of his complaints seem to be on how C++ abstractions can make it hard to review unknown code and how 2004 C++ code bases often had extremely messy code. Rust IMO doesn't suffer from those problems near as much as 2004 C++ did. Also, Google is using Rust for part of their new Fuchsia kernel so at least they think Rust has something good to offer kernel dev.
- roblabla 7y ago
  Small nitpick: Google isn't using Rust for the Fuchsia kernel. The kernel, Zircon, is written entirely in C and is based on Little Kernel. Google is using Rust for some userspace drivers (Fuchsia being a micro-kernel).
  - loeg 7y ago
    > The kernel, Zircon, is written entirely in C No, it's written in C++[1]. They're very different languages. That said, Google seems to think C++ has something to offer to all kinds of development. It seems to work for them, but they are a heavily C++ shop. I wouldn't read too much out of their use of C++ anywhere; it'd be more surprising and interesting to see them use anything else. [1]: https://fuchsia.googlesource.com/fuchsia/+/master/zircon/kernel/kernel/scheduler.cc https://fuchsia.googlesource.com/fuchsia/+/master/zircon/ker...
    - roblabla 7y ago
      Oh that's interesting. Zircon is based on LK[0] which is in C, which is why I thought it was still written in C. I wonder how much is left of the original LK code then. [0]: https://fuchsia.dev/fuchsia-src/concepts/kernel/zx_and_lk https://fuchsia.dev/fuchsia-src/concepts/kernel/zx_and_lk

zozbot234 7y ago

I don't think bitfields are even that portable in C? You end up with a weird mix of bit-endianness and byte-endianness issues. The best approach AFAICT, if you need to model a binary format with arbitrary-sized integers, is to keep the binary structure opaque and have a function to "unpack" it to a record of machine-native types. Modern compilers can easily optimize the resulting code into shifts+bit operations, and it is a reasonably foolproof approach.

pcwalton 7y ago

The problem with C-style bitfield values/arbitrary-sized integers is that they don't have addresses, so you can't take references to them. This means that you need special cases everywhere, and it increases the complexity of the language a lot. In Rust, integers often have methods that take self by reference, which demands an actual address. For these reasons, in Rust I think it's better that arbitrarily-sized integers aren't first class. They're never really first class anyway.

AndyKelley 7y ago
In Zig they do; here is an example: https://clbin.com/HlJ0X https://clbin.com/HlJ0X Output: Test [1/1] test "bit field access"... *align(:3:1) u3 All 1 tests passed. Zig's pointer type has optional metadata to describe alignment, sub-byte offset, and SIMD vector element index. This generally makes things just work, and you'll get a compile error if you expected an unadorned aligned pointer. If I were to swap: - fn deref(ptr: var) u3 { + fn deref(ptr: *u3) u3 { Then I'd get: test.zig:13:23: error: expected type '*u3', found '*align(:3:1) u3' assert(deref(&data.b) == 2); ^
- simias 7y ago
  How do you deal with atomicity though? A big deal in Rust is that you can only have one mutable reference to an object at any given time which prevents all sorts of aliasing issues, but here if you have two mutable fat pointers, one to and and one to b, and you attempt to write them without synchronization you're going to have a big problem, won't you? This is the main risk with bitfields in C, it makes reads and writes that appear to be independent actually access the same memory cell behind the scenes. I'm not convinced by the idea of adding even more magic on top of it to be honest, it is true that packing and unpacking bitfields is annoying and requires some boilerplate but in the end that is how it the hardware does it anyway.
  - AndyKelley 7y ago
    The builtin functions for atomic operations provided by the language do not allow unaligned pointers, so it would be a compile error.