6 ms·

C currently remains the language of system ABIs, and there remains functionality that C can express that Rust cannot (principally bitfields). Furthermore, in t

by jcranmer 11mo ago

C currently remains the language of system ABIs, and there remains functionality that C can express that Rust cannot (principally bitfields). Furthermore, in terms of extensions to the language to support more obtuse architecture, Rust has made a couple of decisions that make it hard for some of those architectures to be supported well. For example, Rust has decided that the array index type, the object size type, and the pointer size type are all the same type, which is not the case for a couple of architectures; it's also the case that things like segmented pointers don't really work in Rust (of course, they barely work in C, but barely is more than nothing).

raggi 11mo ago
That first sentence though. Bitfields and ABI alongside each other. Bitfield packing rules get pretty wild. Sure the user facing API in the language is convenient, but the ABI it produces is terrible (particularly in evolution).
- mjevans 11mo ago
  I would like a revision to bitfields and structs to make them behave the way a programmer things, with the compiler free to suggest changes which optimize the layout. As well as some flag that indicates the compiler should not, it's a finalized structure.
dataflow 11mo ago
In what architecture are those types different? Is there a good reason for it there architecturally, or is it just a toolchain idiosyncrasy in terms of how it's exposed (like LP64 vs. LLP64 etc.)?
- jcranmer 11mo ago
  CHERI has 64-bit object size but 128-bit pointers (because the pointer values also carry pointer provenance metadata in addition to an address). I know some of the pointer types on GPUs (e.g., texture pointers) also have wildly different sizes for the address size versus the pointer size. Far pointers on segmented i386 would be 16-bit object and index size but 32-bit address and pointer size. There was one accelerator architecture we were working that discussed making the entire datapath be 32-bit (taking less space) and having a 32-bit index type with a 64-bit pointer size, but this was eventually rejected as too hard to get working.
  - sroussey 11mo ago
    I guess today, instead of 128bit pointers we have 64bit pointers and secret provenance data inside the cpu, at least on the most recent shipped iPhones and Macs. In the end, I’m not sure that’s better, or maybe we should have had extra large pointers again (in that way back 32bit was so large we stuffed other stuff in there) like CHERI proposes (though I think it still has secret sidecar of data about the pointers). Would love to Apple get closer to Cheri. They could make a big change as they are vertically integrated, though I think their Apple Silicon for Mac moment would have been the time. I wonder what big pointers does to performance.
    - saagarjha 11mo ago
      It's not secret, it just reuses some of the unused address bits.
kbolino 11mo ago
I'm genuinely surprised that usize <=> pointer convertibility exists. Even Go has different types for pointer-width integers (uintptr) and sizes of things (int/uint). I can only guess that Rust's choice was seen as a harmless simplification at the time. Is it something that can be fixed with editions? My guess is no, or at least not easily.
- aw1621107 11mo ago
  > Is it something that can be fixed with editions? My guess is no, or at least not easily. Assuming I'm reading these blog posts [0, 1] correctly, it seems that the size_of::<usize>() == size_of::<*mut u8>() assumption is changeable across editions. Or at the very least, if that change (or a similarly workable one) isn't possible, both blog posts do a pretty good job of pointedly not saying so. [0]: https://faultlore.com/blah/fix-rust-pointers/#redefining-usize https://faultlore.com/blah/fix-rust-pointers/#redefining-usi... [1]: https://tratt.net/laurie/blog/2022/making_rust_a_better_fit_for_cheri_and_other_platforms.html https://tratt.net/laurie/blog/2022/making_rust_a_better_fit_...
  - kbolino 11mo ago
    Great info! Personally, I like 3.1.2 from your link [0] best, which involves getting rid of pointer <=> integer casts entirely, and just adding methods to pointers, like addr and with_addr. This needs no new types and no new syntax, though it does make pointer arithmetic a little more cumbersome. However, it also makes it much clearer that pointers have provenance. I think the answer to "can this be solved with editions" is more "kinda" rather than "no"; you can make hard breaks with a new edition, but since the old editions must still be supported and interoperable, the best you can do with those is issue warnings. Those warnings can then be upgraded to errors on a per-project basis with compiler flags and/or Cargo.toml options.
    - zozbot234 11mo ago
      AIUI, this particular change would only impact some newly supported architectures going forward. These don't need to be buildable with older editions.
vlovich123 11mo ago
Can you expand on bitfields? There’s crates that implement bitfield structs via macros so while not being baked into the language I’m not sure what in practice Rust isn’t able to do on that front.
- ZeWaka 11mo ago
  Yeah, not sure what they're saying... I use bitfields in multiple of my rust projects using those macros.
  - maweki 11mo ago
    I'm not a rust or systems programmer but I think it meant that as an ABI or foreign function interface bitfields are not stable or not intuitive to use, as they can't be declared granularily enough.
    - SAI_Peregrinus 11mo ago
      C's bit-fields ABI isn't great either. In particular, the order of allocation of bit-fields within a unit and alignment of non-bit-field structure members are implementation defined (6.7.2.1). And bit-fields of types other than `_Bool`, `signed int` and `unsigned int` are extensions to the standard, so that somewhat limits what types can have bitfields.
  - pjmlp 11mo ago
    Across binary libraries ABI, regardless of static or dynamically linked?
    - bigstrat2003 11mo ago
      Dynamic linking is not something you do in general in Rust. It's possible, but the compiler currently does not guarantee a stable ABI so it's not something one generally does.
- structural 11mo ago
throw_a_grenade 11mo ago
Also you can't do self-referential strutcs. Double-linked lists are also pain to implement, and they're are heavily used in kernel.
- menaerus 11mo ago
  [flagged]
  - baq 11mo ago
    Don’t spread FUD, you can check some example code yourself. https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/linux.git/tree/drivers/block/rnvme?h=rnvme&id=1db5efbbda1d3cc16f1ac4808b7ef384aac417e8 https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/l...
    - menaerus 11mo ago
      Sorry but what have I said wrong? The nature of code written in kernel development is such that using unsafe is inevitable. Low-level code with memory juggling and patterns that you usually don't find in application code. And yes, I have had a look into the examples - maybe one or two years there was a significant patch submitted to the kernel and number of unsafe sections made me realize at that moment that Rust, in terms of kernel development, might not be what it is advertised for. > https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/l https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/l.. Right? Thank you for the example. Let's first start by saying the obvious - this is not an upstream driver but a fork and it is also considered by its author to be a PoC at best. You can see this acknowledged by its very web page, https://rust-for-linux.com/nvme-driver https://rust-for-linux.com/nvme-driver, by saying "The driver is not currently suitable for general use.". So, I am not sure what point did you try to make by giving something that is not even a production quality code? Now let's move to the analysis of the code. The whole code, without crates, counts only 1500 LoC (?). Quite small but ok. Let's see the unsafe sections: rnvme.rs - 8x unsafe sections, 1x SyncUnsafeCell used for NvmeRequest::cmd (why?) nvme_mq/nvme_prp.rs - 1x unsafe section nvme_queue.rs - 6x unsafe not sections but complete traits nvme_mq.rs - 5x unsafe sections, 2x SyncUnsafeCell used, one for IoQueueOperations::cmd second for AdminQueueOperations::cmd In total, this is 23x unsafe sections/traits over 1500LoC, for a driver that is not even a production quality driver. I don't have time but I wonder how large this number would become if all crates this driver is using were pulled in into the analysis too. Sorry, I am not buying that argument.
gf000 11mo ago
> that C can express that Rust cannot The reverse is probably more true, though. Rust has native SIMD support for example, while in standard C there is no way to express that. 'C is not a low-level language' is a great blog post about the topic.
- kristianp 11mo ago
  So use the fairly common SIMD extensions in C, that's not much of an argument.
- ndiddy 11mo ago
  AFAIK std::simd is still nightly only. You can use the raw intrinsics in std::arch, but that's not any better than "#include <immintrin.h>".
- exDM69 11mo ago
  > Rust has native SIMD support std::simd is nightly only. > while in standard C there is no way to express that. In ISO Standard C(++) there's no SIMD. But in practice C vector extensions are available in Clang and GCC which are very similar to Rust std::simd (can use normal arithmetic operations). Unless you're talking about CPU specific intrinsics, which are available to in both languages (core::arch intrinsics vs. xmmintrin.h) in all big compilers.
qwm 11mo ago
Correct. C has a mostly standardized ABI, Rust does not. C has a spec, Rust does not. That matters in things like kernels.
yencabulator 11mo ago
> For example, Rust has decided that the array index type, the object size type, and the pointer size type are all the same type, [...] Not really, at least not anymore: https://doc.rust-lang.org/std/ptr/index.html#provenance https://doc.rust-lang.org/std/ptr/index.html#provenance This is also how Rust will work on CHERI. You can run your pointer-fiddling code in Miri today to check that you're following the rules.

raggi 11mo ago

That first sentence though. Bitfields and ABI alongside each other. Bitfield packing rules get pretty wild. Sure the user facing API in the language is convenient, but the ABI it produces is terrible (particularly in evolution).

mjevans 11mo ago
I would like a revision to bitfields and structs to make them behave the way a programmer things, with the compiler free to suggest changes which optimize the layout. As well as some flag that indicates the compiler should not, it's a finalized structure.

dataflow 11mo ago

In what architecture are those types different? Is there a good reason for it there architecturally, or is it just a toolchain idiosyncrasy in terms of how it's exposed (like LP64 vs. LLP64 etc.)?

jcranmer 11mo ago
CHERI has 64-bit object size but 128-bit pointers (because the pointer values also carry pointer provenance metadata in addition to an address). I know some of the pointer types on GPUs (e.g., texture pointers) also have wildly different sizes for the address size versus the pointer size. Far pointers on segmented i386 would be 16-bit object and index size but 32-bit address and pointer size. There was one accelerator architecture we were working that discussed making the entire datapath be 32-bit (taking less space) and having a 32-bit index type with a 64-bit pointer size, but this was eventually rejected as too hard to get working.
- sroussey 11mo ago
  I guess today, instead of 128bit pointers we have 64bit pointers and secret provenance data inside the cpu, at least on the most recent shipped iPhones and Macs. In the end, I’m not sure that’s better, or maybe we should have had extra large pointers again (in that way back 32bit was so large we stuffed other stuff in there) like CHERI proposes (though I think it still has secret sidecar of data about the pointers). Would love to Apple get closer to Cheri. They could make a big change as they are vertically integrated, though I think their Apple Silicon for Mac moment would have been the time. I wonder what big pointers does to performance.
  - saagarjha 11mo ago
    It's not secret, it just reuses some of the unused address bits.

kbolino 11mo ago

I'm genuinely surprised that usize <=> pointer convertibility exists. Even Go has different types for pointer-width integers (uintptr) and sizes of things (int/uint). I can only guess that Rust's choice was seen as a harmless simplification at the time. Is it something that can be fixed with editions? My guess is no, or at least not easily.

aw1621107 11mo ago
> Is it something that can be fixed with editions? My guess is no, or at least not easily. Assuming I'm reading these blog posts [0, 1] correctly, it seems that the size_of::<usize>() == size_of::<*mut u8>() assumption is changeable across editions. Or at the very least, if that change (or a similarly workable one) isn't possible, both blog posts do a pretty good job of pointedly not saying so. [0]: https://faultlore.com/blah/fix-rust-pointers/#redefining-usize https://faultlore.com/blah/fix-rust-pointers/#redefining-usi... [1]: https://tratt.net/laurie/blog/2022/making_rust_a_better_fit_for_cheri_and_other_platforms.html https://tratt.net/laurie/blog/2022/making_rust_a_better_fit_...
- kbolino 11mo ago
  Great info! Personally, I like 3.1.2 from your link [0] best, which involves getting rid of pointer <=> integer casts entirely, and just adding methods to pointers, like addr and with_addr. This needs no new types and no new syntax, though it does make pointer arithmetic a little more cumbersome. However, it also makes it much clearer that pointers have provenance. I think the answer to "can this be solved with editions" is more "kinda" rather than "no"; you can make hard breaks with a new edition, but since the old editions must still be supported and interoperable, the best you can do with those is issue warnings. Those warnings can then be upgraded to errors on a per-project basis with compiler flags and/or Cargo.toml options.
  - zozbot234 11mo ago
    AIUI, this particular change would only impact some newly supported architectures going forward. These don't need to be buildable with older editions.

vlovich123 11mo ago

Can you expand on bitfields? There’s crates that implement bitfield structs via macros so while not being baked into the language I’m not sure what in practice Rust isn’t able to do on that front.

ZeWaka 11mo ago
Yeah, not sure what they're saying... I use bitfields in multiple of my rust projects using those macros.
- maweki 11mo ago
  I'm not a rust or systems programmer but I think it meant that as an ABI or foreign function interface bitfields are not stable or not intuitive to use, as they can't be declared granularily enough.
  - SAI_Peregrinus 11mo ago
    C's bit-fields ABI isn't great either. In particular, the order of allocation of bit-fields within a unit and alignment of non-bit-field structure members are implementation defined (6.7.2.1). And bit-fields of types other than `_Bool`, `signed int` and `unsigned int` are extensions to the standard, so that somewhat limits what types can have bitfields.
- pjmlp 11mo ago
  Across binary libraries ABI, regardless of static or dynamically linked?
  - bigstrat2003 11mo ago
    Dynamic linking is not something you do in general in Rust. It's possible, but the compiler currently does not guarantee a stable ABI so it's not something one generally does.
structural 11mo ago

throw_a_grenade 11mo ago

Also you can't do self-referential strutcs. Double-linked lists are also pain to implement, and they're are heavily used in kernel.

menaerus 11mo ago
[flagged]
- baq 11mo ago
  Don’t spread FUD, you can check some example code yourself. https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/linux.git/tree/drivers/block/rnvme?h=rnvme&id=1db5efbbda1d3cc16f1ac4808b7ef384aac417e8 https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/l...
  - menaerus 11mo ago
    Sorry but what have I said wrong? The nature of code written in kernel development is such that using unsafe is inevitable. Low-level code with memory juggling and patterns that you usually don't find in application code. And yes, I have had a look into the examples - maybe one or two years there was a significant patch submitted to the kernel and number of unsafe sections made me realize at that moment that Rust, in terms of kernel development, might not be what it is advertised for. > https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/l https://git.kernel.org/pub/scm/linux/kernel/git/a.hindborg/l.. Right? Thank you for the example. Let's first start by saying the obvious - this is not an upstream driver but a fork and it is also considered by its author to be a PoC at best. You can see this acknowledged by its very web page, https://rust-for-linux.com/nvme-driver https://rust-for-linux.com/nvme-driver, by saying "The driver is not currently suitable for general use.". So, I am not sure what point did you try to make by giving something that is not even a production quality code? Now let's move to the analysis of the code. The whole code, without crates, counts only 1500 LoC (?). Quite small but ok. Let's see the unsafe sections: rnvme.rs - 8x unsafe sections, 1x SyncUnsafeCell used for NvmeRequest::cmd (why?) nvme_mq/nvme_prp.rs - 1x unsafe section nvme_queue.rs - 6x unsafe not sections but complete traits nvme_mq.rs - 5x unsafe sections, 2x SyncUnsafeCell used, one for IoQueueOperations::cmd second for AdminQueueOperations::cmd In total, this is 23x unsafe sections/traits over 1500LoC, for a driver that is not even a production quality driver. I don't have time but I wonder how large this number would become if all crates this driver is using were pulled in into the analysis too. Sorry, I am not buying that argument.

gf000 11mo ago

> that C can express that Rust cannot The reverse is probably more true, though. Rust has native SIMD support for example, while in standard C there is no way to express that. 'C is not a low-level language' is a great blog post about the topic.

kristianp 11mo ago
So use the fairly common SIMD extensions in C, that's not much of an argument.
ndiddy 11mo ago
AFAIK std::simd is still nightly only. You can use the raw intrinsics in std::arch, but that's not any better than "#include <immintrin.h>".
exDM69 11mo ago
> Rust has native SIMD support std::simd is nightly only. > while in standard C there is no way to express that. In ISO Standard C(++) there's no SIMD. But in practice C vector extensions are available in Clang and GCC which are very similar to Rust std::simd (can use normal arithmetic operations). Unless you're talking about CPU specific intrinsics, which are available to in both languages (core::arch intrinsics vs. xmmintrin.h) in all big compilers.

qwm 11mo ago

Correct. C has a mostly standardized ABI, Rust does not. C has a spec, Rust does not. That matters in things like kernels.

yencabulator 11mo ago

> For example, Rust has decided that the array index type, the object size type, and the pointer size type are all the same type, [...] Not really, at least not anymore: https://doc.rust-lang.org/std/ptr/index.html#provenance https://doc.rust-lang.org/std/ptr/index.html#provenance This is also how Rust will work on CHERI. You can run your pointer-fiddling code in Miri today to check that you're following the rules.