6 ms·

Making Rust a Better Fit for Cheri and Other Platforms

101 points by ltratt 5y ago

davidhyde 4y ago
It would also be nice to be able to use unsigned types (like u8, u16 and u32) to index into slices and arrays up until usize (usually a u32 or u64). Using a usize often seems wasteful for indexing into small arrays and slices and casting makes the code look ugly (also dangerous because the effect of casting is checked by the developer, not the compiler). Admittedly, this would have the downside of having code that compiles on one architecture but not another so it’s probably not worth it.
- Ygg2 4y ago
  Rust supports u16 pointer. Suddenly you are casting u32 to u16. And fun begins.
  - davidhyde 4y ago
    Exactly this!
- benschulz 4y ago
  IIRC integer literals are the blocking issue here. Bounds checking (and elision) happens anyway, but when `Index<T>` is implemented for multiple integer types, `foo[0]` becomes ambiguous.
- pwdisswordfish9 4y ago
  What’s wrong with .try_into::<usize>().unwrap()?
  - ekhjarta 4y ago
    It fails at runtime and not at compile time and it only fails if the value is out of range, so you have to test with large enough values to trigger the assertion during testing. We want to be able to index with integer types that are smaller than or has the same size as usize, but not with integer types that are larger than usize. And we want those checks at compile time.
    - dgb23 4y ago
      Unwrap isn’t an evil thing to avoid completely. The type system cannot understand all invariants, but you can, and you can make sure they are correctly implemented.
      - deleted 4y ago
        [deleted]
  - masklinn 4y ago
    .into() will work for u8 and u16, as rust assumes >= 16b pointers.
    - est31 4y ago
      Which totally makes sense. u8 means 256 bytes of addressable memory. You might design chips with such limited memory, but maybe high level (read non assembly) languages are not the right tool for targetting them.
  - edflsafoiewq 4y ago
    It's hideous.
- dgb23 4y ago
  Code looks ugly should maybe not be a big concern in a language like Rust. It’s fundamentally ugly to begin with in many ways. And I think that’s fine because it is a language that is primarily about control. One could argue that this is beautiful in its own way. There’s a clarity and power in being explicit while the constraints let you move on with confidence.
  - scns 4y ago
    Function over form. I think it makes a lot of sense in Rusts' domain.
- ekidd 4y ago
  Rust really doesn't do implicit numeric conversion, and I suspect it would be hard to retrofit. But you could write a function i(...) that promoted u8, etc, to usize: v[i(my_u8_index)] This could be implemented as a compile-time check, with no runtime errors. You'd have to limit which types you support as indices if you want to be portable to platforms with tiny pointers, of course. I guess something like this might be useful inside of specialized library code that worked with lots of small vectors. But in general, Rust heavily favors explicit over implicit in many areas. If you want lots of automatic implicit behavior, something like Scala might be a better choice?
  - samhw 4y ago
    If you were going to do that, it may be easier just to implement `Index`/`IndexMut` over `u8`/`u16`/.. for whatever type (newtype if necessary) you're indexing into. I suppose it's hard to say without context though.
- bestouff 4y ago
  You could use TiVec[1], TiSlice[1] or TiBitset[2] (shameless plug) to have datastructures indexed by the type you need, all checked at compile type. All you need is a From/Into implementation for your chosen index type. 1: https://lib.rs/crates/typed-index-collections https://lib.rs/crates/typed-index-collections 2: https://lib.rs/crates/tibitset https://lib.rs/crates/tibitset
- ATsch 4y ago
  > Using a usize often seems wasteful for indexing into small arrays and slices The opposite is often the case, as the architecture may not support offsetting a pointer by those sizes, requiring lots of casting in tight loops. Making that explicit makes the programmer aware of this. This is one major reason that C compiler developers want all overflow to be undefined behavior, as it allows them to upgrade for loops that use smaller index types to larger ones.
  - staticassertion 4y ago
    That seems easy to optimize out, especially given rust's mutability guarantees. If I have a u8 and I index into an array in a loop the compiler should be free to cast it to a usize above that loop, right? The benefit being you can store much smaller structures, plus in the case of memory unsafety you reduce attacker control. The vast majority of strings I personally use are certainly well under 2^32 bytes, and most are probably under 2^8.
- mjw1007 4y ago
  Right, better control of the index type can help you say what you mean, and help the typechecker catch more mistakes. If I have an array representing a single-byte lookup table, it would be nice if I could say that the array's type is array-indexed-by-u8 rather than array-indexed-by-usize. Ada did this well: it even lets you declare an array whose index is some enumerated type.
  - Ar-Curunir 4y ago
    > _ it even lets you declare an array whose index is some enumerated type._ You can do this in Rust also: make a custom newtype wrapper for your array, and impl Index<CustomEnum> for the wrapped array.
JulianMorrison 4y ago
Re the non-address bits of a pointer: there could be a method `shifted_by_address_width()` that just returns a same-size copy of the data (which remains unmodified) shifted over by the width of an address on the system. That way it's platform agnostic, it's still as wide as the address type, and on systems with no extra bits beyond the address, it's just zero.
mlindner 4y ago
This article feels very wrong. There is so much of exposing the underlying hardware CHERI functionality to the user in everything suggested in this article. It would never fly in Rust. This post feels completely written from the perspective of C-isms rather than from the perspective of Rust. This paragraph in particular: > Fortunately, I believe that Aria's proposal can be adapted such that a Rust-for-CHERI can cope with both pure capability and hybrid modes. In essence, one needs explicit, separate, types for both traditional pointers and capabilities. mut () suffices for the former and a wrapper of some sort, e.g. Cap<mut ()> for the latter. Conceptually this means that mut () is no longer the root of the integer/pointer type system, because one cannot convert Cap<mut ()> to mut () without losing information (the capability's extra bits). My gut feeling is that in practice, most code can treat mut () as the root of the integer/pointer type system, and only code which really cares about capabilities need know of Cap's existence. An additional nice property is that one will be able to write Rust code in a way that can be agnostic about pure capability mode (where size_of::<mut ()>() == size_of::<Cap<mut ()>>()) and hybrid mode (where size_of::<mut ()>() < size_of::<Cap<mut ()>>()). Capabilities should not be visible to users of the Rust language what-so-ever and should only exist in the compiler, and nowhere else. The more correct solution is to just forbid "hybrid" modes, which seem only useful for C projects where there is a lot of legacy code. Something Rust doesn't have. The author also admits as such in their footnote on hybrid mode having "many uses": > In the context of Rust, it's also worth asking whether it's worth making all pointers double width (which, though perhaps small, will undoubtedly have measurable memory and performance costs). After all, most Rust code is safe, and the compiler can guarantee that pointers can't be easily misused for things like buffer overruns. Rather, I see the main utility for a language like Rust being to impose various "sub-process" like compartments, with only relatively small portions of the code needing to use capabilities explicitly. It is perfectly acceptable to have all Rust-on-CHERI systems to have double width pointers in my opinion. This wouldn't affect software on any existing supported platforms. If you're writing new Rust code, for a CHERI system, but not using capabilities, I would wonder why you're using the platform in the first place.
- ltratt 4y ago
  I agree that you don't want non-CHERI Rust code to have to know about capabilities in any way. However, if you are using Rust for CHERI, you need to have some access to capability functions otherwise, even in pure capability mode, you can't reduce a capability's permissions. For example, if you want to write `malloc` in purecap Rust, you'll probably want to hand out capabilities that point to blocks of memory with bounds that only cover that block: you need some way to say "create a new capability which has smaller bounds than its parent capability." As to the utility of hybrid mode, I politely disagree. For example, is it useful for safe Rust code to always pay the size penalty of capabilities when you've proven at compile-time that they can't misuse the underlying pointers? A very different example is that hybrid mode allows you to meaningfully impose sub-process like compartments (in particular with the `DDC` register, which restricts non-capability code to a subset of the virtual address space; see also the `PCC` register and friends). Personally, I think that this latter technique holds a great deal of potential.
kibwen 4y ago
The OP links Gankra's first article, but don't miss the follow-up where she actually implemented some of these ideas as proof-of-concepts in the stdlib and compiler: https://gankra.github.io/blah/tower-of-weakenings/ https://gankra.github.io/blah/tower-of-weakenings/
dathinab 4y ago
I think there are some problems in the article. The main point of the referenced article by Aria Beingessner is Pointer Provenance. That this also helps out with enabling potential support for CHERI is a nice (and intended) side effect. So yes more is needed for supporting CHERI but: 1. The non-address parts, if and how you can access them are fully CHERI specific. Any such code should (for now) not be Generic Rust but architecture specific extensions. (E.g. live in `std::arch::<somethingCHERI>`) 2. "Rust's integer type hierarchy" There is no type hierarchy in the sense there is in C/C++ as there is no sub-typing of integers (i.e. auto-conversions) in rust. Same is true for pointers so I don't think it makes sense to call `mut ()` the "root of the...". (Also if anything that would be `const ()`). Similar it's more a void-pointer (through not quite the same) then a `uintptr_t`. To quote Aria: "I don’t think Rust needs to define a moral equivalent to intptr_t [...]". 3. The 129th bit is a "secret" implementation detail. So not representing it is intended. if your code relies on somehow detecting if it's set or not to work correctly you are doing something wrong. (Through maybe for testing/asserts, still at most a CHERI specific method under std::arch.) 4. The proposed scratch design for hybrid mode fundamentally doesn't work because of pointer provenance. Pointers need to be build-in types. So in hybrid mode you now would have 4 instead of 2 pointer types. Also given that you would want to use all of them in normal code you also would have 4 instead of 2 reference types. As far as I can tell this is also a messy nightmare to use in C/C++ (Through I have to read into it first). Anyway hybrid mode is as far as I can tell irrelevant for "pure" rust applications and only becomes relevant when linking against non-capability-supporting FFI. EDIT: Provenance is often partially lost at FFI boundary anyway and you probably would want to special type the non-capability pointers and then at the boundary convert them to "normal(capability)" pointers. Only handling "normal" pointers in rust (which all have capabilities when compiled for CHERI).
- zozbot234 4y ago
  > So in hybrid mode you now would have 4 instead of 2 pointer types. Also given that you would want to use all of them in normal code you also would have 4 instead of 2 reference types. We'd need more pointer types to support other segmented architectures anyway. Real mode x86 has __near, __far and __huge pointer types.
saati 4y ago
> It's important to note that the cast warnings (or errors) are not enough on their own to help make code correct: additional auditing will be required. However, at the very least, this will give programmers a good idea of where to start auditing their code [4]. Some people might choose not to adapt their code, or find it too difficult to do so, but my guess is that most people will choose to do so. I strongly doubt that most people would change their code for an ISA that barely exists.
- burntsushi 4y ago
  Maybe not. But I might change my code if it makes it easier to reason about its correctness. Especially in unsafe blocks and especially if I don't have to give up anything else (like perf).

101 points by ltratt 5y ago

It would also be nice to be able to use unsigned types (like u8, u16 and u32) to index into slices and arrays up until usize (usually a u32 or u64). Using a usize often seems wasteful for indexing into small arrays and slices and casting makes the code look ugly (also dangerous because the effect of casting is checked by the developer, not the compiler). Admittedly, this would have the downside of having code that compiles on one architecture but not another so it’s probably not worth it.

Ygg2 4y ago
Rust supports u16 pointer. Suddenly you are casting u32 to u16. And fun begins.
- davidhyde 4y ago
  Exactly this!
benschulz 4y ago
IIRC integer literals are the blocking issue here. Bounds checking (and elision) happens anyway, but when `Index<T>` is implemented for multiple integer types, `foo[0]` becomes ambiguous.
pwdisswordfish9 4y ago
What’s wrong with .try_into::<usize>().unwrap()?
- ekhjarta 4y ago
  It fails at runtime and not at compile time and it only fails if the value is out of range, so you have to test with large enough values to trigger the assertion during testing. We want to be able to index with integer types that are smaller than or has the same size as usize, but not with integer types that are larger than usize. And we want those checks at compile time.
  - dgb23 4y ago
    Unwrap isn’t an evil thing to avoid completely. The type system cannot understand all invariants, but you can, and you can make sure they are correctly implemented.
    - deleted 4y ago
      [deleted]
- masklinn 4y ago
  .into() will work for u8 and u16, as rust assumes >= 16b pointers.
  - est31 4y ago
    Which totally makes sense. u8 means 256 bytes of addressable memory. You might design chips with such limited memory, but maybe high level (read non assembly) languages are not the right tool for targetting them.
- edflsafoiewq 4y ago
  It's hideous.
dgb23 4y ago
Code looks ugly should maybe not be a big concern in a language like Rust. It’s fundamentally ugly to begin with in many ways. And I think that’s fine because it is a language that is primarily about control. One could argue that this is beautiful in its own way. There’s a clarity and power in being explicit while the constraints let you move on with confidence.
- scns 4y ago
  Function over form. I think it makes a lot of sense in Rusts' domain.
ekidd 4y ago
Rust really doesn't do implicit numeric conversion, and I suspect it would be hard to retrofit. But you could write a function i(...) that promoted u8, etc, to usize: v[i(my_u8_index)] This could be implemented as a compile-time check, with no runtime errors. You'd have to limit which types you support as indices if you want to be portable to platforms with tiny pointers, of course. I guess something like this might be useful inside of specialized library code that worked with lots of small vectors. But in general, Rust heavily favors explicit over implicit in many areas. If you want lots of automatic implicit behavior, something like Scala might be a better choice?
- samhw 4y ago
  If you were going to do that, it may be easier just to implement `Index`/`IndexMut` over `u8`/`u16`/.. for whatever type (newtype if necessary) you're indexing into. I suppose it's hard to say without context though.
bestouff 4y ago
You could use TiVec[1], TiSlice[1] or TiBitset[2] (shameless plug) to have datastructures indexed by the type you need, all checked at compile type. All you need is a From/Into implementation for your chosen index type. 1: https://lib.rs/crates/typed-index-collections https://lib.rs/crates/typed-index-collections 2: https://lib.rs/crates/tibitset https://lib.rs/crates/tibitset
ATsch 4y ago
> Using a usize often seems wasteful for indexing into small arrays and slices The opposite is often the case, as the architecture may not support offsetting a pointer by those sizes, requiring lots of casting in tight loops. Making that explicit makes the programmer aware of this. This is one major reason that C compiler developers want all overflow to be undefined behavior, as it allows them to upgrade for loops that use smaller index types to larger ones.
- staticassertion 4y ago
  That seems easy to optimize out, especially given rust's mutability guarantees. If I have a u8 and I index into an array in a loop the compiler should be free to cast it to a usize above that loop, right? The benefit being you can store much smaller structures, plus in the case of memory unsafety you reduce attacker control. The vast majority of strings I personally use are certainly well under 2^32 bytes, and most are probably under 2^8.
mjw1007 4y ago
Right, better control of the index type can help you say what you mean, and help the typechecker catch more mistakes. If I have an array representing a single-byte lookup table, it would be nice if I could say that the array's type is array-indexed-by-u8 rather than array-indexed-by-usize. Ada did this well: it even lets you declare an array whose index is some enumerated type.
- Ar-Curunir 4y ago
  > _ it even lets you declare an array whose index is some enumerated type._ You can do this in Rust also: make a custom newtype wrapper for your array, and impl Index<CustomEnum> for the wrapped array.

JulianMorrison 4y ago

Re the non-address bits of a pointer: there could be a method `shifted_by_address_width()` that just returns a same-size copy of the data (which remains unmodified) shifted over by the width of an address on the system. That way it's platform agnostic, it's still as wide as the address type, and on systems with no extra bits beyond the address, it's just zero.

mlindner 4y ago

This article feels very wrong. There is so much of exposing the underlying hardware CHERI functionality to the user in everything suggested in this article. It would never fly in Rust. This post feels completely written from the perspective of C-isms rather than from the perspective of Rust. This paragraph in particular: > Fortunately, I believe that Aria's proposal can be adapted such that a Rust-for-CHERI can cope with both pure capability and hybrid modes. In essence, one needs explicit, separate, types for both traditional pointers and capabilities. mut () suffices for the former and a wrapper of some sort, e.g. Cap<mut ()> for the latter. Conceptually this means that mut () is no longer the root of the integer/pointer type system, because one cannot convert Cap<mut ()> to mut () without losing information (the capability's extra bits). My gut feeling is that in practice, most code can treat mut () as the root of the integer/pointer type system, and only code which really cares about capabilities need know of Cap's existence. An additional nice property is that one will be able to write Rust code in a way that can be agnostic about pure capability mode (where size_of::<mut ()>() == size_of::<Cap<mut ()>>()) and hybrid mode (where size_of::<mut ()>() < size_of::<Cap<mut ()>>()). Capabilities should not be visible to users of the Rust language what-so-ever and should only exist in the compiler, and nowhere else. The more correct solution is to just forbid "hybrid" modes, which seem only useful for C projects where there is a lot of legacy code. Something Rust doesn't have. The author also admits as such in their footnote on hybrid mode having "many uses": > In the context of Rust, it's also worth asking whether it's worth making all pointers double width (which, though perhaps small, will undoubtedly have measurable memory and performance costs). After all, most Rust code is safe, and the compiler can guarantee that pointers can't be easily misused for things like buffer overruns. Rather, I see the main utility for a language like Rust being to impose various "sub-process" like compartments, with only relatively small portions of the code needing to use capabilities explicitly. It is perfectly acceptable to have all Rust-on-CHERI systems to have double width pointers in my opinion. This wouldn't affect software on any existing supported platforms. If you're writing new Rust code, for a CHERI system, but not using capabilities, I would wonder why you're using the platform in the first place.

ltratt 4y ago
I agree that you don't want non-CHERI Rust code to have to know about capabilities in any way. However, if you are using Rust for CHERI, you need to have some access to capability functions otherwise, even in pure capability mode, you can't reduce a capability's permissions. For example, if you want to write `malloc` in purecap Rust, you'll probably want to hand out capabilities that point to blocks of memory with bounds that only cover that block: you need some way to say "create a new capability which has smaller bounds than its parent capability." As to the utility of hybrid mode, I politely disagree. For example, is it useful for safe Rust code to always pay the size penalty of capabilities when you've proven at compile-time that they can't misuse the underlying pointers? A very different example is that hybrid mode allows you to meaningfully impose sub-process like compartments (in particular with the `DDC` register, which restricts non-capability code to a subset of the virtual address space; see also the `PCC` register and friends). Personally, I think that this latter technique holds a great deal of potential.

kibwen 4y ago

The OP links Gankra's first article, but don't miss the follow-up where she actually implemented some of these ideas as proof-of-concepts in the stdlib and compiler: https://gankra.github.io/blah/tower-of-weakenings/ https://gankra.github.io/blah/tower-of-weakenings/

dathinab 4y ago

I think there are some problems in the article. The main point of the referenced article by Aria Beingessner is Pointer Provenance. That this also helps out with enabling potential support for CHERI is a nice (and intended) side effect. So yes more is needed for supporting CHERI but: 1. The non-address parts, if and how you can access them are fully CHERI specific. Any such code should (for now) not be Generic Rust but architecture specific extensions. (E.g. live in `std::arch::<somethingCHERI>`) 2. "Rust's integer type hierarchy" There is no type hierarchy in the sense there is in C/C++ as there is no sub-typing of integers (i.e. auto-conversions) in rust. Same is true for pointers so I don't think it makes sense to call `mut ()` the "root of the...". (Also if anything that would be `const ()`). Similar it's more a void-pointer (through not quite the same) then a `uintptr_t`. To quote Aria: "I don’t think Rust needs to define a moral equivalent to intptr_t [...]". 3. The 129th bit is a "secret" implementation detail. So not representing it is intended. if your code relies on somehow detecting if it's set or not to work correctly you are doing something wrong. (Through maybe for testing/asserts, still at most a CHERI specific method under std::arch.) 4. The proposed scratch design for hybrid mode fundamentally doesn't work because of pointer provenance. Pointers need to be build-in types. So in hybrid mode you now would have 4 instead of 2 pointer types. Also given that you would want to use all of them in normal code you also would have 4 instead of 2 reference types. As far as I can tell this is also a messy nightmare to use in C/C++ (Through I have to read into it first). Anyway hybrid mode is as far as I can tell irrelevant for "pure" rust applications and only becomes relevant when linking against non-capability-supporting FFI. EDIT: Provenance is often partially lost at FFI boundary anyway and you probably would want to special type the non-capability pointers and then at the boundary convert them to "normal(capability)" pointers. Only handling "normal" pointers in rust (which all have capabilities when compiled for CHERI).

zozbot234 4y ago
> So in hybrid mode you now would have 4 instead of 2 pointer types. Also given that you would want to use all of them in normal code you also would have 4 instead of 2 reference types. We'd need more pointer types to support other segmented architectures anyway. Real mode x86 has __near, __far and __huge pointer types.

saati 4y ago

> It's important to note that the cast warnings (or errors) are not enough on their own to help make code correct: additional auditing will be required. However, at the very least, this will give programmers a good idea of where to start auditing their code [4]. Some people might choose not to adapt their code, or find it too difficult to do so, but my guess is that most people will choose to do so. I strongly doubt that most people would change their code for an ISA that barely exists.

burntsushi 4y ago
Maybe not. But I might change my code if it makes it easier to reason about its correctness. Especially in unsafe blocks and especially if I don't have to give up anything else (like perf).