7 ms·

> - Confusing different integer types is much harder not only because the type checker will complain, but also because you don't really come across anything oth

> - Confusing different integer types is much harder not only because the type checker will complain, but also because you don't really come across anything other than usize/isize unless it's a field in a protocol or something. For me it's the opposite: usize is only used for indexing arrays, isize only used for memory offsets (which is a really niche use case outside of low level code), and most application code should be written with u32, u64, i32 and i64. Why is that? Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic. So you often need to index something based on an u32 (or something) and then there's a cast. How do avoid it? Well first off, your own types can implement Index<u32> instead of Index<usize>. But better yet, you can have an Idx type that represents an index, and then implement Index<Idx>. That's seen for example in the petgraph API [0], you can index an graph with either a node index [1] or an edge index [2] [0] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.... [1] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html#impl-Index%3CNodeIndex%3CIx%3E%3E https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.... [2] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html#impl-Index%3CEdgeIndex%3CIx%3E%3E https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph....

proto_lambda 4y ago
> Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic. Fair point, but it really only matters if the overflow is handled gracefully. If it's handled as an unrecoverable error condition, it often 1. doesn't happen during normal operation and 2. can be invoked by a determined attacker, regardless of the exact size. Sure, the exact characteristics might be different, but in the end it doesn't make much of a difference. Petgraph has a great API, and in general using newtypes over integers for collection indices is a great way to avoid another logic bug: mixing up indices for different collections.
- nextaccountic 4y ago
  Defining a newtype avoids confusing indexes of, say, a graph and a Vec, but doesn't avoid confusing indexes of two graphs Rust has a design pattern called lifetime branding (also called generativity), which uses phony lifetimes to prevent, at compile time, confusing indexes of two separate collections of the same type. This can also enable disabling out of bounds checks without triggering UB (because, with branding, we can be sure that the index is on bounds at the moment of creating it; essentially we move bounds check from the indexing time to the index creation time) Here's an earlier mention of that [0] (7 years ago), and here's a crate from 3 years ago, indexing [1] but I'm not sure about recent developments on that. Now, petgraph doesn't use branding for its index types, so if you have two graphs you can confuse their indexes. On the other hand, petgraph was specifically designed so that you can reuse the node and edge numbering across many graphs (so that if you have a subgraph for example, the nodes and edges share the same ids), in this situation it's kind of hard to use branding There's another pattern for not confusing index types which is to make different index types for each different collection and make the collection work only with that type; this is done eg. in typed-indexed-collections [2] - but it doesn't use branding so two collections with same index type have interchangeable indexes Anyway right now this stuff is mostly folklore but I wish it were more used. [0] https://www.reddit.com/r/rust/comments/3oo0oe/sound_unchecked_indexing_with_lifetimebased_value/ https://www.reddit.com/r/rust/comments/3oo0oe/sound_unchecke... [1] https://github.com/bluss/indexing https://github.com/bluss/indexing https://docs.rs/indexing/0.4.1/indexing/ https://docs.rs/indexing/0.4.1/indexing/ https://crates.io/crates/typed-index-collections https://crates.io/crates/typed-index-collections https://www.reddit.com/r/rust/comments/hr6xcu/announcing_typedindexcollections_010_slice_and/ https://www.reddit.com/r/rust/comments/hr6xcu/announcing_typ...
marcosdumay 4y ago
I'm not sure it's the best policy, but I tend to represent every counter, of anything as usize. It doesn't matter if it's a memory map. I imagine that applications with less addressing space also have less of every other resource, and can count less of everything.
- nextaccountic 4y ago
  > I imagine that applications with less addressing space also have less of every other resource, and can count less of everything. That's a fair point, and I've seen here on HN a design approach that everything should have a limit (because programs often explode in unexpected ways, such as degraded performance, when operating outside the programmer's expectations) But, usize doesn't necessarily capture the right limit, so.. I'm not sure
  - marcosdumay 4y ago
    Well, yes. I favor your program blowing up on integer overflow too, so it's very predictable how it behaves. And if it's not overflow that is the concern here, well, it's not solved by going with another type either. Almost all the time, usize has a convenient limit that is much larger than needed. On the few exceptions, yes, you have to design your types and deal with type conversions.
- yakubin 4y ago
  It's sometimes good to use a smaller size. Many string/dynamic array libraries e.g. will use 32-bit types for size and capacity even on 64-bit platforms to conserve space, and by that virtue also be more cache-efficient. I think the Zig compiler also explicitly uses 32-bit types in a lot of places, even though it's targeted mostly for 64-bit platforms.
saghm 4y ago
> So you often need to index something based on an u32 (or something) and then there's a cast. How do avoid it? I imagine the nuance here is in the "or something", since that could imply something signed or larger than 32 bits, but for that specific case, I feel like a large amount of code that people write nowadays can be comfortably assumed not to need to run on 16-bit processors. Although `usize `only implements `TryFrom<u32>` and not `From<u32>`, even the standard library documentation for Rust simplifies things by saying that `usize` will be either 4 or 8 bytes (https://doc.rust-lang.org/std/primitive.usize.html https://doc.rust-lang.org/std/primitive.usize.html), although I understand this isn't a hard contract and could change in the future. I guess it depends a bit on the use case, but I think that most of the time just using `try_from` and panicking with an appropriate error message on error would be fine, and if you want to be extra sure, you can always use something like `const_assert!` (https://docs.rs/static_assertions/1.1.0/static_assertions/macro.const_assert.html https://docs.rs/static_assertions/1.1.0/static_assertions/ma...) with the `BITS` constant (https://doc.rust-lang.org/std/primitive.usize.html#associatedconstant.BITS https://doc.rust-lang.org/std/primitive.usize.html#associate...) to catch it at compile time to be extra certain.
- nextaccountic 4y ago
  Oh, u32 is generally fine exactly because 16 bits is so niche. But you still have a cast between u32 and usize (and .into() isn't that much better). Casts or conversions on each collection access makes code hard to read So if your application can work with u32 everywhere (on either 32 bits or 64 bits platforms), you should use u32 and impl Index<u32> on your collection, to avoid the cast. It's only if you need the less common use case of u32 on 32 bits platforms, u64 in 64 bits platforms that you should bother with usize
  - phkahler 4y ago
    >> So if your application can work with u32 everywhere (on either 32 bits or 64 bits platforms), you should use u32 and impl Index<u32> on your collection, to avoid the cast. This seems strange. Why does Rust not automatically use whatever size index you hand it?
stouset 4y ago
Also what’s fun is that when you have usize and u64, which are both the same size on 64-bit systems, you might instinctively cast between them. But this could truncate the u64 on 32-bit systems, and it could truncate the usize on future 128-bit systems! So neither type implements From/Into the other, the best you get are TryFrom/TryInto where you either have to handle an error or discard it after “proving” overflow can’t happen. This is unambiguously the correct behavior, but it is sort of frustrating at times.
AnimalMuppet 4y ago
> For me it's the opposite: usize is only used for indexing arrays, isize only used for memory offsets (which is a really niche use case outside of low level code), and most application code should be written with u32, u64, i32 and i64. Why is that? Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic. While that may make it platform agnostic, doesn't that make it non-time-agnostic? What I mean is, I lived through the 16-to-32-bit transition. If Rust had been around then, and you wrote code that used u16, and you tried to use it now, you would probably bump against that 16-bit limit. The platforms grew with time, but the code didn't. We're starting into the 32-to-64-bit transition (we've been slowly easing into it for years, in fact). Don't lock yourself into 32-bit code (unless the algorithm requires it). You're locking yourself out of the future.
- nextaccountic 4y ago
  64 bits is huge! We won't be needing more than 64 bits to address memory anytime soon. In fact, current processors use only 48 bits, leaving the rest unused / up for grabs with flags. And note that the usize that is the index of Vec (and arrays, etc) is not an address but just a counter; it can count up to more than 10^19 elements. You probably won't be alive to see Vecs grow that big, even a Vec<u8>. The most interesting platform with 128 bits pointers is CHERI, but its address space fits on 64 bits; the other 64 bits is used for a tag, to validate pointers (and prevent certain kinds of buggy memory access that can lead to corruption). If and when Rust support CHERI it should probably do so with usize = 64 bits, because that's how much you need to count to address a Vec<u8> over the whole address space.
  - phkahler 4y ago
    I've often thought a 48bit architecture might be ideal. 256TB of address space. Use 48bit posits instead of double precision floating point for similar precision. A 48bit pixel could be 16bit rgb or 12bit rgba. I've also tried to re-encode the RISC-V instruction set and I think 24-bit opcodes with immediate data following them would be better than what they've got. Here's to 48bit CPUs! Hahaha.
  - 4y ago

proto_lambda 4y ago

> Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic. Fair point, but it really only matters if the overflow is handled gracefully. If it's handled as an unrecoverable error condition, it often 1. doesn't happen during normal operation and 2. can be invoked by a determined attacker, regardless of the exact size. Sure, the exact characteristics might be different, but in the end it doesn't make much of a difference. Petgraph has a great API, and in general using newtypes over integers for collection indices is a great way to avoid another logic bug: mixing up indices for different collections.

nextaccountic 4y ago
Defining a newtype avoids confusing indexes of, say, a graph and a Vec, but doesn't avoid confusing indexes of two graphs Rust has a design pattern called lifetime branding (also called generativity), which uses phony lifetimes to prevent, at compile time, confusing indexes of two separate collections of the same type. This can also enable disabling out of bounds checks without triggering UB (because, with branding, we can be sure that the index is on bounds at the moment of creating it; essentially we move bounds check from the indexing time to the index creation time) Here's an earlier mention of that [0] (7 years ago), and here's a crate from 3 years ago, indexing [1] but I'm not sure about recent developments on that. Now, petgraph doesn't use branding for its index types, so if you have two graphs you can confuse their indexes. On the other hand, petgraph was specifically designed so that you can reuse the node and edge numbering across many graphs (so that if you have a subgraph for example, the nodes and edges share the same ids), in this situation it's kind of hard to use branding There's another pattern for not confusing index types which is to make different index types for each different collection and make the collection work only with that type; this is done eg. in typed-indexed-collections [2] - but it doesn't use branding so two collections with same index type have interchangeable indexes Anyway right now this stuff is mostly folklore but I wish it were more used. [0] https://www.reddit.com/r/rust/comments/3oo0oe/sound_unchecked_indexing_with_lifetimebased_value/ https://www.reddit.com/r/rust/comments/3oo0oe/sound_unchecke... [1] https://github.com/bluss/indexing https://github.com/bluss/indexing https://docs.rs/indexing/0.4.1/indexing/ https://docs.rs/indexing/0.4.1/indexing/ https://crates.io/crates/typed-index-collections https://crates.io/crates/typed-index-collections https://www.reddit.com/r/rust/comments/hr6xcu/announcing_typedindexcollections_010_slice_and/ https://www.reddit.com/r/rust/comments/hr6xcu/announcing_typ...

marcosdumay 4y ago

I'm not sure it's the best policy, but I tend to represent every counter, of anything as usize. It doesn't matter if it's a memory map. I imagine that applications with less addressing space also have less of every other resource, and can count less of everything.

nextaccountic 4y ago
> I imagine that applications with less addressing space also have less of every other resource, and can count less of everything. That's a fair point, and I've seen here on HN a design approach that everything should have a limit (because programs often explode in unexpected ways, such as degraded performance, when operating outside the programmer's expectations) But, usize doesn't necessarily capture the right limit, so.. I'm not sure
- marcosdumay 4y ago
  Well, yes. I favor your program blowing up on integer overflow too, so it's very predictable how it behaves. And if it's not overflow that is the concern here, well, it's not solved by going with another type either. Almost all the time, usize has a convenient limit that is much larger than needed. On the few exceptions, yes, you have to design your types and deal with type conversions.
yakubin 4y ago
It's sometimes good to use a smaller size. Many string/dynamic array libraries e.g. will use 32-bit types for size and capacity even on 64-bit platforms to conserve space, and by that virtue also be more cache-efficient. I think the Zig compiler also explicitly uses 32-bit types in a lot of places, even though it's targeted mostly for 64-bit platforms.

saghm 4y ago

> So you often need to index something based on an u32 (or something) and then there's a cast. How do avoid it? I imagine the nuance here is in the "or something", since that could imply something signed or larger than 32 bits, but for that specific case, I feel like a large amount of code that people write nowadays can be comfortably assumed not to need to run on 16-bit processors. Although `usize `only implements `TryFrom<u32>` and not `From<u32>`, even the standard library documentation for Rust simplifies things by saying that `usize` will be either 4 or 8 bytes (https://doc.rust-lang.org/std/primitive.usize.html https://doc.rust-lang.org/std/primitive.usize.html), although I understand this isn't a hard contract and could change in the future. I guess it depends a bit on the use case, but I think that most of the time just using `try_from` and panicking with an appropriate error message on error would be fine, and if you want to be extra sure, you can always use something like `const_assert!` (https://docs.rs/static_assertions/1.1.0/static_assertions/macro.const_assert.html https://docs.rs/static_assertions/1.1.0/static_assertions/ma...) with the `BITS` constant (https://doc.rust-lang.org/std/primitive.usize.html#associatedconstant.BITS https://doc.rust-lang.org/std/primitive.usize.html#associate...) to catch it at compile time to be extra certain.

nextaccountic 4y ago
Oh, u32 is generally fine exactly because 16 bits is so niche. But you still have a cast between u32 and usize (and .into() isn't that much better). Casts or conversions on each collection access makes code hard to read So if your application can work with u32 everywhere (on either 32 bits or 64 bits platforms), you should use u32 and impl Index<u32> on your collection, to avoid the cast. It's only if you need the less common use case of u32 on 32 bits platforms, u64 in 64 bits platforms that you should bother with usize
- phkahler 4y ago
  >> So if your application can work with u32 everywhere (on either 32 bits or 64 bits platforms), you should use u32 and impl Index<u32> on your collection, to avoid the cast. This seems strange. Why does Rust not automatically use whatever size index you hand it?

stouset 4y ago

Also what’s fun is that when you have usize and u64, which are both the same size on 64-bit systems, you might instinctively cast between them. But this could truncate the u64 on 32-bit systems, and it could truncate the usize on future 128-bit systems! So neither type implements From/Into the other, the best you get are TryFrom/TryInto where you either have to handle an error or discard it after “proving” overflow can’t happen. This is unambiguously the correct behavior, but it is sort of frustrating at times.

AnimalMuppet 4y ago

> For me it's the opposite: usize is only used for indexing arrays, isize only used for memory offsets (which is a really niche use case outside of low level code), and most application code should be written with u32, u64, i32 and i64. Why is that? Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic. While that may make it platform agnostic, doesn't that make it non-time-agnostic? What I mean is, I lived through the 16-to-32-bit transition. If Rust had been around then, and you wrote code that used u16, and you tried to use it now, you would probably bump against that 16-bit limit. The platforms grew with time, but the code didn't. We're starting into the 32-to-64-bit transition (we've been slowly easing into it for years, in fact). Don't lock yourself into 32-bit code (unless the algorithm requires it). You're locking yourself out of the future.

nextaccountic 4y ago
64 bits is huge! We won't be needing more than 64 bits to address memory anytime soon. In fact, current processors use only 48 bits, leaving the rest unused / up for grabs with flags. And note that the usize that is the index of Vec (and arrays, etc) is not an address but just a counter; it can count up to more than 10^19 elements. You probably won't be alive to see Vecs grow that big, even a Vec<u8>. The most interesting platform with 128 bits pointers is CHERI, but its address space fits on 64 bits; the other 64 bits is used for a tag, to validate pointers (and prevent certain kinds of buggy memory access that can lead to corruption). If and when Rust support CHERI it should probably do so with usize = 64 bits, because that's how much you need to count to address a Vec<u8> over the whole address space.
- phkahler 4y ago
  I've often thought a 48bit architecture might be ideal. 256TB of address space. Use 48bit posits instead of double precision floating point for similar precision. A 48bit pixel could be 16bit rgb or 12bit rgba. I've also tried to re-encode the RISC-V instruction set and I think 24-bit opcodes with immediate data following them would be better than what they've got. Here's to 48bit CPUs! Hahaha.
- 4y ago