4 ms·

If you can indulge a non-Rust point of view, if I'd been faced with this design problem, I think I would have put a "this is a rodata-backed string" bit into St

by rsc 6y ago

If you can indulge a non-Rust point of view, if I'd been faced with this design problem, I think I would have put a "this is a rodata-backed string" bit into String, and then the representation would be {ptr, len, rodata-bit} (or squeeze a bit out of the ptr or len if the space matters). Then "abc" could have type String, and the only difference between let abc1: String = "abc" and let a: String = "a" let abc2: String = a + "bc" would be that (assuming the compiler doesn't get cute) abc1 is pointing at the rodata bytes for "abc" and abc2 is pointing at allocated bytes. (But it can tell them apart so the deallocation is not ambiguous.) It seems like this would have avoided a lot of the ink spilled over &str vs String. I know it's too late now, but was this considered at all, and if so what were the reasons not to adopt it? Thanks.

brundolf 6y ago
I hadn't heard of rodata before, but based on a quick Google, I think what you're describing is similar to Cow<str>. I can't speak to the reasons why this wasn't made the default, but I believe it is at least possible.
- cesarb 6y ago
  It's probably the same reason why IIRC the default Rust String does not have a "small string optimization": it would add a lot of unpredictable branches. And it would be even worse, since unlike the "small string optimization", even small mutations which don't change the size would have to allocate when the original was read-only.
  - pornel 6y ago
    Another problem is that `&str` is more than just "read-only". It also tracks lifetime to prevent use-after-free. A universal owned-or-readonly-borrowed String would be unsafe without also adding reference counting or GC to keep the borrowed-from string alive.
devit 6y ago
Cow<'static, str> is exactly what you are asking for. In general the String type is not very good, and you should use something appropriate to your use of strings for any string that is used more than a constant times in your program, which can be, for instance: - String - Box<[str]> - Cow<'static, str> - Cow<'a, str> - Interned strings - Rc<[str]> or Arc<[str]> - A rope data structure In fact I think putting String in the standard library might have been a mistake since it's almost always a suboptimal choice for anything except string builders in local variables.
- TheGrassyKnoll 6y ago
  COW == Copy on Write Its interesting that certain file systems use COW schemes: https://en.wikipedia.org/wiki/Btrfs
holmium 6y ago
In addition to what the others have said, Rust's `String` is three pointer sized words: `(ptr, len, capacity)`. So, a `String` has 50% of the overhead of a `&str`.
tene 6y ago
When you want a reference that can transparently become an owned value when mutated, Rust has a type Cow that implements this generally. It's an enum that has two variants, Owned and Borrowed. If you mutate a Cow::Borrowed, it first copies the data to a new owned allocation, and replaces itself with Cow::Owned. However, the difference between String and &str has nothing to do with mutability, or whether the data is in a read-only page or not. If you have &mut str, you can mutate the values, and if you don't declare your String as mut, it's not mutable. The difference is that String owns its allocation, whereas &str is a reference to memory that someone else owns. It's exactly the same as Vec vs &[u8], and if you check the source, you'll see that String is just a wrapper around Vec<u8>: https://doc.rust-lang.org/src/alloc/string.rs.html#278-280 https://doc.rust-lang.org/src/alloc/string.rs.html#278-280 The general principal is that if you own the allocation, you can reallocate it to change its size. If you only have a reference to memory that someone else owns, you can't do that. Consider for example Go's slices, which work kind of like you describe, where they point to the original array's memory until someone grows the array, at which time they might or might not make a new allocation. Appending to a Go slice from some inner function can suddenly break code that calls it, because the slice it's operating on suddenly points to new memory. Rust's Big Idea is to make ownership and borrowing more-explicit. Having your default stdlib text type be ambiguous about whether it's owning or borrowing is both weird, and also makes things a lot more awkward to deal with. If you use Cow<str>, you'll see that its API declares that it borrows from some source, and can't outlive that source. That's fine if what it's borrowing from is static text in the binary, but that really constrains what you can do with a string that you've dynamically allocated. Just like all other data structures, having a distinction between an owned value and a reference to the value is very useful. It's easy to build a variety of shared or ambiguous ownership structures on top of owned values and references, but it's much more complicated to go the other direction.
int_19h 6y ago
Where would abc allocate the bytes from, if a is in rodata?

by rsc 6y ago

brundolf 6y ago

I hadn't heard of rodata before, but based on a quick Google, I think what you're describing is similar to Cow<str>. I can't speak to the reasons why this wasn't made the default, but I believe it is at least possible.

cesarb 6y ago
It's probably the same reason why IIRC the default Rust String does not have a "small string optimization": it would add a lot of unpredictable branches. And it would be even worse, since unlike the "small string optimization", even small mutations which don't change the size would have to allocate when the original was read-only.
- pornel 6y ago
  Another problem is that `&str` is more than just "read-only". It also tracks lifetime to prevent use-after-free. A universal owned-or-readonly-borrowed String would be unsafe without also adding reference counting or GC to keep the borrowed-from string alive.

devit 6y ago

Cow<'static, str> is exactly what you are asking for. In general the String type is not very good, and you should use something appropriate to your use of strings for any string that is used more than a constant times in your program, which can be, for instance: - String - Box<[str]> - Cow<'static, str> - Cow<'a, str> - Interned strings - Rc<[str]> or Arc<[str]> - A rope data structure In fact I think putting String in the standard library might have been a mistake since it's almost always a suboptimal choice for anything except string builders in local variables.

TheGrassyKnoll 6y ago
COW == Copy on Write Its interesting that certain file systems use COW schemes: https://en.wikipedia.org/wiki/Btrfs

holmium 6y ago

In addition to what the others have said, Rust's `String` is three pointer sized words: `(ptr, len, capacity)`. So, a `String` has 50% of the overhead of a `&str`.

tene 6y ago

When you want a reference that can transparently become an owned value when mutated, Rust has a type Cow that implements this generally. It's an enum that has two variants, Owned and Borrowed. If you mutate a Cow::Borrowed, it first copies the data to a new owned allocation, and replaces itself with Cow::Owned. However, the difference between String and &str has nothing to do with mutability, or whether the data is in a read-only page or not. If you have &mut str, you can mutate the values, and if you don't declare your String as mut, it's not mutable. The difference is that String owns its allocation, whereas &str is a reference to memory that someone else owns. It's exactly the same as Vec vs &[u8], and if you check the source, you'll see that String is just a wrapper around Vec<u8>: https://doc.rust-lang.org/src/alloc/string.rs.html#278-280 https://doc.rust-lang.org/src/alloc/string.rs.html#278-280 The general principal is that if you own the allocation, you can reallocate it to change its size. If you only have a reference to memory that someone else owns, you can't do that. Consider for example Go's slices, which work kind of like you describe, where they point to the original array's memory until someone grows the array, at which time they might or might not make a new allocation. Appending to a Go slice from some inner function can suddenly break code that calls it, because the slice it's operating on suddenly points to new memory. Rust's Big Idea is to make ownership and borrowing more-explicit. Having your default stdlib text type be ambiguous about whether it's owning or borrowing is both weird, and also makes things a lot more awkward to deal with. If you use Cow<str>, you'll see that its API declares that it borrows from some source, and can't outlive that source. That's fine if what it's borrowing from is static text in the binary, but that really constrains what you can do with a string that you've dynamically allocated. Just like all other data structures, having a distinction between an owned value and a reference to the value is very useful. It's easy to build a variety of shared or ambiguous ownership structures on top of owned values and references, but it's much more complicated to go the other direction.

int_19h 6y ago

Where would abc allocate the bytes from, if a is in rodata?