18 ms·

What is `Box<str>` and how is it different from `String` in Rust?

215 points by asimpletune 4y ago

sampo 4y ago
Title is: What is Box<str> and how is it different from String in Rust?
- dang 4y ago
  Fixed now. Thanks!
FullyFunctional 4y ago
This is missing a conversation about https://lib.rs/crates/compact_str https://lib.rs/crates/compact_str (and a few alternatives like it). TL;DR: String takes the space of three pointers, that is, 24 bytes on 64-bit archs. compact_str fits up to 24 byte strings in the same space and reverts to String for longer strings. ADD: that is, avoids heap allocation for those, unlike both Box<str> and String.
- tialaramex 4y ago
  Box<str> is still going to be smaller if you know how big the text is because (unlike CompactString and String) it doesn't need to carry a capacity value. In exchange of course you can't append things to it (without re-allocating) CompactString is a very clever† SSO implementation, and I'll remember it is there if I run into a situation where it might help but I firmly agree with Rust's choice not to implement the SSO optimisation in the standard library's String type. † Storing 23 UTF-8 codepoints as one of several representations in a 24 byte data structure makes sense, you can see how to write a fairly safe SSO optimisation for Rust which does that, but the CompactString scheme relies on the fact Rust's strings are by definition UTF-8 encoded to squeeze the discriminant into the same space as the last possible byte of an actual UTF-8 string, so it can store a 24 byte value like "ABCDEFGHIJKLMNOPQRSTUVWX" inline despite also distinguishing the case where it needs a heap pointer for larger strings. That's very clever.
  - rtfeldman 4y ago
    > I firmly agree with Rust's choice not to implement the SSO optimisation in the standard library's String type. Out of curiosity, why is that? I don't know much about how or why that decision was made, but I'm curious.
    - lifthrasiir 4y ago
      SSO means that pretty every string operation has multiple code paths, which can be highly unpredictable. Basically it is a trade-off between memory usage and performance, and the standard library is not really a good place to make that trade-off. By comparison many C++ codes (still) copy strings all over the place for no good reason, so SSO in the standard library has a much greater appeal.
- pornel 4y ago
  A nice thing is that all string types have &str as the lowest common denominator, so even if you use SSO or on-stack or any other fancy string type, it's automatically compatible with almost everything.
dochtman 4y ago
The tl;dr doesn't quite make sense to me. To me the core difference is that a Box<str> takes one less word on the stack, because by virtue of the str being immutable it doesn't need to track the capacity of the allocation as distinct from the length. This is analogous to Box<[u8]> vs Vec<u8> (and in fact those are the same data types except for the guarantee of valid UTF-8).
- tialaramex 4y ago
  One notable difference is that ToOwned for &str gives you a String, whereas ToOwned for &[u8] gives you a [u8] by cloning the slice you have. In fact all four standard library types that are ToOwned without invoking Clone are more or less strings (str, CStr, OsStr, Path)
  - afdbcreid 4y ago
    What? No: https://doc.rust-lang.org/1.61.0/src/alloc/slice.rs.html#854-878 https://doc.rust-lang.org/1.61.0/src/alloc/slice.rs.html#854.... ToOwned for [T] gives you Vec<T>.
    - tialaramex 4y ago
      Huh. TIL Actually now that I think harder about this, WTF did I think was really going on here before. If we clone the slice to make an array, where does the array go? We don't know how big that array is, so we can't put it on the stack. Yeah, that was crazy. Thanks for pointing it out.
  - kzrdude 4y ago
    Perspective on ToOwned: It's just Clone with an extension to a number of DST types that can't be Clone themselves. They are dynamically sized types, hence no surprise that they are string-like.
- tines 4y ago
  C++ programmer here: which one guarantees valid utf8, and why would a primitive container make guarantees about the values it's storing?
terhechte 4y ago
I recently gave a Rust workshop to Kotlin and Swift developers. Strings in Rust are a really, really difficult topic for complete newcomers because they're understood as a basic type whereas in Rust they require having read half the Rust book to grasp. Consider: I can teach a lot of Rust basic with `usize`. Defining funcions, calling functions, enums because they're `Copy` and because there's only one type. String requires knowing about &str which requires knowing about deref which requires knowing about (&String -> &str), it also requires understanding lifetimes, moving, heap and stack, cloning. Then, if you want to work with the file system you also need to understand Paths, OsString and AsRef. With Kotlin and Swift, for all these things, you really just need one type, String, and you handle it just like usize. It is really a bid of a hurdle for new developers coming from higher level languages (especially if they just give it a quick try).
- nicoburns 4y ago
  On the plus side, String makes a really good example to explain ownership, moving, stack vs heap, etc. All of which you need at least a basic understanding of to do anything non-trivial in Rust. I kind of feel like it goes without saying that Rust isn't ideal for beginners. For developers who already have a good knowledge of other languages I feel like learning about these things shouldn't be a problem, as becoming familiar with these concepts is one of the main benefits of learning Rust.
  - smaddox 4y ago
    > I kind of feel like it goes without saying that Rust isn't ideal for beginners. I think that depends on, first, what the goal is, and second, what you're comparing to. It think Rust is easier on beginners, in many ways, than C. And C is easier on beginners, in many ways, than assembly or machine code. But if you want to really understand computer programming, starting at machine code or at least assembly isn't a crazy way to start.
    - tialaramex 4y ago
      Beginning with machine code for some simple architecture (maybe RISC-V these days?) might be one good route in. I can also see (having experienced it myself, albeit I already knew C etc. these were not requirements and many of my classmates did not) beginning with a pure functional language where all the practicalities are abstracted entirely. Today the University where I learned this begins with Java, which I am confident is the wrong choice, but the person who part-designed their curriculum, and is a friend, disagrees with me and he's the one getting paid to teach them.
sirwhinesalot 4y ago
It's unfortunate that strings are badly named in rust. They got that better with Path and PathBuf. str is fixed size, like a Java String String is growable, like a Java StringBuilder After that, we get into memory ownership, with &str not owning memory, and Box<str> owning memory, but you rarely need the latter, so it's really &str vs String that you need to care about. EDIT: changed immutable to fixed and mutable to growable to better reflect the real difference, though typically you almost always use immutable &str and &mut String. I thank the commenters below for pointing it out, I don't want to make the problem even more confusing than it already is.
- marcosdumay 4y ago
  > but you rarely need the latter AFAIK, it's because people go with String when what they actually mean is Box<str>. Since they have similar costs, nobody ever sees the need to change it, and the String type does have a much better name. But the need is there all the time. People just satisfy it differently.
  - sirwhinesalot 4y ago
    I think it's mainly because unlike Java, where a StringBuilder is effectively an optimisation over concatenating Strings, in Rust managing that memory would be a total pain, so you tend to keep the mutable thing around. Once that happens, Box<str> becomes kinda unnecessary. There are many cases where it would be the correct type, for example reading from a file in a read-only manner, but most of the time you're going to be doing something to that text, so it makes more sense to just load it up as a String already and avoid the unnecessary copy. Either way, it's mostly a naming problem. &str/String sucks :(
  - afdbcreid 4y ago
    It's not just the naming: I prefer String if I don't care about the data structure size, since converting a String to Box<str> can be costly (allocation and copy if the capacity isn't exact) and carrying a capacity is free pretty much all of the time.
- Arnavion 4y ago
the__alchemist 4y ago
I'm working on a PC-based configuration for a drone flight controller. PC-side is std Rust with a stack available. Firmware is `no-std`, running on a microcontroller. It has waypoints you can program when connected to a PC using USB. They have names that need to be represented as some sort of string. I'm using `u8` arrays for the strings on both sides; seems the easiest to serialize, and Rust has `str::from_utf8` etc to handle conversion to/from the UI. `String` is unsupported on the MCU side since there's no allocation. I find this low-level approach ergonomic given it's easy to [de]serialize over USB.
OJFord 4y ago
If OP is here, then in this listing: let boxed_str: Box<str> = "hello".into(); println!("size of boxed_str on stack: {}", std::mem::size_of_val(&boxed_str)); let s = String::from("hello!"); println!("size of string on stack: {}", std::mem::size_of_val(&s)); I know it's not the point and doesn't make a difference, but you might want to make the two 'strings' the same (not with & without '!'), just to be clearer.
School-Cotton 4y ago
This might clarify the situation, for C or C++ folks: // heap-allocated, fixed-size struct BoxStr { unsigned length; // INVARIANT: this points to a heap allocation of length bytes, and is valid utf8 unsigned char *data; } // heap-allocated, resizable struct String { unsigned length; unsigned capacity; // INVARIANT: heap allocation of capacity bytes, the first length of which are valid utf8 unsigned char *data; } Of course you could resize BoxStr, but only by reallocating `data` to the exact desired length every time, which will kill your asymptotic complexity.
- tylerhou 4y ago
  Is your first example really equivalent to Box<str>? I would have expected something like using BoxStr = std::unique_ptr<Str>; where Str is defined as struct Str { size_t len; char data[]; }; The difference is that the len is stored on the heap, and the data is stored inline with the length. Unfortunately C++ does not support flexible array members so this syntax is not actually valid. Edit: Never mind, after reading the article Rust does use the above representation because Box holds a “fat” pointer to str, which stores it’s length on the stack. So BoxStr is the correct equivalent, because &[u8] is not equivalent to u8*, it’s equivalent to std::span<u8>.
  - steveklabnik 4y ago
    Your parent is correct, the length is stored alongside the pointer, not on the heap with its data. This is true for any "dynamically sized type," not just Box<str>. &str is also a (pointer, length) pair, for example.
codedokode 4y ago
Is there official documentation about what `str` (without an ampersand) is? For example, documentation [1] says that `str` is a "string slice" (without explaining what "string slice" mean), and then goes on with description of &str. And a book on Rust [2] says: > A string slice is a reference to part of a String This seems wrong, because &str can reference static strings which are not String. And if str, or "string slice" is a "reference", then &str is a reference to a reference? And later: > The type that signifies “string slice” is written as &str But the documentation said that "string slice" is str, not &str. Also, I wonder, what do square brackets mean when they are used without an ampersand (as s[0..2] instead of &s[0..2])? Also, is an ampersand in &str the same as an ampersand in &u8 (meaning an immutable reference to u8) or does it have other meaning? [1] https://doc.rust-lang.org/std/primitive.str.html https://doc.rust-lang.org/std/primitive.str.html [2] https://doc.rust-lang.org/book/ch04-03-slices.html#string-slices https://doc.rust-lang.org/book/ch04-03-slices.html#string-sl...
- deleted 4y ago
  [deleted]
- LegionMammal978 4y ago
  > Is there official documentation about what `str` (without an ampersand) is? For example, documentation [1] says that `str` is a "string slice" (without explaining what "string slice" mean), and then goes on with description of &str. A `str` is really just a `[u8]` with extra semantics. Thus, a `&str` is really a `&[u8]`, a `&mut str` is a `&mut [u8]`, a `Box<str>` is a `Box<[u8]>`, etc. So we call it a "string slice", since it mostly acts like a regular `[T]` slice. In general, the term "slice" can either refer to the unsized type `[T]` or the reference `&[T]`/`&mut [T]` interchangeably. You could also call the latter a "slice reference" where the distinction is important; e.g., a `Box<[T]>` would be a "boxed slice", while `Box<&[T]>` would be a "boxed slice reference" or "boxed reference to a slice". But most of the time, the correct meaning can be inferred from context. > Also, I wonder, what do square brackets mean when they are used without an ampersand (as s[0..2] instead of &s[0..2])? `s[0..2]` is a place expression that refers to the raw `str` subslice. But since `str` is an unsized type [0], it cannot appear on its own; it must appear behind some reference type. Thus, `&s[0..2]` creates a `&str`, and `&mut s[0..2]` creates a `&mut str`. However, the ampersand isn't always necessary: you can write `s[0..2].to_owned()` to use the `str` as a method receiver, which implicitly creates a reference. [0] https://doc.rust-lang.org/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait https://doc.rust-lang.org/book/ch19-04-advanced-types.html#d...
jez 4y ago
Do any of the string types in the Rust standard library implement the same sort of small string optimization that C++ libraries implement for std::string? (explained here[1]) Some quick searching turned up a few rust-lang internals posts and GitHub issues, but it was hard to see whether anything came of them. I understand that it’s probably possible to implement a comparable String API in a crate that uses small string optimizations, but being able to avoid a dedicated crate makes interoperability with other libraries much easier. [1] https://tc-imba.github.io/posts/cpp-sso/ https://tc-imba.github.io/posts/cpp-sso/
- steveklabnik 4y ago
  Rust's standard library strings cannot because of a specific API, as_mut_vec, which is incompatible with the internal representation necessary to do SSO.
  - mwcampbell 4y ago
    Do you think including this API was a mistake?
    - steveklabnik 4y ago
      No. There’s a discussion below about SSO. I generally agree with it. It’s a trade off, not a universal improvement. Keeping things basic for the basic case is a good thing.
      - mwcampbell 4y ago
        Yet the standard library's HashMap and BTreeMap are very sophisticated. IMO, the more optimizations are built into the standard library, the more effectively Rust can achieve its goal of empowering everyone to build reliable and efficient software. The as_mut_vec function doesn't even really prevent Rust's standard String type from implementing SSO. That function, which is already unsafe, could just come with a warning in the documentation that calling it will force the string into its heap-allocated form. But no, I don't care enough about this to do the work myself, so I'll say no more.
Thorentis 4y ago
This type of complexity is a huge downside for a language. A modern language, designed with all the prior knowledge we now have about language design, should not be this complicated for something as essential as a String. For this reason, I do not like Rust. For similar reasons, I do not like Go. And I'm extremely disappointed that despite being developed so recently, they make these same mistakes. I honestly think Python is one of the best languages (note I am strictly talking about the language, let's put aside performance). If there was work done to make it compilable I would be very happy. But even then, for my own uses this isn't an issue. I just wish there was something as fantastic in the system language space.
- saghm 4y ago
  To be fair, Python had to make a _ton_ of breaking changes to the way their strings worked that arguably are still reverberating in the industry (python2 is still not fully gone, although it's finally starting to fade away). I'm also not convinced that it's possible to differentiate the performance aspects from the way that Rust does strings; unless you're willing to just sacrifice a decent amount of performance by default, I'm not sure there's any way to avoid multiple string types. I don't think that the Rust string API is any sort of platonic ideal for a systems language; I just haven't really seen any alternatives to needing both a mutable, heap-modified string type alongside an immutable reference string type that doesn't require allocation to create. C++ has `std::string_view` alongside `std::string`, Java has both `String` and `StringBuffer`; I think the pain in Rust has a lot more to do with the ambiguity when just saying the word "string" out loud and some confusion about how `str` essentially requires using some sort of reference, making `&String` a weird type that 99% of the time you ignore and the other 1% you need to work around due to some type inference in a closure or something.
- unclad5968 4y ago
  What's the alternative? If there was just one string type, "str", then you would have to implement your own "String" anyway for any strings you intend to manipulate. Computers are complex. Sometimes you need a heap allocated vector of bytes, and sometimes you need a heap allocated array of bytes.
jollybean 4y ago
I really loathe to read this. I love the borrow checker, but I despise arbitrary complexity. I feel that Rust is far more complicated than it needs to be, and that this is our 'first attempt' with borrowing and lifetimes, and that there is an 'easier way'. Engineers love to solve problems, if we can solve the 'life time' issue, we 'feel' very accomplished. And I mean literally a 'feeling'. We are not bound to business and customer success in the same way, and so we tend to value one issue over the other due to our instinct. I'm still skeptical about the extra time being worth the added cost of development in so many cases.
hota_mazi 4y ago
Rust just opens up the kimono and exposes bare metal details about strings. A string with quotes goes into the executable. A `String` exists at runtime and is an entire string. An `&str` is a slice of a string: it points to an existing string for a certain number of characters. Not all programs or programmers require this amount of knowledge, but if you want to understand programming at a deeper level, Rust is a great tool to dive into the deep end to understand complex topics about programming languages.
n8henrie 4y ago
Many thanks to OP for some real-life examples of using rust-lldb! I keep wanting to learn to use it, and I think this was really helpful.

215 points by asimpletune 4y ago

sampo 4y ago

Title is: What is Box<str> and how is it different from String in Rust?

dang 4y ago
Fixed now. Thanks!

FullyFunctional 4y ago

This is missing a conversation about https://lib.rs/crates/compact_str https://lib.rs/crates/compact_str (and a few alternatives like it). TL;DR: String takes the space of three pointers, that is, 24 bytes on 64-bit archs. compact_str fits up to 24 byte strings in the same space and reverts to String for longer strings. ADD: that is, avoids heap allocation for those, unlike both Box<str> and String.

tialaramex 4y ago
Box<str> is still going to be smaller if you know how big the text is because (unlike CompactString and String) it doesn't need to carry a capacity value. In exchange of course you can't append things to it (without re-allocating) CompactString is a very clever† SSO implementation, and I'll remember it is there if I run into a situation where it might help but I firmly agree with Rust's choice not to implement the SSO optimisation in the standard library's String type. † Storing 23 UTF-8 codepoints as one of several representations in a 24 byte data structure makes sense, you can see how to write a fairly safe SSO optimisation for Rust which does that, but the CompactString scheme relies on the fact Rust's strings are by definition UTF-8 encoded to squeeze the discriminant into the same space as the last possible byte of an actual UTF-8 string, so it can store a 24 byte value like "ABCDEFGHIJKLMNOPQRSTUVWX" inline despite also distinguishing the case where it needs a heap pointer for larger strings. That's very clever.
- rtfeldman 4y ago
  > I firmly agree with Rust's choice not to implement the SSO optimisation in the standard library's String type. Out of curiosity, why is that? I don't know much about how or why that decision was made, but I'm curious.
  - lifthrasiir 4y ago
    SSO means that pretty every string operation has multiple code paths, which can be highly unpredictable. Basically it is a trade-off between memory usage and performance, and the standard library is not really a good place to make that trade-off. By comparison many C++ codes (still) copy strings all over the place for no good reason, so SSO in the standard library has a much greater appeal.
pornel 4y ago
A nice thing is that all string types have &str as the lowest common denominator, so even if you use SSO or on-stack or any other fancy string type, it's automatically compatible with almost everything.

dochtman 4y ago

The tl;dr doesn't quite make sense to me. To me the core difference is that a Box<str> takes one less word on the stack, because by virtue of the str being immutable it doesn't need to track the capacity of the allocation as distinct from the length. This is analogous to Box<[u8]> vs Vec<u8> (and in fact those are the same data types except for the guarantee of valid UTF-8).

tialaramex 4y ago
One notable difference is that ToOwned for &str gives you a String, whereas ToOwned for &[u8] gives you a [u8] by cloning the slice you have. In fact all four standard library types that are ToOwned without invoking Clone are more or less strings (str, CStr, OsStr, Path)
- afdbcreid 4y ago
  What? No: https://doc.rust-lang.org/1.61.0/src/alloc/slice.rs.html#854-878 https://doc.rust-lang.org/1.61.0/src/alloc/slice.rs.html#854.... ToOwned for [T] gives you Vec<T>.
  - tialaramex 4y ago
    Huh. TIL Actually now that I think harder about this, WTF did I think was really going on here before. If we clone the slice to make an array, where does the array go? We don't know how big that array is, so we can't put it on the stack. Yeah, that was crazy. Thanks for pointing it out.
- kzrdude 4y ago
  Perspective on ToOwned: It's just Clone with an extension to a number of DST types that can't be Clone themselves. They are dynamically sized types, hence no surprise that they are string-like.
tines 4y ago
C++ programmer here: which one guarantees valid utf8, and why would a primitive container make guarantees about the values it's storing?

terhechte 4y ago

I recently gave a Rust workshop to Kotlin and Swift developers. Strings in Rust are a really, really difficult topic for complete newcomers because they're understood as a basic type whereas in Rust they require having read half the Rust book to grasp. Consider: I can teach a lot of Rust basic with `usize`. Defining funcions, calling functions, enums because they're `Copy` and because there's only one type. String requires knowing about &str which requires knowing about deref which requires knowing about (&String -> &str), it also requires understanding lifetimes, moving, heap and stack, cloning. Then, if you want to work with the file system you also need to understand Paths, OsString and AsRef. With Kotlin and Swift, for all these things, you really just need one type, String, and you handle it just like usize. It is really a bid of a hurdle for new developers coming from higher level languages (especially if they just give it a quick try).

nicoburns 4y ago
On the plus side, String makes a really good example to explain ownership, moving, stack vs heap, etc. All of which you need at least a basic understanding of to do anything non-trivial in Rust. I kind of feel like it goes without saying that Rust isn't ideal for beginners. For developers who already have a good knowledge of other languages I feel like learning about these things shouldn't be a problem, as becoming familiar with these concepts is one of the main benefits of learning Rust.
- smaddox 4y ago
  > I kind of feel like it goes without saying that Rust isn't ideal for beginners. I think that depends on, first, what the goal is, and second, what you're comparing to. It think Rust is easier on beginners, in many ways, than C. And C is easier on beginners, in many ways, than assembly or machine code. But if you want to really understand computer programming, starting at machine code or at least assembly isn't a crazy way to start.
  - tialaramex 4y ago
    Beginning with machine code for some simple architecture (maybe RISC-V these days?) might be one good route in. I can also see (having experienced it myself, albeit I already knew C etc. these were not requirements and many of my classmates did not) beginning with a pure functional language where all the practicalities are abstracted entirely. Today the University where I learned this begins with Java, which I am confident is the wrong choice, but the person who part-designed their curriculum, and is a friend, disagrees with me and he's the one getting paid to teach them.

sirwhinesalot 4y ago

It's unfortunate that strings are badly named in rust. They got that better with Path and PathBuf. str is fixed size, like a Java String String is growable, like a Java StringBuilder After that, we get into memory ownership, with &str not owning memory, and Box<str> owning memory, but you rarely need the latter, so it's really &str vs String that you need to care about. EDIT: changed immutable to fixed and mutable to growable to better reflect the real difference, though typically you almost always use immutable &str and &mut String. I thank the commenters below for pointing it out, I don't want to make the problem even more confusing than it already is.

marcosdumay 4y ago
> but you rarely need the latter AFAIK, it's because people go with String when what they actually mean is Box<str>. Since they have similar costs, nobody ever sees the need to change it, and the String type does have a much better name. But the need is there all the time. People just satisfy it differently.
- sirwhinesalot 4y ago
  I think it's mainly because unlike Java, where a StringBuilder is effectively an optimisation over concatenating Strings, in Rust managing that memory would be a total pain, so you tend to keep the mutable thing around. Once that happens, Box<str> becomes kinda unnecessary. There are many cases where it would be the correct type, for example reading from a file in a read-only manner, but most of the time you're going to be doing something to that text, so it makes more sense to just load it up as a String already and avoid the unnecessary copy. Either way, it's mostly a naming problem. &str/String sucks :(
- afdbcreid 4y ago
  It's not just the naming: I prefer String if I don't care about the data structure size, since converting a String to Box<str> can be costly (allocation and copy if the capacity isn't exact) and carrying a capacity is free pretty much all of the time.
Arnavion 4y ago

the__alchemist 4y ago

I'm working on a PC-based configuration for a drone flight controller. PC-side is std Rust with a stack available. Firmware is `no-std`, running on a microcontroller. It has waypoints you can program when connected to a PC using USB. They have names that need to be represented as some sort of string. I'm using `u8` arrays for the strings on both sides; seems the easiest to serialize, and Rust has `str::from_utf8` etc to handle conversion to/from the UI. `String` is unsupported on the MCU side since there's no allocation. I find this low-level approach ergonomic given it's easy to [de]serialize over USB.

OJFord 4y ago

If OP is here, then in this listing: let boxed_str: Box<str> = "hello".into(); println!("size of boxed_str on stack: {}", std::mem::size_of_val(&boxed_str)); let s = String::from("hello!"); println!("size of string on stack: {}", std::mem::size_of_val(&s)); I know it's not the point and doesn't make a difference, but you might want to make the two 'strings' the same (not with & without '!'), just to be clearer.

School-Cotton 4y ago

This might clarify the situation, for C or C++ folks: // heap-allocated, fixed-size struct BoxStr { unsigned length; // INVARIANT: this points to a heap allocation of length bytes, and is valid utf8 unsigned char *data; } // heap-allocated, resizable struct String { unsigned length; unsigned capacity; // INVARIANT: heap allocation of capacity bytes, the first length of which are valid utf8 unsigned char *data; } Of course you could resize BoxStr, but only by reallocating `data` to the exact desired length every time, which will kill your asymptotic complexity.

tylerhou 4y ago
Is your first example really equivalent to Box<str>? I would have expected something like using BoxStr = std::unique_ptr<Str>; where Str is defined as struct Str { size_t len; char data[]; }; The difference is that the len is stored on the heap, and the data is stored inline with the length. Unfortunately C++ does not support flexible array members so this syntax is not actually valid. Edit: Never mind, after reading the article Rust does use the above representation because Box holds a “fat” pointer to str, which stores it’s length on the stack. So BoxStr is the correct equivalent, because &[u8] is not equivalent to u8*, it’s equivalent to std::span<u8>.
- steveklabnik 4y ago
  Your parent is correct, the length is stored alongside the pointer, not on the heap with its data. This is true for any "dynamically sized type," not just Box<str>. &str is also a (pointer, length) pair, for example.

codedokode 4y ago

Is there official documentation about what `str` (without an ampersand) is? For example, documentation [1] says that `str` is a "string slice" (without explaining what "string slice" mean), and then goes on with description of &str. And a book on Rust [2] says: > A string slice is a reference to part of a String This seems wrong, because &str can reference static strings which are not String. And if str, or "string slice" is a "reference", then &str is a reference to a reference? And later: > The type that signifies “string slice” is written as &str But the documentation said that "string slice" is str, not &str. Also, I wonder, what do square brackets mean when they are used without an ampersand (as s[0..2] instead of &s[0..2])? Also, is an ampersand in &str the same as an ampersand in &u8 (meaning an immutable reference to u8) or does it have other meaning? [1] https://doc.rust-lang.org/std/primitive.str.html https://doc.rust-lang.org/std/primitive.str.html [2] https://doc.rust-lang.org/book/ch04-03-slices.html#string-slices https://doc.rust-lang.org/book/ch04-03-slices.html#string-sl...

deleted 4y ago
[deleted]
LegionMammal978 4y ago
> Is there official documentation about what `str` (without an ampersand) is? For example, documentation [1] says that `str` is a "string slice" (without explaining what "string slice" mean), and then goes on with description of &str. A `str` is really just a `[u8]` with extra semantics. Thus, a `&str` is really a `&[u8]`, a `&mut str` is a `&mut [u8]`, a `Box<str>` is a `Box<[u8]>`, etc. So we call it a "string slice", since it mostly acts like a regular `[T]` slice. In general, the term "slice" can either refer to the unsized type `[T]` or the reference `&[T]`/`&mut [T]` interchangeably. You could also call the latter a "slice reference" where the distinction is important; e.g., a `Box<[T]>` would be a "boxed slice", while `Box<&[T]>` would be a "boxed slice reference" or "boxed reference to a slice". But most of the time, the correct meaning can be inferred from context. > Also, I wonder, what do square brackets mean when they are used without an ampersand (as s[0..2] instead of &s[0..2])? `s[0..2]` is a place expression that refers to the raw `str` subslice. But since `str` is an unsized type [0], it cannot appear on its own; it must appear behind some reference type. Thus, `&s[0..2]` creates a `&str`, and `&mut s[0..2]` creates a `&mut str`. However, the ampersand isn't always necessary: you can write `s[0..2].to_owned()` to use the `str` as a method receiver, which implicitly creates a reference. [0] https://doc.rust-lang.org/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait https://doc.rust-lang.org/book/ch19-04-advanced-types.html#d...

jez 4y ago

Do any of the string types in the Rust standard library implement the same sort of small string optimization that C++ libraries implement for std::string? (explained here[1]) Some quick searching turned up a few rust-lang internals posts and GitHub issues, but it was hard to see whether anything came of them. I understand that it’s probably possible to implement a comparable String API in a crate that uses small string optimizations, but being able to avoid a dedicated crate makes interoperability with other libraries much easier. [1] https://tc-imba.github.io/posts/cpp-sso/ https://tc-imba.github.io/posts/cpp-sso/

steveklabnik 4y ago
Rust's standard library strings cannot because of a specific API, as_mut_vec, which is incompatible with the internal representation necessary to do SSO.
- mwcampbell 4y ago
  Do you think including this API was a mistake?
  - steveklabnik 4y ago
    No. There’s a discussion below about SSO. I generally agree with it. It’s a trade off, not a universal improvement. Keeping things basic for the basic case is a good thing.
    - mwcampbell 4y ago
      Yet the standard library's HashMap and BTreeMap are very sophisticated. IMO, the more optimizations are built into the standard library, the more effectively Rust can achieve its goal of empowering everyone to build reliable and efficient software. The as_mut_vec function doesn't even really prevent Rust's standard String type from implementing SSO. That function, which is already unsafe, could just come with a warning in the documentation that calling it will force the string into its heap-allocated form. But no, I don't care enough about this to do the work myself, so I'll say no more.

Thorentis 4y ago

This type of complexity is a huge downside for a language. A modern language, designed with all the prior knowledge we now have about language design, should not be this complicated for something as essential as a String. For this reason, I do not like Rust. For similar reasons, I do not like Go. And I'm extremely disappointed that despite being developed so recently, they make these same mistakes. I honestly think Python is one of the best languages (note I am strictly talking about the language, let's put aside performance). If there was work done to make it compilable I would be very happy. But even then, for my own uses this isn't an issue. I just wish there was something as fantastic in the system language space.

saghm 4y ago
To be fair, Python had to make a _ton_ of breaking changes to the way their strings worked that arguably are still reverberating in the industry (python2 is still not fully gone, although it's finally starting to fade away). I'm also not convinced that it's possible to differentiate the performance aspects from the way that Rust does strings; unless you're willing to just sacrifice a decent amount of performance by default, I'm not sure there's any way to avoid multiple string types. I don't think that the Rust string API is any sort of platonic ideal for a systems language; I just haven't really seen any alternatives to needing both a mutable, heap-modified string type alongside an immutable reference string type that doesn't require allocation to create. C++ has `std::string_view` alongside `std::string`, Java has both `String` and `StringBuffer`; I think the pain in Rust has a lot more to do with the ambiguity when just saying the word "string" out loud and some confusion about how `str` essentially requires using some sort of reference, making `&String` a weird type that 99% of the time you ignore and the other 1% you need to work around due to some type inference in a closure or something.
unclad5968 4y ago
What's the alternative? If there was just one string type, "str", then you would have to implement your own "String" anyway for any strings you intend to manipulate. Computers are complex. Sometimes you need a heap allocated vector of bytes, and sometimes you need a heap allocated array of bytes.

jollybean 4y ago

I really loathe to read this. I love the borrow checker, but I despise arbitrary complexity. I feel that Rust is far more complicated than it needs to be, and that this is our 'first attempt' with borrowing and lifetimes, and that there is an 'easier way'. Engineers love to solve problems, if we can solve the 'life time' issue, we 'feel' very accomplished. And I mean literally a 'feeling'. We are not bound to business and customer success in the same way, and so we tend to value one issue over the other due to our instinct. I'm still skeptical about the extra time being worth the added cost of development in so many cases.

hota_mazi 4y ago

Rust just opens up the kimono and exposes bare metal details about strings. A string with quotes goes into the executable. A `String` exists at runtime and is an entire string. An `&str` is a slice of a string: it points to an existing string for a certain number of characters. Not all programs or programmers require this amount of knowledge, but if you want to understand programming at a deeper level, Rust is a great tool to dive into the deep end to understand complex topics about programming languages.

n8henrie 4y ago

Many thanks to OP for some real-life examples of using rust-lldb! I keep wanting to learn to use it, and I think this was really helpful.