7 ms·

One thing I've often wondered about Rust strings. I often hear that &str is 'a string slice'. But, Rust has a notation for slices -- &[T]. Why are strings the o

One thing I've often wondered about Rust strings. I often hear that &str is 'a string slice'. But, Rust has a notation for slices -- &[T]. Why are strings the only thing (that I know of) that don't use the same slice notation as everything else?

brundolf 6y ago
That's a great question. I don't have a complete answer, but I do know that &str has lots of string-specific functionality that's really helpful. The .chars() method for example gives you an iterator over actual unicode chars, as opposed to bytes, because the former can have variable byte-widths. There may be other reasons; I'm not sure.
littlestymaar 6y ago
> But, Rust has a notation for slices -- &[T] &[T] is an “array-slice”, even if it's called just “slice”. See this example ([..] is the syntax to create a slice of something): https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1fe3556fb5dea35603eb7d26bd7d5863 https://play.rust-lang.org/?version=stable&mode=debug&editio...
edflsafoiewq 6y ago
What would they be, &[u8]? That's already a thing: it's an arbitrary byte sequence. &str is specifically UTF-8 data. &OsStr and &Path are the same way.
- __s 6y ago
  Indeed &str has as_bytes which returns itself as &[u8] But, str is a subset of &[u8], the type's contract is that it is unsafe to have non valid UTF-8 data, hence https://doc.rust-lang.org/std/str/fn.from_utf8.html https://doc.rust-lang.org/std/str/fn.from_utf8.html can error, offering the unsafe variant https://doc.rust-lang.org/std/str/fn.from_utf8_unchecked.html https://doc.rust-lang.org/std/str/fn.from_utf8_unchecked.htm... This is all very different than &[char] which would be an array of 4 byte characters (or, a UCS4 string)
lmkg 6y ago
The regular slice type &[T] lets you access and manipulate individual elements. But Rust strings enforce the invariant that they are valid Unicode, which puts restrictions on element-wise operations. Calling &str a "string slice" is really more about the contrast with String, and how the relationship there mirrors the relationship between &[T] and Vec<T>. It's more of an analogy than a concrete description of the interface.
tomjakubowski 6y ago
`str` contractually guarantees UTF-8 contents, so because of multi-byte codepoints it cannot be sliced at arbitrary indexes like a `[u8]` can be. As a side note, it is possible to define your own "unsized" slice type which wraps `[u8]`. This can be useful for binary serialization formats which can be subdivided / sliced into smaller data units.
- Const-me 6y ago
  > `str` contractually guarantees UTF-8 contents I don’t think any other languages do that. Instead, most of them are implementing as much as they can while viewing the storage as a blob of UTF8/UTF16 bytes/words, and throw exceptions from the methods which interpret the data as codepoints. Strings are used a lot in all kinds of APIs. For instance, strings are used for file and directory names. The OS kernels don’t require these strings to be valid UTF-8 (Linux) or UTF-16 (Windows). To address the use case, Rust standard library needs yet another string type, OsString. This contributes to complexity and learning curve.
  - estebank 6y ago
    The number of things that you have to learn remains constant. I could even make the argument that the number of things you need to learn up-front is lowered when only talking about the distinction between String and OsString/CString. The difference is that rustc will be pedantic and complain about all of these cases, asking you to specify exactly what you wanted, while other languages will fail at runtime.
    - Const-me 6y ago
      > rustc will be pedantic and complain about all of these cases, asking you to specify exactly what you wanted So, they offloading complexity to programmers. Being such a programmer, I don’t like their attitude. > other languages will fail at runtime In practice, other languages usually printing squares, or sometimes character escape codes after backslash, for encoding errors in their strings. That’s not always the best thing to do, but I think that’s what people want in majority of use cases.
steveklabnik 6y ago
So, there's like, a few things here. First is, technically they both do use the same notation, &T, where T=str and T=[u8]. This is the whole "unsized types" thing. &Path is another example of this, String : &str :: PathBuf : &Path. Beyond that though, &[T] implies a slice of Ts, that is, multiple Ts in a row. But a &str is a slice of a single string. So &[str] would feel wrong; that is, a &str is a slice of a String or another &str or something else, but isn't like, a list of multiple things. It's String, not Vec<str>. Basically, Strings are just weird.
- nicklecompte 6y ago
  In particular: strings aren’t actually simple arrays of characters in Rust like they are in C, but there is an underlying array on the heap, and the notion of “slicing” that array of characters still makes sense semantically.
  - knodi123 6y ago
    > strings aren’t actually simple arrays of characters in Rust Are you talking about them being a container with a pointer to the actual array, and also a size and etc?
    - Skunkleton 6y ago
      char is usually some single byte of data. Characters can be multiple bytes. Slicing a string on character boundaries is more coarse grained than slicing on byte boundaries.
      - estebank 6y ago
        char in Rust is 32bits, so it has a 1:1 mapping to Unicode glyphs. You might also want to care about grapheme clusters, but those are not part of the stdlib.
        toast0 6y ago
kzrdude 6y ago
"string slice" is just the name of the one thing, and "slice" the name of the other, and they are different things with similar names and related features.
aliceryhl 6y ago
Because there's no possible choice for T. You can't use u8, because then you would allow non-utf8 data. You can't use char, because a &[char] uses four bytes per character, whereas a &str stores the characters in utf-8, which is a variable-width encoding. A &str is really a different kind of thing from other slices. In any other slice, each element in the slice takes up a constant number of bytes, but this is not the case for a &str.
hansihe 6y ago
In Rust a slice `&[T]` is a fixed length sequence of `T`s laid out linearly in memory. Every `T` is required to be of the same size. Strings in Rust are (normally) represented as UTF-8. Both `String` and `str` represent data that is guaranteed to be valid UTF-8. This means that if Rusts UTF-8 strings were represented as normal slices, they would have to be slices of UTF-8 code-units. Rust wants to provide a safe and correct String data type, and therefore, indexing a string on a byte (code-unit) level would be incorrect behavior. Having a custom type `String` and `str` instead of just a `Vec<u8>` enables you to have more correct behavior implemented on top of the data type that doesn't implement normal slice indexing and such. --- As a note, even though you probably don't want to normally, you can quite easily access the backing data of your string using `String::as_bytes`
mamcx 6y ago
Another: String is inmutable AND whole. &[T] is parts, and when declared as &mut [T] is mutable. Because unicode, &[T] make easy to write wrong code (that asume here T = Char). It CAN'T be Char, because char is larger than u8: https://doc.rust-lang.org/std/primitive.char.html https://doc.rust-lang.org/std/primitive.char.html and it mean unicode point. In other words: Rust is using types to PREVENT the wrong behavior.
- pornel 6y ago
  Fun fact: `&mut str` exists. You don't get random access, but in controlled scenarios it's fine to mutate str in-place, e.g. `make_ascii_lowercase` https://doc.rust-lang.org/stable/std/primitive.str.html#method.make_ascii_lowercase https://doc.rust-lang.org/stable/std/primitive.str.html#meth...
stouset 6y ago
Long story short, Rust slices are contiguous collections of identically-sized elements. Strings, by being UTF-8, are not guaranteed to be identically-sized. So they require a distinct type that lets you work with them in spite of this detail.

brundolf 6y ago

That's a great question. I don't have a complete answer, but I do know that &str has lots of string-specific functionality that's really helpful. The .chars() method for example gives you an iterator over actual unicode chars, as opposed to bytes, because the former can have variable byte-widths. There may be other reasons; I'm not sure.

littlestymaar 6y ago

> But, Rust has a notation for slices -- &[T] &[T] is an “array-slice”, even if it's called just “slice”. See this example ([..] is the syntax to create a slice of something): https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1fe3556fb5dea35603eb7d26bd7d5863 https://play.rust-lang.org/?version=stable&mode=debug&editio...

edflsafoiewq 6y ago

What would they be, &[u8]? That's already a thing: it's an arbitrary byte sequence. &str is specifically UTF-8 data. &OsStr and &Path are the same way.

__s 6y ago
Indeed &str has as_bytes which returns itself as &[u8] But, str is a subset of &[u8], the type's contract is that it is unsafe to have non valid UTF-8 data, hence https://doc.rust-lang.org/std/str/fn.from_utf8.html https://doc.rust-lang.org/std/str/fn.from_utf8.html can error, offering the unsafe variant https://doc.rust-lang.org/std/str/fn.from_utf8_unchecked.html https://doc.rust-lang.org/std/str/fn.from_utf8_unchecked.htm... This is all very different than &[char] which would be an array of 4 byte characters (or, a UCS4 string)

lmkg 6y ago

The regular slice type &[T] lets you access and manipulate individual elements. But Rust strings enforce the invariant that they are valid Unicode, which puts restrictions on element-wise operations. Calling &str a "string slice" is really more about the contrast with String, and how the relationship there mirrors the relationship between &[T] and Vec<T>. It's more of an analogy than a concrete description of the interface.

tomjakubowski 6y ago

`str` contractually guarantees UTF-8 contents, so because of multi-byte codepoints it cannot be sliced at arbitrary indexes like a `[u8]` can be. As a side note, it is possible to define your own "unsized" slice type which wraps `[u8]`. This can be useful for binary serialization formats which can be subdivided / sliced into smaller data units.

Const-me 6y ago
> `str` contractually guarantees UTF-8 contents I don’t think any other languages do that. Instead, most of them are implementing as much as they can while viewing the storage as a blob of UTF8/UTF16 bytes/words, and throw exceptions from the methods which interpret the data as codepoints. Strings are used a lot in all kinds of APIs. For instance, strings are used for file and directory names. The OS kernels don’t require these strings to be valid UTF-8 (Linux) or UTF-16 (Windows). To address the use case, Rust standard library needs yet another string type, OsString. This contributes to complexity and learning curve.
- estebank 6y ago
  The number of things that you have to learn remains constant. I could even make the argument that the number of things you need to learn up-front is lowered when only talking about the distinction between String and OsString/CString. The difference is that rustc will be pedantic and complain about all of these cases, asking you to specify exactly what you wanted, while other languages will fail at runtime.
  - Const-me 6y ago
    > rustc will be pedantic and complain about all of these cases, asking you to specify exactly what you wanted So, they offloading complexity to programmers. Being such a programmer, I don’t like their attitude. > other languages will fail at runtime In practice, other languages usually printing squares, or sometimes character escape codes after backslash, for encoding errors in their strings. That’s not always the best thing to do, but I think that’s what people want in majority of use cases.

steveklabnik 6y ago

So, there's like, a few things here. First is, technically they both do use the same notation, &T, where T=str and T=[u8]. This is the whole "unsized types" thing. &Path is another example of this, String : &str :: PathBuf : &Path. Beyond that though, &[T] implies a slice of Ts, that is, multiple Ts in a row. But a &str is a slice of a single string. So &[str] would feel wrong; that is, a &str is a slice of a String or another &str or something else, but isn't like, a list of multiple things. It's String, not Vec<str>. Basically, Strings are just weird.

nicklecompte 6y ago
In particular: strings aren’t actually simple arrays of characters in Rust like they are in C, but there is an underlying array on the heap, and the notion of “slicing” that array of characters still makes sense semantically.
- knodi123 6y ago
  > strings aren’t actually simple arrays of characters in Rust Are you talking about them being a container with a pointer to the actual array, and also a size and etc?
  - Skunkleton 6y ago
    char is usually some single byte of data. Characters can be multiple bytes. Slicing a string on character boundaries is more coarse grained than slicing on byte boundaries.
    - estebank 6y ago
      char in Rust is 32bits, so it has a 1:1 mapping to Unicode glyphs. You might also want to care about grapheme clusters, but those are not part of the stdlib.
      - toast0 6y ago

kzrdude 6y ago

"string slice" is just the name of the one thing, and "slice" the name of the other, and they are different things with similar names and related features.

aliceryhl 6y ago

Because there's no possible choice for T. You can't use u8, because then you would allow non-utf8 data. You can't use char, because a &[char] uses four bytes per character, whereas a &str stores the characters in utf-8, which is a variable-width encoding. A &str is really a different kind of thing from other slices. In any other slice, each element in the slice takes up a constant number of bytes, but this is not the case for a &str.

hansihe 6y ago

In Rust a slice `&[T]` is a fixed length sequence of `T`s laid out linearly in memory. Every `T` is required to be of the same size. Strings in Rust are (normally) represented as UTF-8. Both `String` and `str` represent data that is guaranteed to be valid UTF-8. This means that if Rusts UTF-8 strings were represented as normal slices, they would have to be slices of UTF-8 code-units. Rust wants to provide a safe and correct String data type, and therefore, indexing a string on a byte (code-unit) level would be incorrect behavior. Having a custom type `String` and `str` instead of just a `Vec<u8>` enables you to have more correct behavior implemented on top of the data type that doesn't implement normal slice indexing and such. --- As a note, even though you probably don't want to normally, you can quite easily access the backing data of your string using `String::as_bytes`

mamcx 6y ago

Another: String is inmutable AND whole. &[T] is parts, and when declared as &mut [T] is mutable. Because unicode, &[T] make easy to write wrong code (that asume here T = Char). It CAN'T be Char, because char is larger than u8: https://doc.rust-lang.org/std/primitive.char.html https://doc.rust-lang.org/std/primitive.char.html and it mean unicode point. In other words: Rust is using types to PREVENT the wrong behavior.

pornel 6y ago
Fun fact: `&mut str` exists. You don't get random access, but in controlled scenarios it's fine to mutate str in-place, e.g. `make_ascii_lowercase` https://doc.rust-lang.org/stable/std/primitive.str.html#method.make_ascii_lowercase https://doc.rust-lang.org/stable/std/primitive.str.html#meth...

stouset 6y ago

Long story short, Rust slices are contiguous collections of identically-sized elements. Strings, by being UTF-8, are not guaranteed to be identically-sized. So they require a distinct type that lets you work with them in spite of this detail.