5 ms·

So is the problem with strings that Rust internally uses null-terminated (and if so, is there some way to make that safe), or that the OS inevitably uses null-t

So is the problem with strings that Rust internally uses null-terminated (and if so, is there some way to make that safe), or that the OS inevitably uses null-terminated so you have to handle that at the edges? Because the latter is... kind of inevitable, unless you want to completely throw everything out and start over.

steveklabnik 3y ago
Rust’s native strings are not null terminated, though there are alternative string types (CString, CStr) that are. If you want to call a function that requires a zero terminated string, you’ll have to convert if you don’t have one already. And yep, the latter is true. Just how it is. It’s unfortunate, but it’s also inevitable.
- hi-v-rocknroll 3y ago
  Different flavors for different use-cases. There was even $-terminated ones prevalent on DOS which got it from CP/M which got it from DEC OS/8. Pascal-style strings (length + data or length + pointer to data) have the nice properties of O(1) size inquiry, simpler bounds checking, and easier to represent strings that contain embedded NULs. A downside is though you have to pick an unsigned integer width of power of 2, usually 0 to 3. Another upside is (length + pointer to data) immutable substrings (with longer lifetimes that the parent string) are very cheap because they just point to wherever in the start and specify how much of the parent string with the length. There was a language IIRC that implemented a more complicated Pascal-style strings perhaps for COW or reference counting of the underlying allocation.
fl0ki 3y ago
This was a really weird complaint. I'm actually curious what language the author considered superior in this regard. Almost every language has agreed that string slicing is important enough that it's worth the slight impedance mismatch with legacy APIs. Given the system APIs (heck, ABIs) will remain, the options are to {expose|hide} this detail {safely|unsafely}. Out of these, Rust seems to have made a fair choice: most APIs hide it safely, but you also get everything you need to engage with this detail when you must, safely if possible and well-contained otherwise. How many languages engage with this concern so completely that they have compiler support for null-terminated C string literals? https://doc.rust-lang.org/edition-guide/rust-2021/c-string-literals.html https://doc.rust-lang.org/edition-guide/rust-2021/c-string-l... Even C++ eventually got std::string_view, it just got it with temporal unsafety and no type-level way to prevent you from using a non-terminated string slice to call an API expecting termination. This was a problem even when it was Google's internal StringPiece and you were expected to take the subtle hint that StringPiece::data() didn't imply termination the same way that string::c_str() did. The many bugs that followed proved this was not a sufficient hint, reminding us yet again why comments are no substitute for type systems, and thus why Rust's extra guard rails around this problem are justified for the majority of cases.
- dorinlazar 3y ago
  The author here. The point there is not a critique of Rust. I'll offer a bit of context - I started my „from scratch” utility library with a std::string-like type that operates on reference counted buffers. In the end, any operations I try to do I have to put the sliced string in a zero-terminated string because that's how one interacts with the operating system and with other libraries as well. Which is a bit dumb, unfortunately. A lot of C (and C++) memory-related issues are caused by this assumption that by just passing a `char *` one sends a string out into the world. A point that I made on other occasions in other contexts is that in the end POSIX makes us do this, and perhaps it's time to rewrite from the bottoms up. That aside, the issue of safety of Rust doesn't change: just because one has the `unsafe` keyword doesn't mean that someone will not use it badly. Maybe the reader of this will not use unsafe unwisely, they will eventually call unsafe-by-design APIs. At their layer they will write safe code, but below them there's a rusty bucket, if one can pardon my pun.

steveklabnik 3y ago

Rust’s native strings are not null terminated, though there are alternative string types (CString, CStr) that are. If you want to call a function that requires a zero terminated string, you’ll have to convert if you don’t have one already. And yep, the latter is true. Just how it is. It’s unfortunate, but it’s also inevitable.

hi-v-rocknroll 3y ago
Different flavors for different use-cases. There was even $-terminated ones prevalent on DOS which got it from CP/M which got it from DEC OS/8. Pascal-style strings (length + data or length + pointer to data) have the nice properties of O(1) size inquiry, simpler bounds checking, and easier to represent strings that contain embedded NULs. A downside is though you have to pick an unsigned integer width of power of 2, usually 0 to 3. Another upside is (length + pointer to data) immutable substrings (with longer lifetimes that the parent string) are very cheap because they just point to wherever in the start and specify how much of the parent string with the length. There was a language IIRC that implemented a more complicated Pascal-style strings perhaps for COW or reference counting of the underlying allocation.

fl0ki 3y ago

This was a really weird complaint. I'm actually curious what language the author considered superior in this regard. Almost every language has agreed that string slicing is important enough that it's worth the slight impedance mismatch with legacy APIs. Given the system APIs (heck, ABIs) will remain, the options are to {expose|hide} this detail {safely|unsafely}. Out of these, Rust seems to have made a fair choice: most APIs hide it safely, but you also get everything you need to engage with this detail when you must, safely if possible and well-contained otherwise. How many languages engage with this concern so completely that they have compiler support for null-terminated C string literals? https://doc.rust-lang.org/edition-guide/rust-2021/c-string-literals.html https://doc.rust-lang.org/edition-guide/rust-2021/c-string-l... Even C++ eventually got std::string_view, it just got it with temporal unsafety and no type-level way to prevent you from using a non-terminated string slice to call an API expecting termination. This was a problem even when it was Google's internal StringPiece and you were expected to take the subtle hint that StringPiece::data() didn't imply termination the same way that string::c_str() did. The many bugs that followed proved this was not a sufficient hint, reminding us yet again why comments are no substitute for type systems, and thus why Rust's extra guard rails around this problem are justified for the majority of cases.

dorinlazar 3y ago
The author here. The point there is not a critique of Rust. I'll offer a bit of context - I started my „from scratch” utility library with a std::string-like type that operates on reference counted buffers. In the end, any operations I try to do I have to put the sliced string in a zero-terminated string because that's how one interacts with the operating system and with other libraries as well. Which is a bit dumb, unfortunately. A lot of C (and C++) memory-related issues are caused by this assumption that by just passing a `char *` one sends a string out into the world. A point that I made on other occasions in other contexts is that in the end POSIX makes us do this, and perhaps it's time to rewrite from the bottoms up. That aside, the issue of safety of Rust doesn't change: just because one has the `unsafe` keyword doesn't mean that someone will not use it badly. Maybe the reader of this will not use unsafe unwisely, they will eventually call unsafe-by-design APIs. At their layer they will write safe code, but below them there's a rusty bucket, if one can pardon my pun.