3 ms·
Everything labelled as "string" is a valid UTF-8 string in Rust, and to my knowledge this decision was made very early in the history of Rust (before 0.1). Many
by lifthrasiir 4y ago
Everything labelled as "string" is a valid UTF-8 string in Rust, and to my knowledge this decision was made very early in the history of Rust (before 0.1). Many "modern" languages (including modern enough C++) have a distinction between Unicode strings and byte strings however they are called and Rust just followed the suit.
- M2Ys4U 4y agoOsStr and OsString aren't necessarily UTF-8, the data for those are "in the operating system’s preferred representation". str/String and CStr/CString are defined as UTF-8, though.
- puffoflogic 4y ago> in the operating system’s preferred representation This is unfortunately misleading and a common misconception about OsStr. The documentation now explains: > OsStr losslessly represents a borrowed reference to a platform string. However, this representation is not necessarily in a form native to the platform What this means is that valid sequences of Unicode scalars are encoded as utf8 in OsStr on both Windows and Linux. The difference between OsStr and str is that the former can round-trip with the native encoding; that means that for Windows, there's a special way it encodes unpaired surrogates (wtf8) and on Linux it's actually just an arbitrary byte sequence. This choice of representation means that on every platform: you can always borrow an OsStr from a str (as_ref) and you can sometimes borrow a str from an OsStr (to_str). This cross-borrowing wouldn't be possible if OsStr were UCS-2 on Windows.