4 ms·
Amusingly, Rust strings, whether &str or String, are unable to represent filenames, which in many, many programs is the overwhelmingly most common use for chara
by ncmncm 6y ago
Amusingly, Rust strings, whether &str or String, are unable to represent filenames, which in many, many programs is the overwhelmingly most common use for character sequences that people want to call strings.
The Rust people invented the wonderful "WTF-8" notion to talk about these things. It gets awkward when you want to display a filename in a box on the screen because those boxes like to hold actual strings qua strings, not these godforsaken abominations that show up in file system directory listings.
Handling WTF-8 will take a whole nother article. I don't know a name for WTF-8 sequences; I have been calling them sthrings, which is hard to say, and awkward, but that is kind of appropriate to the case.
- int_19h 6y agoIt doesn't really make things any more complicated than they already are. If you take a filename in C, and then have to display it somewhere, you're facing the same problem - except that you might not even be aware of it, because all types are the same, and you won't notice the problem unless you happen to run into an unprintable filename. Rust is doing the right thing here by forcing developers to deal with those issues explicitly, rather than swiping them under the rug. The real issue is filenames that aren't proper strings - i.e. an OS/FS design defect - but this ship has sailed long ago.
- ncmncm 6y agoThat sthrings are equally as awkward to handle correctly in C, C++, Python, Ruby, Javascript, and Perl as in Rust makes them no less awkward. Nobody said Rust has done anything wrong to ban them from String. Many languages make the distinction, including (probably) C++23. Filename sthrings stink in any language. With the usual solution, using "invalid character blocks", two different files with different names may show up in a list looking the same, so you are still in an awkward place. Even with fully valid Unicode filenames, you can get a similar problem, from legitimate characters that just look the same. Even if filesystems forbade creating files with ill-formed names, we would still have the second problem. Filenames are just a hard problem. It is too much to ask of a language or library to solve them. Even filesystems that forbid ill-formed filenames do not solve them all.
- zajio1am 6y ago> If you take a filename in C, and then have to display it somewhere, you're facing the same problem If you have such problem, it is often better to ignore it and pass data around without any conversion (to force data to be valid utf-8). For example, if i have ls-like tool, who just read filenames and display it, then it will work even with non-utf-8 filenames when used like: for A in $(ls-like) ; do ... But if the ls-like too tries to be smart to interpret data when they are displayed to stdout then it will break them.
- estebank 6y agoDoing that is how you end up with binary data being interpreted as escape sequences, either mangling the screen or, when unlucky, putting your shell in a seemingly unresponsive state. https://unix.stackexchange.com/questions/79684/fix-terminal-after-displaying-a-binary-file https://unix.stackexchange.com/questions/79684/fix-terminal-...
- int_19h 6y agoIf you're merely passing the data, sure. But the scenario I described was specifically displaying data. For the terminal, your approach would only work if the expected encoding of the filename is the same as the output encoding of the terminal - which is not always the case. But then there's also GUI etc. Ultimately, some bit of code (e.g. the terminal emulator) will still need to interpret that filename as a string in order to display it as one - and if it's not really a string, it won't be able to do that properly. Filenames are part of the UX - the filesystem a user-facing artifact, not an implementation detail - so they have to be constrained in such ways that make sense for humans, even if arrays of octets would do just fine as unique IDs.
- Olreich 6y agoThe problem isn’t the forcing you to handle it, it’s the horrific API to handling it. It feels like all the OS representations were haphazardly bolted onto the core string API which is already kinda janky. Rust just needs to sit down and make a good API for the whole set of string-ish types they have.
- eska 6y agoThis is what you want https://doc.rust-lang.org/nightly/std/path/struct.Path.html#method.to_string_lossy https://doc.rust-lang.org/nightly/std/path/struct.Path.html#...