5 ms·
Rust still has its warts. When dealing with an archive, say, you can find yourself needing to deal with Windows strings on Unix, or vice versa, but Rust only p
by masfuerte 3y ago
Rust still has its warts. When dealing with an archive, say, you can find yourself needing to deal with Windows strings on Unix, or vice versa, but Rust only provides the string type for the platform you are running on. It could do with UnixString and WindowsString in addition to OsString.
MFC had a similar problem for years. It had a CString class which was ANSI or Unicode, depending on how your compiled your app, but moderately often you needed the other one, so it should have had a CStringA and CStringW too, with nice conversions between them.
- tialaramex 3y agoOsString isn't "the native string type", it's a container for whatever was convenient for Rust's internals on that OS, there are convenience functions so it probably feels like it's "UnixString" or "WindowsString" but it's neither. In most of these file format cases what you've got is &[u8] or &[u16] and maybe it's the NonZero variant instead, so I think it's fine to be explicit that's what is going on and maybe in the process remind you to check - is this data UTF16LE? UTF16 with a BOM? UCS2 with a nod and a wink? Just arbitrary 16-bit integers and good luck? But like I said, I favoured "picking nits" long before I learned Rust, so mileage may vary.
- pornel 3y agoIMHO Rust got OSString wrong – it indirectly promised that UTF-8 can always be cast to it without any copying or allocations, so on Windows it has to use WTF-8 rather than UCS2/UTF-16. Instead of being OS’s preferred string type, it’s merely a trick to preserve unpaired surrogates.
- tialaramex 3y agoHuh. Where is the "indirect promise" ? Is there a (conventionally hinted as "free", so it's not good if there's one which isn't very cheap) as function? Like as_os_string or something?
- LegionMammal978 3y agoIt's because str implements AsRef<OsStr> [0]. The function signature promises that whenever you have a borrowed &'a str, the standard library can give you a borrowed &'a OsStr with the same data. Since references can't have destructors (they don't own the data like an OsString does), it means that the standard library can't give you a newly-allocated string without leaking it. Since obviously it isn't going to do that, the &OsStr must instead just act as a view into the underlying &str. And the conversion can't enforce any extra restrictions on the input string without breaking backward compatibility. The overall effect is that whatever format OsStr uses, it has to be a superset of UTF-8. [0] https://doc.rust-lang.org/1.77.2/std/ffi/struct.OsStr.html#impl-AsRef%3COsStr%3E-for-str https://doc.rust-lang.org/1.77.2/std/ffi/struct.OsStr.html#i...
- tialaramex 3y agoAh, an AsRef<OsStr>, yep that would do it. Thanks.
- jcranmer 3y agoI wouldn't say it's necessarily wrong; it may be (accidental) foresight. Windows has added a UTF-8 code page, which means you can get as near enough as makes no difference full support with the A functions instead of the W functions. That said, even now in 2024, it's not clear how much of a bet Windows is making on UTF-8 versus UTF-16.
- pornel 3y agoI would be surprised if the UTF-8 support in Windows was anything deeper than the A functions creating a W string and calling W functions, which is what Rust is doing already.
- tialaramex 3y agoThe question is whether that's all they ever do, or whether gradually some parts of Windows begin to decide to be UTF-8 first and their W functions now translate the opposite way. This must be tempting for some networking stuff, where you're otherwise taking a perf hit compared to Linux.
- pjmlp 3y agoThe first time I had to deal with this was on Turbo Pascal for Windows, where we got C strings alongside Pascal ones. Then when 32 bit came to be we had the other variations on top. Got to love leaky abstractions.