3 ms·

With the path example… just try to combine this with flags, so we do something like: $ ./my_program --file="$(printf "\xbd\xb2\x3d\xbc\x20\xe2\x8c\x98")"

With the path example… just try to combine this with flags, so we do something like: $ ./my_program --file="$(printf "\xbd\xb2\x3d\xbc\x20\xe2\x8c\x98")" Well, just try to write the program that does that in Rust, without using some option-parsing library that hides all the details, and then try to figure out how to get it to work equally on Windows. To spoil the answer, it turns out that OsString only exposes a couple conversion routines and can’t be manipulated, and people have been trying to figure out a way to add a string-like API to it for years. Rust’s “do it the right way even if that exposes lots of complexity” approach here has its drawbacks.

tedmielczarek 7y ago
The point isn't that it should be easy to do terrible broken things like this. The point is that you will encounter these things in the real world and have to deal with them in some way. Rust's OsStr[ing] let you do that. If you really have a burning need to create files whose paths are not valid Unicode you can do that in Rust but you will have to jump through some hoops. I don't see that as a problem.
- klodolph 7y ago
  I’m not saying that it should be easy, just that the API shouldn’t introduce significant amounts of additional complexity. For the very simple case of "I want a command-line option which specifies a path as an OsString", Rust’s way of doing things makes things hard. By comparison, in C++, I am used to dealing with paths as std::string on Unix and std::wstring or std::u16string on Windows, and this C++ approach is a lot easier. Rust’s OsString design is too smart by half, and if I use env::args_os(), I can’t easily do simple tasks like "test if this string starts with '-'" or "split this string by the first '=', if it exists". As far as I can tell, the way to go is to convert OsString to Vec<u8>, do your processing there, and then convert back… but that only works on Unix, because arbitrary Vec<u8> aren’t safe to convert back to OsString on Windows because they may not be valid WTF-8. So you can take the approach on Windows of going through encode_wide().collect() and it just goes downhill from there. :-(
  - tedmielczarek 7y ago
    > For the very simple case of "I want a command-line option which specifies a path as an OsString", Rust’s way of doing things makes things hard. By comparison, in C++, I am used to dealing with paths as std::string on Unix and std::wstring or std::u16string on Windows, and this C++ approach is a lot easier. FYI, you can convert an `OsStr` (or `OsString`) into a [`Path`](https://doc.rust-lang.org/std/path/struct.Path.html https://doc.rust-lang.org/std/path/struct.Path.html) with zero overhead and get all the useful things you want to do with paths: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=6c799098caca18c9a2b19e84fc8f08e1 https://play.rust-lang.org/?version=stable&mode=debug&editio... > Rust’s OsString design is too smart by half, and if I use env::args_os(), I can’t easily do simple tasks like "test if this string starts with '-'" or "split this string by the first '=', if it exists". As far as I can tell, the way to go is to convert OsString to Vec<u8>, do your processing there, and then convert back For commandline arguments that you actually need to parse there's nothing wrong with asserting that you only accept valid Unicode, and then you can just use `to_str()`: https://doc.rust-lang.org/std/ffi/struct.OsStr.html#method.to_str https://doc.rust-lang.org/std/ffi/struct.OsStr.html#method.t... If you have a `Vec<u8>` or `String` or whatever you can just treat it as an `OsStr` (with no overhead) because both of those (and a bunch of other things) implement `AsRef<OsStr>`: https://doc.rust-lang.org/std/path/struct.Path.html#method.new https://doc.rust-lang.org/std/path/struct.Path.html#method.n...
dpc_pw 7y ago
If you want to half-ass it like Go you go https://doc.rust-lang.org/std/ffi/struct.OsString.html#method.to_string_lossy https://doc.rust-lang.org/std/ffi/struct.OsString.html#metho... or https://doc.rust-lang.org/std/ffi/struct.OsString.html#method.into_string https://doc.rust-lang.org/std/ffi/struct.OsString.html#metho... if you want to potentially get an error. If you want to deal with bytes / invalid Unicode, you go https://doc.rust-lang.org/std/ffi/index.html#conversions https://doc.rust-lang.org/std/ffi/index.html#conversions
- klodolph 7y ago
  I'm aware of the conversions, unfortunately, you can’t really do any processing before you convert and you can’t (unlike C++) write generic code that works on both types of converted values. On Unix you get Vec<u8> and on Windows you get an iterator over u16. This is hot garbage, to say the least, if you want to do any kind of processing. I can go into more details, but in C++ you would just be working with std::string and std::wstring, depending on platform, and at least in that case you can hide everything away like this: #if defined WIN32 using OsChar = wchar_t; #else using OsChar = char; #endif using OsString = std::basic_string<OsChar>; This is only the beginning, but you can see how the C++ version is much easier to work with, even though it doesn’t hide the problem from you. Note that I’m not advocating that you make everything in your code into OsString, just that it’s common to need to do some small amount of manipulation of OsString and Rust makes this much harder than it should be.
  - dpc_pw 7y ago
    With Rust you can also convert to `Vec<T>` where T is either `u8` or `u16` and use generics to work on any. ️ And there are probably handful of libraries that would help with all that too. Also - whatever convenient functionality you might want, can be added in the future without issues. Hardly a language flaw - just a minor unimplemented functionality.
    - klodolph 7y ago
      > With Rust you can also convert to `Vec<T>` where T is either `u8` or `u16` and use generics to work on any. That’s a very cumbersome way of doing things. I would love to see an illustration. It also involves a ton of conversions: if want to parse a command-line flag which contains a file path, it would go: wchar_t -> OsString -> Vec<u16> -> OsString -> wchar_t. It also makes it difficult to use Rust APIs in a more or less idiomatic way. > Hardly a language flaw - just a minor unimplemented functionality. It’s a flaw in the standard library, not the language. When you say that it’s minor, all you’re doing is saying, “I don’t care about the things you care about.” That’s not really an argument, just a statement of your own personal opinion.

tedmielczarek 7y ago

The point isn't that it should be easy to do terrible broken things like this. The point is that you will encounter these things in the real world and have to deal with them in some way. Rust's OsStr[ing] let you do that. If you really have a burning need to create files whose paths are not valid Unicode you can do that in Rust but you will have to jump through some hoops. I don't see that as a problem.

klodolph 7y ago
I’m not saying that it should be easy, just that the API shouldn’t introduce significant amounts of additional complexity. For the very simple case of "I want a command-line option which specifies a path as an OsString", Rust’s way of doing things makes things hard. By comparison, in C++, I am used to dealing with paths as std::string on Unix and std::wstring or std::u16string on Windows, and this C++ approach is a lot easier. Rust’s OsString design is too smart by half, and if I use env::args_os(), I can’t easily do simple tasks like "test if this string starts with '-'" or "split this string by the first '=', if it exists". As far as I can tell, the way to go is to convert OsString to Vec<u8>, do your processing there, and then convert back… but that only works on Unix, because arbitrary Vec<u8> aren’t safe to convert back to OsString on Windows because they may not be valid WTF-8. So you can take the approach on Windows of going through encode_wide().collect() and it just goes downhill from there. :-(
- tedmielczarek 7y ago
  > For the very simple case of "I want a command-line option which specifies a path as an OsString", Rust’s way of doing things makes things hard. By comparison, in C++, I am used to dealing with paths as std::string on Unix and std::wstring or std::u16string on Windows, and this C++ approach is a lot easier. FYI, you can convert an `OsStr` (or `OsString`) into a [`Path`](https://doc.rust-lang.org/std/path/struct.Path.html https://doc.rust-lang.org/std/path/struct.Path.html) with zero overhead and get all the useful things you want to do with paths: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=6c799098caca18c9a2b19e84fc8f08e1 https://play.rust-lang.org/?version=stable&mode=debug&editio... > Rust’s OsString design is too smart by half, and if I use env::args_os(), I can’t easily do simple tasks like "test if this string starts with '-'" or "split this string by the first '=', if it exists". As far as I can tell, the way to go is to convert OsString to Vec<u8>, do your processing there, and then convert back For commandline arguments that you actually need to parse there's nothing wrong with asserting that you only accept valid Unicode, and then you can just use `to_str()`: https://doc.rust-lang.org/std/ffi/struct.OsStr.html#method.to_str https://doc.rust-lang.org/std/ffi/struct.OsStr.html#method.t... If you have a `Vec<u8>` or `String` or whatever you can just treat it as an `OsStr` (with no overhead) because both of those (and a bunch of other things) implement `AsRef<OsStr>`: https://doc.rust-lang.org/std/path/struct.Path.html#method.new https://doc.rust-lang.org/std/path/struct.Path.html#method.n...

dpc_pw 7y ago

If you want to half-ass it like Go you go https://doc.rust-lang.org/std/ffi/struct.OsString.html#method.to_string_lossy https://doc.rust-lang.org/std/ffi/struct.OsString.html#metho... or https://doc.rust-lang.org/std/ffi/struct.OsString.html#method.into_string https://doc.rust-lang.org/std/ffi/struct.OsString.html#metho... if you want to potentially get an error. If you want to deal with bytes / invalid Unicode, you go https://doc.rust-lang.org/std/ffi/index.html#conversions https://doc.rust-lang.org/std/ffi/index.html#conversions

klodolph 7y ago
I'm aware of the conversions, unfortunately, you can’t really do any processing before you convert and you can’t (unlike C++) write generic code that works on both types of converted values. On Unix you get Vec<u8> and on Windows you get an iterator over u16. This is hot garbage, to say the least, if you want to do any kind of processing. I can go into more details, but in C++ you would just be working with std::string and std::wstring, depending on platform, and at least in that case you can hide everything away like this: #if defined WIN32 using OsChar = wchar_t; #else using OsChar = char; #endif using OsString = std::basic_string<OsChar>; This is only the beginning, but you can see how the C++ version is much easier to work with, even though it doesn’t hide the problem from you. Note that I’m not advocating that you make everything in your code into OsString, just that it’s common to need to do some small amount of manipulation of OsString and Rust makes this much harder than it should be.
- dpc_pw 7y ago
  With Rust you can also convert to `Vec<T>` where T is either `u8` or `u16` and use generics to work on any. ️ And there are probably handful of libraries that would help with all that too. Also - whatever convenient functionality you might want, can be added in the future without issues. Hardly a language flaw - just a minor unimplemented functionality.
  - klodolph 7y ago
    > With Rust you can also convert to `Vec<T>` where T is either `u8` or `u16` and use generics to work on any. That’s a very cumbersome way of doing things. I would love to see an illustration. It also involves a ton of conversions: if want to parse a command-line flag which contains a file path, it would go: wchar_t -> OsString -> Vec<u16> -> OsString -> wchar_t. It also makes it difficult to use Rust APIs in a more or less idiomatic way. > Hardly a language flaw - just a minor unimplemented functionality. It’s a flaw in the standard library, not the language. When you say that it’s minor, all you’re doing is saying, “I don’t care about the things you care about.” That’s not really an argument, just a statement of your own personal opinion.