14 ms·

and yet they may be breaking Cygwin support? I think the problem is more like, there are valid files that aren't representable as rust strings?

by cozzyd 3y ago

__jem 3y ago
That's why OsString exists.
- jshier 3y ago
  The Cygwin issue isn't strings (well, that could be another issue) but that Rust doesn't support Cygwin in the first place, at least according to the comments in the linked thread.
- cozzyd 3y ago
  Well, for whatever reason, it doesn't meet their requirements... See the String section of https://github.com/fish-shell/fish-shell/blob/master/doc_internal/rust-devel.md https://github.com/fish-shell/fish-shell/blob/master/doc_int... edit: here is a more specific rationale: https://github.com/fish-shell/fish-shell/pull/9512#discussion_r1092221110 https://github.com/fish-shell/fish-shell/pull/9512#discussio...
benj111 3y ago
>here are valid files that aren't representable as rust strings Like? I don't use rust but I assume it's capable of utf 8 ?
- bjkchoy 3y ago
  Rust strings enforce utf-8 encoding, yes. However, it seems Windows (which uses utf-16) allows ill-formed UTF-16, in particular it tolerates unpaired surrogates. You can read more here http://simonsapin.github.io/wtf-8/ http://simonsapin.github.io/wtf-8/
- deleted 3y ago
  [deleted]
- paledot 3y ago
  Rust is capable of UTF-8. It is not capable of not UTF-8. Any sequence of bytes that is not a valid UTF-8 string cannot be represented by the String type.
- steveklabnik 3y ago
  The siblings are correct but also not precise in a way in which could be misleading. Rust has one built-in string type: str, commonly written &str. This is a sequence of UTF-8 encoded bytes. Rust also has several standard library string types: * String is an owned version of &str. Also UTF-8. * CString/CStr: like String and &str, but null terminated, with no specific encoding * OsString/OsStr: an "os native" encoded string, in practice, this is the WTF-8 spec. And of course, all sorts of user library-created string types. The issue at hand isn't that there's some sort of special string type that Rust can't represent, it's that they have an existing codebase that uses a specific string type, so they're going to keep using that type for compatibility reasons, rather than one of the more usual Rust types. This means they won't have to transcode between the two types.
  - paledot 3y ago
    <3 WTF-8 If we're competing to be pedantic here, the problem is that Windows encodes paths as UTF-16 and Linux can support just about any random jumble of bytes in a path regardless of if those bytes are valid UTF-8 or not. Neither of these play nicely with Rust's "if it can be represented, it's valid" approach to code safety, so OsStr(ing) exists as a more permissive but less powerful analogue for such cases.

and yet they may be breaking Cygwin support? I think the problem is more like, there are valid files that aren't representable as rust strings?

by cozzyd 3y ago

and yet they may be breaking Cygwin support? I think the problem is more like, there are valid files that aren't representable as rust strings?

__jem 3y ago

That's why OsString exists.

jshier 3y ago
The Cygwin issue isn't strings (well, that could be another issue) but that Rust doesn't support Cygwin in the first place, at least according to the comments in the linked thread.
cozzyd 3y ago
Well, for whatever reason, it doesn't meet their requirements... See the String section of https://github.com/fish-shell/fish-shell/blob/master/doc_internal/rust-devel.md https://github.com/fish-shell/fish-shell/blob/master/doc_int... edit: here is a more specific rationale: https://github.com/fish-shell/fish-shell/pull/9512#discussion_r1092221110 https://github.com/fish-shell/fish-shell/pull/9512#discussio...

benj111 3y ago

>here are valid files that aren't representable as rust strings Like? I don't use rust but I assume it's capable of utf 8 ?

bjkchoy 3y ago
Rust strings enforce utf-8 encoding, yes. However, it seems Windows (which uses utf-16) allows ill-formed UTF-16, in particular it tolerates unpaired surrogates. You can read more here http://simonsapin.github.io/wtf-8/ http://simonsapin.github.io/wtf-8/
deleted 3y ago
[deleted]
paledot 3y ago
Rust is capable of UTF-8. It is not capable of not UTF-8. Any sequence of bytes that is not a valid UTF-8 string cannot be represented by the String type.
steveklabnik 3y ago
The siblings are correct but also not precise in a way in which could be misleading. Rust has one built-in string type: str, commonly written &str. This is a sequence of UTF-8 encoded bytes. Rust also has several standard library string types: * String is an owned version of &str. Also UTF-8. * CString/CStr: like String and &str, but null terminated, with no specific encoding * OsString/OsStr: an "os native" encoded string, in practice, this is the WTF-8 spec. And of course, all sorts of user library-created string types. The issue at hand isn't that there's some sort of special string type that Rust can't represent, it's that they have an existing codebase that uses a specific string type, so they're going to keep using that type for compatibility reasons, rather than one of the more usual Rust types. This means they won't have to transcode between the two types.
- paledot 3y ago
  <3 WTF-8 If we're competing to be pedantic here, the problem is that Windows encodes paths as UTF-16 and Linux can support just about any random jumble of bytes in a path regardless of if those bytes are valid UTF-8 or not. Neither of these play nicely with Rust's "if it can be represented, it's valid" approach to code safety, so OsStr(ing) exists as a more permissive but less powerful analogue for such cases.