2 ms·

> Swift has a single String type Which as someone who greatly appreciates the separation Rust offers, this makes me curious: does the string type either not en

> Swift has a single String type Which as someone who greatly appreciates the separation Rust offers, this makes me curious: does the string type either not enforce its contents being Unicode, or is it not capable of storing arbitrary OS paths? Because AFAICT, the two are mutually exclusive. The rub here is that OS paths — on POSIX operating systems¹ — are byte strings that we like to pretend are text; but since no particular encoding is enforced, you end up with effectively just byte strings. A String type that validates itself as valid Unicode is not capable of representing these (as arbitrary byte strings are not necessarily valid Unicode). But if we let the String type not validate, then I can pass non-Unicode to functions that take a "String" — something I'd rather not do. There's also Python's byte smuggling, but I find that a bit weird. ¹While I call out POSIX, I'm told Window's pathnames are sequences of unsigned 16-bit values that most people expect to be UTF-16 — but that fact isn't enforced by the OS.

millstone 11y ago
The filesystem used on OS X and iOS is HFS+, which stores filenames as a normalized UTF-16. The POSIX C interface (open(2), etc) takes UTF-8. If you try to make a file with invalid UTF-8, the filesystem will silently munge it. For example, if your path has a trailing 0xc3 byte, that byte gets rewritten into the string "%c3". So you don't actually get arbitrary byte strings. I don't know what happens if you do something like mount an NFS share with filenames that aren't valid Unicode. I would guess a fair amount of stuff will break. Swift strings are Unicode, and Unicode is certainly capable of representing arbitrary byte streams, by just converting to and from any 8 bit fixed-width encoding (not UTF-8!). There are explicit conversion functions like -fileSystemRepresentation which handle the conversion between filesystem paths and Strings.

millstone 11y ago

The filesystem used on OS X and iOS is HFS+, which stores filenames as a normalized UTF-16. The POSIX C interface (open(2), etc) takes UTF-8. If you try to make a file with invalid UTF-8, the filesystem will silently munge it. For example, if your path has a trailing 0xc3 byte, that byte gets rewritten into the string "%c3". So you don't actually get arbitrary byte strings. I don't know what happens if you do something like mount an NFS share with filenames that aren't valid Unicode. I would guess a fair amount of stuff will break. Swift strings are Unicode, and Unicode is certainly capable of representing arbitrary byte streams, by just converting to and from any 8 bit fixed-width encoding (not UTF-8!). There are explicit conversion functions like -fileSystemRepresentation which handle the conversion between filesystem paths and Strings.