3 ms·

So just handwave it and it eventually goes away? Nope, as long as: 1. Python standard library itself does not adhere to PEP 383. (Which is not likely to be fix

by rini17 7y ago

So just handwave it and it eventually goes away? Nope, as long as: 1. Python standard library itself does not adhere to PEP 383. (Which is not likely to be fixed if everyone has such dismissive attitude.) and 2. Operating systems do not enforce valid UTF-8 on filenames. (This is unlikely to change sooner than in few decades, if at all.)

matheusmoreira 7y ago
> Operating systems do not enforce valid UTF-8 on filenames. Should they? There is no difference from a file system perspective. We'd still run into problems even if they did: the line feed is a valid UTF-8 character and is one of the characters with special meaning in many programs. Dealing with file names properly is a chore even on bash. https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls
- rini17 7y ago
  It looks to me OP thinks we should "because it's pain otherwise". My own opinion is more like - it is tied to C paradigm (where you don't have any reliable runtime typing/encoding info, everything is just a bag of bytes), and it can't really be solved without moving on to something else. Not sure if that ever happens.
- skissane 7y ago
  > We'd still run into problems even if they did: the line feed is a valid UTF-8 character and is one of the characters with special meaning in many programs. They could ban new line from file names too. See this proposal: http://austingroupbugs.net/view.php?id=251 http://austingroupbugs.net/view.php?id=251
duskwuff 7y ago
> Operating systems do not enforce valid UTF-8 on filenames. macOS does. In fact, it goes a step farther and normalizes filenames (NFD). I'm not aware of any serious problems this has caused. It turns out that -- to a reasonable approximation -- nobody actually writes software which puts arbitrary binary data in filenames.
- np_tedious 7y ago
  Does this also involve case sensitivity? I've encountered errors with nodejs imports where getting filename casing wrong didn't matter on OSX (dev environment) but caused errors in ci/prod (Linux). Never gave it much thought after fixing, but this reminded me
  - duskwuff 7y ago
    Not precisely. File systems are case-insensitive by default on macOS, but this can be changed on a FS-by-FS basis. Unicode normalization is -- as far as I'm aware -- mandatory.

by rini17 7y ago

matheusmoreira 7y ago

> Operating systems do not enforce valid UTF-8 on filenames. Should they? There is no difference from a file system perspective. We'd still run into problems even if they did: the line feed is a valid UTF-8 character and is one of the characters with special meaning in many programs. Dealing with file names properly is a chore even on bash. https://mywiki.wooledge.org/BashPitfalls https://mywiki.wooledge.org/BashPitfalls

rini17 7y ago
It looks to me OP thinks we should "because it's pain otherwise". My own opinion is more like - it is tied to C paradigm (where you don't have any reliable runtime typing/encoding info, everything is just a bag of bytes), and it can't really be solved without moving on to something else. Not sure if that ever happens.
skissane 7y ago
> We'd still run into problems even if they did: the line feed is a valid UTF-8 character and is one of the characters with special meaning in many programs. They could ban new line from file names too. See this proposal: http://austingroupbugs.net/view.php?id=251 http://austingroupbugs.net/view.php?id=251

duskwuff 7y ago

> Operating systems do not enforce valid UTF-8 on filenames. macOS does. In fact, it goes a step farther and normalizes filenames (NFD). I'm not aware of any serious problems this has caused. It turns out that -- to a reasonable approximation -- nobody actually writes software which puts arbitrary binary data in filenames.

np_tedious 7y ago
Does this also involve case sensitivity? I've encountered errors with nodejs imports where getting filename casing wrong didn't matter on OSX (dev environment) but caused errors in ci/prod (Linux). Never gave it much thought after fixing, but this reminded me
- duskwuff 7y ago
  Not precisely. File systems are case-insensitive by default on macOS, but this can be changed on a FS-by-FS basis. Unicode normalization is -- as far as I'm aware -- mandatory.