4 ms·
There was a related discussion on the Mercurial mailing list a while back. Not about Python 2 vs 3, but about filename encoding. Mercurial follows a policy of
by twic 13y ago
There was a related discussion on the Mercurial mailing list a while back. Not about Python 2 vs 3, but about filename encoding.
Mercurial follows a policy of treating filenames as byte strings. Matt Mackall is very clear about this. Because unix treats filenames as byte strings, this makes Mercurial interoperate with other programs on a unix machine pretty well: you can manage files of any encoding, you can embed filenames in file contents (eg in build scripts) and be confident they will always be byte-for-byte identical with the names managed by Mercurial, etc.
However, it also means Mercurial falls flat on its face when it's asked to share files between machines using different encodings. Names which work fine on one machine will, to human eyes, be garbled nonsense on the other.
This is a problem which does actually happen; there is a slow trickle of bug reports about it. And because of the commitment to unix-style filenames, it will probably never be fixed. List members did try and come up with some ideas to fix it which preserved the unix semantics normal cases, but they weren't popular.
And before anyone gets lippy, i assume Git has the same problem.
Ultimately, i would say this comes down to a conflict between two fundamentally different kinds of users of strings: machines and people. Machines are best served by strings of bytes. People are best served by strings of characters. Usually. And sadly, unix's lack of a known filesystem encoding is too well-established for there to be much chance of building a bridge.
- PeterisP 13y agoIt seems that git doesn't have this problem - just tried adding an encoding-sensitive nonASCII filename, and it worked correctly when pulled between different operating systems (macos, win7, ubuntu).
- twic 13y agoWhat encodings were you using on those machines? The operating system is irrelevant; it's the encoding that matters.
- andreasvc 13y agoWhat do you mean by "share files between machines"? Do you mean over a protocol? In that case the protocol over the wire should be well-defined and would avoid problems. If you mean by sharing files over an USB-stick then it's not so much an application problem as an OS issue. I don't think the argument about machines wanting bytes is true. Machines will accept anything as long as it is well-defined. I'm really curious why there isn't yet some Linux or Posix standard that mandates utf-8. What's the problem with just decreeing that version +1 of the standard now expects utf-8?
- twic 13y agoWhat do you mean by "share files between machines"? Commit files into a repository on one machine. Move it to another on a USB stick, by FTP, with the DVCS's transport protocol, whatever. All of those result in repositories containing byte-for-byte identical commits. In that case the protocol over the wire should be well-defined and would avoid problems. Oh, all of these are well-defined. They're defined to produce filenames which comprise the same sequence of bytes everywhere. That's the problem! If you mean by sharing files over an USB-stick then it's not so much an application problem as an OS issue. Bear in mind that the problem is not what the OS does with the names of files being moved around, it's with what the DVCS does with the names that are embedded in the content of its data files.
- andreasvc 13y agoAh, thanks for clarifying.
- marcosdumay 13y agoWell, honestly, the other way (interpreting filenames) would create worse interoperability problems.