5 ms·

Issues with Cross-Platform Version Control

23 points by nreece 18y ago

durin42 18y ago
A good basic summary of this would be "don't use anything other than basic ASCII in filenames." There are two very common desktop platforms (OS X, Windows) that have outright broken filename handling (OS X in particular has really odd rules for storing Unicode in filenames compared to, oh, the world). Anything requiring character encoding in a filename, much like a source code file, is a bag of hurt waiting to happen. I know there have been bugs in Subversion related to filename encoding issues, and it's been controversial in the hg world as well, where the formal solution is to treat filenames as opaque bytes because anything else becomes painful rapidly. (Unsurprisingly, this problem shows itself in other places. os.listdir() in Python 3 returned Unicode strings, which is full of suck because on a Linux system filenames are bags of bytes, so two files in a directory could very well be in different encodings. Until recently (at least, have not checked recently) os.listdir() would simply silently ignore any file whose name didn't properly decode into a Unicode from the on-disk bytes.)
- yummyfajitas 18y ago
  Quite right. The simplest solution at the level of version control is this: >hg add PRN Error: cannot add file named "PRN" - will cause cross platform errors. >hg add filename. Error: cannot add file ending with "." - will cause cross platform errors. >hg add "-rf *" Error: files cannot begin with "-" - will cause cross platform errors. Most people will never see these errors.
jonhohle 18y ago
> checkin a file with a name that begins with a dash. Now check it out under Linux and try manipulating it with command-line utilities $> echo 'hello world' > -h $> cat ./-h hello world what's the problem?
- wmf 18y ago
  Try "cat *" and see what happens. This has more details than you ever wanted to know about filenames: http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html http://www.dwheeler.com/essays/fixing-unix-linux-filenames.h...
  - jonhohle 18y ago
    if I expected a filename with a leading dash, I'd follow the same convention as I did before: "cat ./*". You're link suggests the same thing.
- blasdel 18y ago
  All of the utilities that use GNU getopt support using ' -- ' to delimit the end of the parameters: $> echo 'hello world' > -h $> cat -- -h hello world
thwarted 18y ago
> What if somebody creates a file named "-rf *" and a Linux user tries to rm it? They'll delete their working copy if the naive Linux user doesn't properly escape that. At the worst, they'll loose some recent changes and have to re-checkout. This is why you're using version control, right?
pmjordan 18y ago
It'd be nice if he'd gone into a little detail on how to solve these issues rather than just listing problems. Interesting stuff nonetheless, some of which I've been aware, some are news to me. (I've got a project using git but originating from a svn repo that I work with on Windows, Linux and Mac, which required a couple of contortions initally)

23 points by nreece 18y ago

A good basic summary of this would be "don't use anything other than basic ASCII in filenames." There are two very common desktop platforms (OS X, Windows) that have outright broken filename handling (OS X in particular has really odd rules for storing Unicode in filenames compared to, oh, the world). Anything requiring character encoding in a filename, much like a source code file, is a bag of hurt waiting to happen. I know there have been bugs in Subversion related to filename encoding issues, and it's been controversial in the hg world as well, where the formal solution is to treat filenames as opaque bytes because anything else becomes painful rapidly. (Unsurprisingly, this problem shows itself in other places. os.listdir() in Python 3 returned Unicode strings, which is full of suck because on a Linux system filenames are bags of bytes, so two files in a directory could very well be in different encodings. Until recently (at least, have not checked recently) os.listdir() would simply silently ignore any file whose name didn't properly decode into a Unicode from the on-disk bytes.)

yummyfajitas 18y ago
Quite right. The simplest solution at the level of version control is this: >hg add PRN Error: cannot add file named "PRN" - will cause cross platform errors. >hg add filename. Error: cannot add file ending with "." - will cause cross platform errors. >hg add "-rf *" Error: files cannot begin with "-" - will cause cross platform errors. Most people will never see these errors.

jonhohle 18y ago

> checkin a file with a name that begins with a dash. Now check it out under Linux and try manipulating it with command-line utilities $> echo 'hello world' > -h $> cat ./-h hello world what's the problem?

wmf 18y ago
Try "cat *" and see what happens. This has more details than you ever wanted to know about filenames: http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html http://www.dwheeler.com/essays/fixing-unix-linux-filenames.h...
- jonhohle 18y ago
  if I expected a filename with a leading dash, I'd follow the same convention as I did before: "cat ./*". You're link suggests the same thing.
blasdel 18y ago
All of the utilities that use GNU getopt support using ' -- ' to delimit the end of the parameters: $> echo 'hello world' > -h $> cat -- -h hello world

thwarted 18y ago

> What if somebody creates a file named "-rf *" and a Linux user tries to rm it? They'll delete their working copy if the naive Linux user doesn't properly escape that. At the worst, they'll loose some recent changes and have to re-checkout. This is why you're using version control, right?

pmjordan 18y ago

It'd be nice if he'd gone into a little detail on how to solve these issues rather than just listing problems. Interesting stuff nonetheless, some of which I've been aware, some are news to me. (I've got a project using git but originating from a svn repo that I work with on Windows, Linux and Mac, which required a couple of contortions initally)