5 ms·
That is quite the obscure and interesting issue to run into! Who puts colons in their filenames though? I haven't ever seen that used...
by artifaxx 10y ago
That is quite the obscure and interesting issue to run into! Who puts colons in their filenames though? I haven't ever seen that used...
- Tharkun 10y agoWhy wouldn't you put colons in filenames? Unless of course you use Windows. Colons, spaces, backslashes, whatever.
- belovedeagle 10y agoThe colon should be reserved in Unix to separate path names a la PATH.
- TazeTSchnitzel 10y agoIt isn't, though.
- ambrop7 10y agoThe PATH should be a list of strings not a string where : means something special.
- the_mitsuhiko 10y agoAnd then how do you escape it?
- zeveb 10y agoIn a sane system, PATH=/home/me/bin:/usr/local/bin:/usr/bin would be (setf bin-dirs '(#P"/home/me/bin/" #P"/usr/local/bin/" #P"/usr/bin/")) and if I wanted to have a quote in a file I could just write #P"/really-weird-name\"-isn't-it台北/bin/".
- geofft 10y agoTransmit it as a JSON list or whatever. We're already assuming we're breaking UNIX userspace compatibility, so any option for reliably transmitting lists of strings is fine, and we have lots of those.
- ambrop7 10y agoBy list I mean an abstract data structure, you don't "escape" it. You could encode it in a specific format like JSON, if you want.
- yrro 10y agoNUL
- TazeTSchnitzel 10y agoUseful if only for timestamps. 2016-07-20T16:27:00Z!
- 7952 10y agoArtificially restricting filenames is an antipattern that makes things harder to read. I use software that sometimes (but mostly not) needs files in dos 8.3. Because of this people seem to think it a good idea to use really short acronymed file names as a matter of course. If it makes sense to use a special character then people should be able to.
- artifaxx 10y agoI agree if you are designing for a single operating system. The restriction isn't artificial if the software under design is multi-platform, since Windows for example doesn't work with colons in file names.
- xg15 10y agoThis would be the way to go if all program invocations consistently used some kind of common, higher-level data structure a la powershell. As long as programs rely on parsing their command line according to some syntax the developer just made up, I'm very glad there are commonly agreed sets of "safe" and "unsafe" characters. Dealing with shell escapes is horrible enough today (e.g. quoting rules under windows, filenames that start with a dash under linux...) and this would make matters even worse.
- wahern 10y agoQuoting is a much bigger problem than differentiating flags from paths, and on Unix that's a solved problem: the shell always handles quoting, and unix programs only expect a list of words which can contain arbitrary characters (except NUL, of course). If you invoke a program directly using the exec-family of syscalls, you don't need to quote anything. Whereas AFAIU Windows programs expect quoted words to be passed via main(), and must parse them. The only benefit is that you can disambiguate a filename with a dash (or slash) based on whether it was escaped, but that's a quite rare necessity, and of course still relies on the caller quoting them. (Does cmd.exe quote pathname expansions?) The dash problem is also solved as long as programs use getopt() or getopt_long(). First, getopt() knows which flags take arguments and which don't. Knowing this, if a flag takes an argument it doesn't matter whether the argument begins with a dash or not. One consequence is that there's no such thing as an "optional" argument to a flag when using getopt and friends, as that ambiguity cannot be handled cleanly. People who roll their own argument processing code just so they can get "optional" arguments to flags invariably don't appreciate the security problem. Second, a double-dash (--) terminates the argument list. getopt stops consuming command-line arguments at that point, and optind will index the first non-flag argument. So if passing a list of filenames to a command, the correct idiom in Unix is something like, `foo -- /path/to/*`. Of course, that presumes that the foo is using getopt or getopt_long, or a compatible argument processing implementation. Fortunately the vast majority do. Smart programmers should rarely if ever roll their own argument processing code. Any headaches (real or imagined) related to a mismatch between the semantics offered by getopt and what the application might want is usually dwarfed by the usability and security benefits of adhering to the system facilities. On a related note, I've always disliked the way GNU's getopt and getopt_long permuted (reordered) argument lists. I have an inkling it could introduce needless security issues, though I haven't thought it through carefully.
- artifaxx 10y agoI assume any files I make should be usable by any major operating system, so to be sure I avoid any special characters in file names.
- amock 10y agoMac OS 9 and earlier used colons as path separators and some support for that made it into OS X, although it might be gone by now. Apple's guidelines https://support.apple.com/en-us/HT202808 https://support.apple.com/en-us/HT202808 recommend not using colons.
- lloeki 10y agoIt's not really gone, and is the path separator in the "UI layer" (Cocoa&al.) but the POSIX layer uses `/`. This gets automatically mapped back and forth. Try to create a file or folder containing `/` in the UI (Textedit, Finder) and look at its name via the terminal. Now, `touch foo:bar` and look at it in the Finder.
- lscharen 10y agoAlso from the Apple lineage, GS/OS allowed either slashes or colons as valid path separators. See the last paragraph on page 109 of the GS/OS Internals manual. http://www.brutaldeluxe.fr/documentation/gsos/Apple_IIgs_GSOS_internals_v1.pdf http://www.brutaldeluxe.fr/documentation/gsos/Apple_IIgs_GSO...
- Pxtl 10y agoCould say the same about spaces and a plethora of systems fail catastrophically on those.
- nilved 10y agoNot any well-written ones. Software should be able to handle all valid names on the platform. For Unix, any name that excludes forward slashes and null bytes.
- awqrre 10y agoDo you also use slashes? what about backticks and quotes?
- LukeShu 10y agoThe Maildir format uses colons in filenames, which has created problems with running certain email software on Windows.
- lorenzhs 10y agoI do: "find ~ -iname ":" | wc -l" yields 49 entries. Some of those are Xorg-related (which identifies displays with a syntax like ":0.0", which ends up in some log file names) or gvfs stuff. But most are PDFs, usually research papers which I tend to save as "title: subtitle - authors.pdf".
- artifaxx 10y agoIt is certainly interesting to see the different assumptions people make. Thanks for the details.
- RX14 10y agoHigh score! In all my filesystem I get about 150,000 files with a colon in their name. Many of them are part of the filesystems of containers which seem to use colons in the names of their libraries and dpkg packages. Many archlinux packages have colons in their names too. I also have some media files and documents with colons in names too. It's actually really common, for someone who's used *nix for ages and expects every character to be a valid filename.
- marcosdumay 10y agoLots of people, that's who. Besides the other responses, colon is a standard path separator on URIs. If you need more than one kind of them (the obvious one being a slash), the colon is often the most reasonable option. And if you decide to save data on disk, with parts of the URI as file-name (what is also very reasonable)... Probably, the main reason this problem does not pop everywhere is that people hacking completely new tools rarely do that on Windows. And when they port, it gets hidden together with another hundred other little incompatibilities.
- msravi 10y agoWell, it isn't uncommon to create a log file with a timestamp and a name like foobar.$datetime.log with $datetime expanding to something like 2016-07-20T22:08:38
- mtone 10y agoAs a lazy user, I often copy paste the document title as the filename. Windows generously trims any disallowed character, but if it didn't, lots of my files would contain colons. I actually wish colons were supported, since it's so prevalent in document titles. Question marks, too, while at it.
- hashhar 10y agoI think I read somewhere that Microsoft were investigating ways in which to get rid of the FAT32 backward compatibility issues with NTFS like the 255 character path element limit and the character limits in filenames. You can manage such files using the NTFS way of addressing them, "\\?\C:\Example\file:with?illegalstuff"
- nitrogen 10y agoIs there an official name for those kinds of long paths in a single namespace? I found UNC, but it seems to apply only to the network variety. https://en.wikipedia.org/wiki/Path_(computing)#Uniform_Naming_Convention https://en.wikipedia.org/wiki/Path_(computing)#Uniform_Namin...
- thoth 10y agoI'm not sure Microsoft gives an official name for "\\?\". The closest I can find is "To specify an extended-length path, use the "\\?\" prefix." (from https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...). So maybe it is the "extended-length path prefix".
- cataphract 10y agoYou can also use that prefix to access volumes that have no mount point, as in \\?\Volume{<UUID>}, among other things like UNC paths (\\?\UNC\...). So that's not a good name.
- deleted 10y ago[deleted]
- zeveb 10y agoI use colons when indicating the ISO 8601[1] timestamp for stuff; it's much more readable with ':' than not, e.g. 2016-07-20T17:04:30Z vice 20160720T170430Z. I also use them when naming stuff with hashes or UUIDs. Not having colons in filenames seems just weird to me. Heck, even on Unix I'm annoyed that I can't simply escape slashes! It'd be nice to name files with the URLs they are taken from. [1] https://en.wikipedia.org/wiki/ISO_8601 https://en.wikipedia.org/wiki/ISO_8601
- kazinator 10y agoWho puts colons into their filenames? Oh, short-sighted people like the designers of OBS (OpenSUSE Build System) who introduced a colon convention into project names. http://lists.opensuse.org/opensuse-buildservice/2008-12/msg00166.html http://lists.opensuse.org/opensuse-buildservice/2008-12/msg0... This causes a problem even in the POSIX environment, because the colon is used in PATH and PATH-like environment variables. Usually there is no escape mechanism.
- Houshalter 10y agoI try to save papers sometimes and give the filename the same name as the paper, which often has a colon in it.
- chippy 10y ago> Who puts colons in their filenames though? Twitter does https://pbs.twimg.com/media/Cn1tGFoXgAApMt3.jpg:large https://pbs.twimg.com/media/Cn1tGFoXgAApMt3.jpg:large