18 ms·

The weird world of Windows file paths

435 points by sharevari 4y ago

deleted 4y ago
[deleted]
EvanAnderson 4y ago
In UNC paths you can append “$NOCSC$” to the hostname to force the client to bypass the “Offline Files” cache. (There are probably other wild undocumented bits like this one hiding in other places in the Windows stack.)
- mdaniel 4y ago
  Do you happen to have a source where you learned that? I'm always interested in "teaching myself to fish"
  - c0nsumer 4y ago
    Lots of this stuff we just find out while working various deep MS cases, and then the info just leaks out.
  - EvanAnderson 4y ago
    I don't recall. Like the other reply to you says, these get leaked in support, etc. I'll also run "strings" or even Ghidra on closed-source binaries when I'm troubleshooting issues. There's usually good fun to be had from Microsoft binaries doing that. I've discovered undocumented debugging switches, registry entries, etc. (In version 10.0.19041.985 of cscsvc.dll in Windows 10 I'm seeing the string "If you hit this breakpoint, send debugger remote to BrianAu." Presumably that's "Brian Aust", referenced in a chat[0] re: Offline Files.) [0] https://techcommunity.microsoft.com/t5/storage-at-microsoft/offline-files-chat-july-10-2007/ba-p/423395 https://techcommunity.microsoft.com/t5/storage-at-microsoft/...
softfalcon 4y ago
After years of working exclusively on Windows, I took a job that required me to build file management, except now, on macOS and Linux (along with Windows). All I can say is, this article is the tip of the ice berg on Windows I/O weirdness. You really don't realize how strange it is until you are actively comparing it to an equivalent implementation on the two other competing operating systems day-to-day. Each of them has their quirks, but I think Windows takes the cake for "out there" hacks you can do to get things running. I sometimes like to ponder what the business case behind all of them was, then I Google it and find the real reasons are wilder than most ideas I can imagine. Fun stuff!
- ChuckNorris89 4y ago
  From my experience as an EE, working with serial ports is much nicer on Windows (COM1, COM2, etc.) than on Linux where serial ports are abstracted to files like /dev/ttyACM0 and has a lot more gotchas. PowerShell is also quite a powerful alternative to Bash/Mingw, although it came out much later. Windows might do some things differently than UNIX-like OSs, but it does them really well.
  - klysm 4y ago
    I’ve found the assignment of com ports in windows really annoying
  - mikub 4y ago
    I just started with using serial ports on windows while doing some Raspberry Pico hobby projects. Something that I find strange is that every new device gets assigned a new comport, I mean let's say I do this for a while one day I will have a comport 100, 200 and so on. Is that right, or does it somehow reset the comports?
    - zwieback 4y ago
      That's how it works and generally it's to the user's advantage. We often set specific parameters based on the device's serial number so getting the same COM port is nice, sometimes the devices are so simple that you cannot query its serial number. Sometimes I'll do a "blank slate" and delete all my accumulated COM ports in Device Manager (need to enable "Show Hidden Devices").
e4m2 4y ago
A lower-level, more security oriented look at some of the same issues: https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html https://googleprojectzero.blogspot.com/2016/02/the-definitiv...
nicetryguy 4y ago
As a related anecdote, you NEVER want to put whitespace in filepaths. Ever. Tons of programs will just break, even in 2023, even Microsoft ones (looking at you Powershell), and i'd imagine forevermore. Your life will end up as a series of awful MYPATH~1\ kludges. You have been warned. EDIT: There seems to be different camps here and perhaps a generational divide. Maybe the kids haven't (or never will be) burned with this one, but i've seen too much, wasted too many hours, wrote too many workarounds, and will forever remain #TeamNoWhiteSpace.
- eru 4y ago
  Tools in the Unix world aren't too robust in that regard either. Gnu Make is particularly egregious.
  - Joker_vD 4y ago
    There is also gnu patch: the filenames in the unified patch file format can't contain spaces and can't be quoted so... just don't use spaces, I guess.
    - mdaniel 4y ago
      how old is your copy of patch? $ cat > my.patch <<PATCH diff --git a/alpha beta/charlie b/alpha beta/charlie new file mode 100644 index 0000000..3b18e51 --- /dev/null +++ b/alpha beta/charlie @@ -0,0 +1 @@ +hello world PATCH $ patch -p1 < my.patch patching file 'alpha beta/charlie' $ patch --version GNU patch 2.7.6
- dr-detroit 4y ago
  [dead]
- ygra 4y ago
  Curious about PowerShell misbehaving there. Care to share any details? Typically you never want to handle paths to files as strings, or at least resolve them to File/DirectoryInfo objects as soon as possible. But the only real blunder regarding paths in PowerShell I'm aware of is the Path parameter to various cmdlets and having [] in file names (which is why LiteralPath always exists alongside Path parameters).
c0nsumer 4y ago
This is a great article and really illustrates just how hard Windows works to be backwards compatible. Lots of these (eg: the COM/LPT stuff) could be dropped and wouldn't affect most people either way, but for those things depending on it, it would be a profoundly breaking change.
- neerajsi 4y ago
  Newest versions of Windows do let you use these names.
  - c0nsumer 4y ago
    How new? 'echo foo > COM1' returns 'The system cannot find the file specified.' on Windows 11. (Machine doesn't have a COM1; if this wasn't being redirected to the port, it'd have gone into a file of that name.)
    - ChrisSD 4y ago
      You can now use ".\COM1" or "COM1.txt" but not a bar COM1.
      - c0nsumer 4y ago
        Thanks! That's good to know.
tragomaskhalos 4y ago
No coverage of this nonsense would be complete without also mentioning that CON, AUX and PRN and a couple of others are verboten as filenames in Win. Although apparently you can defeat this via e.g. \\?\C:\con
- dr-detroit 4y ago
  [dead]
- loudgas 4y ago
  The article mentions this in the Disallowed Names section.
- jeroenhd 4y ago
  One of the early stupid annoying teenager programs I wrote was a tool that would spam your desktop with CON.001, CON.002, and so on through the \\?\ trick. Windows explorer could not delete the file. You have to specify the \\?\ path to get the delete call to work, but that didn't work well with cmd.exe's `del` command. I've since used these files to create directories that can't be deleted by automated cleanups and such, like a special folder in %TEMP% that one program needed but didn't create on its own.
dataflow 4y ago
The (second-)worst offense I'm aware of here is that alternate data stream names can have otherwise special characters in them, like backslashes. So if you (for example) want to strip off the last path component, you technically cannot do this by just stripping everything after the last backslash. In fact this probably isn't the worst thing - it's even worse than this. Because you first need to strip off the prefix that represent the volume (like C:\) before you can look for a colon. But the prefix can be something like \\.\C:\ or \\.\HarddiskVolume2\, or even \\?\GLOBALROOT\DosDevices\HarddiskVolume2\. Or it can be any mount point inside another volume! (Remember that feature inside Disk Management?) Moreover you can't even assume the colon and alternate data streams are even a thing on the file system - it's an NTFS feature. So you gotta query the file system name first. And if the file system is something else with its own special syntax you don't know, then in general you can't find the file name and strip the last component at all. All of which I think means it's impossible to figure out the prefix length without performing syscalls on the target system, and that the answer might vary if the mounts change at run time.
- flangola7 4y ago
  What is an alternate data stream?
  - dataflow 4y ago
    C:\foo has a default (primary) data stream; the name of that stream is empty, so it's omitted entirely when writing the name. But the file can also have C:\foo:bar on NTFS. It's a different stream that's part of the original. (Look up "NTFS ADS" or just "NTFS streams".) These are often used to store information tied to a file that shouldn't affect the file contents.
    - KMag 4y ago
      In the late 1990s, there was a bug in MS IIS where if you requested http://example.com/page.php http://example.com/page.php , it would execute the PHP script, but if you requested http://example.com/page.php http://example.com/page.php: , it would give you the PHP source code. Even more than today, it was common to hard-code database connection details, including passwords, into the source code.
mdaniel 4y ago
> UNC paths have a peculiar way of indicating the drive letter, we must use $ instead of :. I don't believe that's true, I am almost positive they're SMB shares, just like any other, but are created by the system, which is why "accessing drives in this way will only work if you’re logged in as an administrator."
- kunwon1 4y ago
  The dollar sign indicates that the share is 'hidden' and can't be enumerated by traditional means. The C$ share is created by default and provides root level access to the system drive, and is locked down by default for this reason you are correct that they are just SMB shares like any other. They can be removed, though many management processes across different applications assume that those shares will be present
banana_giraffe 4y ago
The bit about "UNC Paths" is a bit simplified. The "$" shares are administrative shares. They're created by default, you can delete or disable them (though, if you delete them, they'll be recreated on a reboot). You can also add normal users to them. It should also be noted that while the single driver letter ones are automatically created, the "$" at the end just marks them as hidden. You can create your own hidden shares if you ever want to.
- masfuerte 4y ago
  You can also change the permissions on them so you don't need to be an admin to access them.
zwieback 4y ago
Needs mention of auto-generated short file names (8.3 alias, typically with ~ in them) on volumes that don't support long names
- Someone 4y ago
  Not only there. Windows originally created them on disks that supported long names, too. That was necessary to support the use case where an older OS tried to read the disk (could happen because the user rebooted into an old DOS, for example, or if an external disk was moved to a different computer) https://en.wikipedia.org/wiki/8.3_filename#VFAT_and_computer-generated_8.3_filenames https://en.wikipedia.org/wiki/8.3_filename#VFAT_and_computer...: “VFAT, a variant of FAT with an extended directory format, was introduced in Windows 95 and Windows NT 3.5. It allowed mixed-case Unicode long filenames (LFNs) in addition to classic 8.3 names by using multiple 32-byte directory entry records for long filenames (in such a way that only one will be recognised by old 8.3 system software as a valid directory entry). To maintain backward-compatibility with legacy applications (on DOS and Windows 3.1), on FAT and VFAT filesystems an 8.3 filename is automatically generated for every LFN, through which the file can still be renamed, deleted or opened, although the generated name (e.g. OVI3KV~N) may show little similarity to the original. On NTFS filesystems the generation of 8.3 filenames can be turned off. The 8.3 filename can be obtained using the Kernel32.dll function GetShortPathName“
  - zwieback 4y ago
    Right. Back in the 90s I worked on a network server to allow AppleTalk clients into DOS or OS/2 based networks. The Mac users enjoyed their filename freedom but the PC clients had trouble with the super-weird 8.3 short names. You couldn't really tell what the Mac filenames were. The other direction worked great, though, DOS filenames always worked on the Mac side of the network.
eduction 4y ago
The fact these paths are considered at all "weird" just underlines how much we live in a Unix world. Filesystem paths used to all be weird in the sense that there was more OS diversity. I'm sure some people here remember that classic MacOS paths used colon as the separator: Hard Drive:My Folder:My Document VMS (designed by the same person as Windows NT by the way) had paths that looked like this (per Wikipedia): NODE"accountname password"::device:[directory.subdirectory]filename.type;ver RSTS/E had [project,user] in the filename. Multics paths: >dir1>dir2>dir3>filename Apple Lisa used dashes as the separator. Etc.
- jeroenhd 4y ago
  As someone who grew up with Windows, I don't think these paths are that weird at all. Drive letter working directories just make sense, for example. The weirdest part is the (edit: HFS) compatibility mode (file.ext:substream). One fun surprise is that because of codepage reasons, the Windows will use ¥ as a path separator in Japanese. In Korean, it's ₩. These characters represent U+005C, which is \ in Latin-compatible character sets.
  - eduction 4y ago
    that's amazing. and point taken that Windows is probably extra weird due to its longevity, evolution, and backward compatibility.
    - deleted 4y ago
      [deleted]
    - PaulDavisThe1st 4y ago
      Windows is younger than Unix, and Unix filesystem evolved has "evolved less" due to getting it right the first time, removing backward compatibility issues.
  - naikrovek 4y ago
    that's not a compatibility thing, is it? it's just the alternate streams feature that NTFS implemented.
throwaway09223 4y ago
The bit about allowing / as a path separator is one of my favorite bits of DOS/Windows trivia. As a unix guy it's fun to give a windows person a path with the slashes wrong like "z:/foo/bar", being corrected for a unix-ism, then having it actually work! In practice I think the biggest problem with using forward slashes on Windows is confusing programs which expect "/" to indicate program switches. The non-uniformity of shell parsing is also a big unix/win design difference.
- naikrovek 4y ago
  yeah almost no one knows that forward slashes have been acceptable as path separators since (I wanna say) Win95. perhaps MSDOS.
  - throwaway09223 4y ago
    It actually predates MSDOS, and I believe dates back to PCDOS 2 when support for directories was first added.
  - zwieback 4y ago
    I'd qualify that as "almost no non-programmers know". Forward slashes are so useful in languages that use \ as an escape sequence that most programmers do know this.
  - layer8 4y ago
    It doesn’t work everywhere. For example tab completion in cmd.exe doesn’t work for a path containing forward slashes (even when quoted), because forward slash is the prefix character for command-line options.
    - naikrovek 4y ago
      right but that's a cmd.exe thing, not a Windows thing. Windows supports it, CMD doesn't. programs that you run from a CMD prompt support other options flag syntaxes, so it's just a cmd.exe feature. CMD.exe is its own thing with its own backwards compatibility requirements and the case could be made that cmd.exe is "Windows" as much as anything else is, so I get it.
cryptonector 4y ago
> Say you, for whatever incomprehensible reason, need to access a file named .., which would normally be resolved to the parent directory during normalisation, you can do so via a literal device path. Oh no. No. Windows allows files to be named `..`?!
- Karellen 4y ago
  > Windows allows files to be named `..`?! Maybe not? Under unix, if you create a symlink to a directory, e.g. `~/syslogs` is a symlink to `/var/log`, then `..` can be used to traverse the "true" parent directory. So `~/syslogs/../lib` will traverse `/var/log/..` and refer to `/var/lib`, not to `~/lib`. However, a "normalising" path interpreter will just take something like `~/syslogs/../lib` and change it to `~/lib` without consulting the filesystem. Given that (AIUI) Windows has supported symlinks for a while now (?), it's possible that files called `..` aren't actually allowed, but the ability to access `..` is still necessary. (Notably, the article does point out that filenames ending in `.` are disallowed - which should exclude `..` as a name one can give a file.)
- jagged-chisel 4y ago
  Seems like that violates the "can't end in a period" rule
  - chungy 4y ago
    Just using Cygwin will show that file names can indeed end in periods (and spaces). The article is very much restraining itself to the standard limitations imposed by the Win32 API, but not what the operating system actually allows. Case sensitivity has always been a thing, since Windows NT 3.1, for example; the "forbidden" characters are not so forbidden with the right file access APIs.
- Jap2-0 4y ago
  I can't find a way to create one - so if Windows allows that, it's begrudgingly at best. You can put `..` earlier in the file name, though.
- 4y ago
usrusr 4y ago
It's not really a new version of Windows if it does not introduce a new variation of universal path addressing to end all variations of universal path addressing.
NelsonMinar 4y ago
Add \\wsl$ to the list of "weird paths". This one fits in the general schema the article is about but has the special meaning of "your files in the Linux VM hiding in the Windows box" (aka WSL). It's running as a Plan9 network filesystem.
- jeroenhd 4y ago
  Is it a weird path? It's just a simulated computer with hostname wsl$. It looks like this is just a network share server running on the loopback interface. Similarly, \\wsl.localhost\ will also show you your WSL files. The dollar sign has some weird special meaning to the Windows security system (trusted account, local computer, etc., in AD it's related to this mess: https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/the-mysterious-case-of-the-a-k-a-dollar-sign-character-in-hybrid/ba-p/768326 https://techcommunity.microsoft.com/t5/core-infrastructure-a...) but it's essentially just a standard Windows UNC paths.
  - pixl97 4y ago
    $ sign just really means "hide this", though it's commonly used on administrative shares.
helloooooooo 4y ago
This is a better explainer: https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html?m=1 https://googleprojectzero.blogspot.com/2016/02/the-definitiv...
0xbadcafebee 4y ago
If they could slowly start adopting Unix file paths and slowly phase out Windows file paths, I think more people would start to use Windows. I would love to want to use Windows. It's a platform that has first class hardware support and paid support, and it's designed (well, in theory) with users and application platforms in mind. And it actually has a bunch of advanced features that Linux and Mac doesn't have.
- wvenable 4y ago
  Why would it matter? 99% of these are effectively edge cases and feature-wise they're pretty equivalent.
- naikrovek 4y ago
  people don't choose to use Windows or not based on the file and path syntax.
- chasil 4y ago
  Windows was built with a POSIX layer; it already does this. "Broad software compatibility was initially achieved with support for several API 'personalities', including Windows API, POSIX, and OS/2 APIs – the latter two were phased out starting with Windows XP." https://en.wikipedia.org/wiki/Windows_NT https://en.wikipedia.org/wiki/Windows_NT
jaclaz 4y ago
Another lesser known fact: The volume id string (what you get with mountvol) is - at least up to Windows 10[1] a UUID version 1 according to RFC 4122, i.e. time and node based: https://www.famkruithof.net/guid-uuid-make.html https://www.famkruithof.net/guid-uuid-make.html https://www.famkruithof.net/guid-uuid-timebased.html https://www.famkruithof.net/guid-uuid-timebased.html Since windows creates the UUID the first time it "sees" a volume, and - usually - uses the network card MAC as node, by decoding the UUID you can get the MAC address of the PC and the time the volume was seen (this can be useful for forensics, expecially with removable devices and to verify there has been no manipulation of the MountedDevices in the Registry). [1]possibly windows 11 changed that, or at least the UUID's shown in the article are type 4
mixmastamyk 4y ago
The period as the final character limit has gotten me when copying albums to a fat32 drive. Turns out a few album names end in a period. :-/
naikrovek 4y ago
eh some of these are slightly wrong but yeah it's weird. MSDOS included some compatibility for CP\M and everything since has maintained compatibility with the version prior, except for a few exceptions. so even today we have compatibility built into Windows for things that don't exist anymore in any real capacity. Microsoft is very serious about backwards compatibility.
- pjmlp 4y ago
  Old Microsoft was very serious, young blood Microsoft not so much.
  - naikrovek 4y ago
    I am not sure where you are coming from with that comment. Microsoft hasn't broken backwards compatibility on anything that I'm aware of in some time.
    - pjmlp 4y ago
      .NET Framework to .NET Core transition, Xamarin.Forms to MAUI, XNA, the multiple rewrites on the WinRT platform since Windows 8, Windows 8 to 10 users pace drivers framework, .NET Native, C++/CX, Win2D, WinRT for XBox replaced by WinGDK, .NET Sharepoint Framework via a JavaScript one,
      - naikrovek 4y ago
        exactly zero of those things break backwards compatibility. programs written using ANY of those technologies run on Windows 11 unmodified, and they will for Windows 12, too. backwards compatibility means new OS versions can run programs written for older OS versions. backwards compatibility promises do not prevent you from coming up with new ways to write programs.
        pjmlp 4y ago
        Except when the old way no longer works, the bugs don't get fixed,tooling disappears from Visual Studio. I am on the Microsoft ecosystem since MS-DOS 3.3.
dark-star 4y ago
So many things wrong with this article. Some things that I noticed by skimming over it: > UNC paths can also be used to access local drives in a similar way: > \\127.0.0.1\C$\Users\Alan Wilder > UNC paths have a peculiar way of indicating the drive letter, we must use $ instead of :. This is actually incorrect... He's actually accessing some random share that has no real connection to a drive. Yes, sometimes (quite often), the C$ share corresponds to the C: drive's root, but this is by no means given, as one can easily either delete the C$ share, or have it pointing to somewhere else entirely > When the current directory is accessed via a UNC path, a current drive-relative path is interpreted relative to the current root share, say \\Earth\Asia. This is also wrong. There is no "current directory" on an UNC share (which can easily be shown by trying to open a command prompt on a UNC share, it will show an error and start you somewhere on C:\users), and the example he gives just tries to access the share "Asia" on the server "Earth" > Less commonly used, paths specifying a drive without a backslash, e.g. E:Kreuzberg, are interpreted relative to the current directory of that drive. This really only makes sense in the context of the command line shell, which keeps track of a current working directory for each drive. Also wrong, it's not the command line shell that keeps track of the current directories, it's the Windows kernel itself. But I agree that such a scenario is quite useless as you can never be quite sure on what CWD you are on a given drive > For the most part, : is also banned. However, there is an exotic exception in the form of NTFS alternate data streams. Yeah, well, surprise: the ":" is not part of the file name, it's just a separator between filename and stream name. This is like saying that "you cannot have \ characters in a file name, but in directory names it is allowed". No, it's not. It's a separator
- ChrisSD 4y ago
  > There is no "current directory" on an UNC share SetCurrentDirectory allows setting the current directory to a UNC share. https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setcurrentdirectory https://learn.microsoft.com/en-us/windows/win32/api/winbase/... > Also wrong, it's not the command line shell that keeps track of the current directories, it's the Windows kernel itself. But I agree that such a scenario is quite useless as you can never be quite sure on what CWD you are on a given drive Not for a long time. It's set as a special (hidden) environment variable like `=C:=C:\current\directory`. https://devblogs.microsoft.com/oldnewthing/20100506-00/?p=14133 https://devblogs.microsoft.com/oldnewthing/20100506-00/?p=14...
janvdberg 4y ago
These path shenanigans get even trickier when invoking WSL or Cygwin. Here's an alias from my Cygwin .bashrc (which took me way too long to figure out) where both the *nix and Windows style paths are invoked: alias ms='/cygdrive/c/Windows/System32/OpenSSH/ssh.exe -A -i 'C:\Users\me\.ssh\mm-id_rsa' me@myserver'
- mynameisvlad 4y ago
  I believe it’s because it’s executing a Windows program (ssh.exe) located in Cygwin’s mount for your C: drive and that program therefore expects Windows-style paths.
  - janvdberg 4y ago
    I know now :)
- Calzifer 4y ago
  Or MSYS2. Other tricky variants is using Windows commands (which typically use '/' as option marker), e.g. $ tasklist /V ERROR: argument/option invalid - "C:/msys64/V" or the time I wanted to use xmlstarlet and any xpath expression was interpreted as Windows path :( At least for MSYS2 the environment variable MSYS2_ARG_CONV_EXC can be used to prevent conversion. https://www.msys2.org/docs/filesystem-paths/#process-arguments https://www.msys2.org/docs/filesystem-paths/#process-argumen...
usefulcat 4y ago
> That’s what sticking to a policy of total backwards compatibility for several decades gets you! Kind of? Surely unix file path conventions are of a similar age (if not older), yet somehow they seem to exhibit much less weirdness..
- userbinator 4y ago
  That's because the Unix convention is far simpler: everything except / and \0 (the null byte) is allowed in filenames, which are also case-sensitive (exact byte match).
Calzifer 4y ago
One thing that make Windows paths wired is that Windows API, NTFS and most Windows tools have different restrictions on file paths. NTFS would accept almost anything. The Windows API (I think of the old Win32 one) would apply most restrictions the article mentions. But for example not the normalization part. A filename can end with a space, no problem. That lead me once to a minor bug in .NET Framework. One of the path related functions (I think it was Directory.move) did not correctly apply this normalization and could produce directories with trailing whitespace. Good luck removing/fixing those in Windows Explorer.
- pixl97 4y ago
  >NTFS would accept almost anything. So for the longest time Adobe software had random bugs where it would create a series of folders name "Application Data" repeating recursively 3000+ characters deep. Yea, that was fun to try to delete.
  - proactivesvcs 4y ago
    Sounds like it wasn't handling junctions correctly. I wonder what obscure/ancient code caused that.
tpmx 4y ago
This got me to try out out the Fileside app (an Explorer/Finder alternative) from the blog post author. It's available for Windows and Mac. I think this is an interesting space with room for innovation. Fileside starts out with a grid of four directories: Home, Documents, Desktop and Downloads. You can customize and name new grid layouts that are shown in a sidebar for quick switching. This seems like a neat idea for specific recurring manual workflows. It's doesn't seem to be targetted to the minimalist crowd. Directory entries beginning with a dot are visible (but greyed out). Full Unix-style permissions are shown for each entry, etc. It looks like it's Electron-based and implemented in a javascript SPA framework. It doesn't use the default system font (SF Pro) on Mac. A bunch of other things also don't look or behave as you expect. The font weight in the size column maps to each file's relative size. All the way from very thin to very bold. Kind of cute. The path completion seems pretty good - as could be guessed from the blog post. I think this app sometimes confuses power with details/verbosity. There are some gold nuggets in there though.
antx 4y ago
"Legal" characters is a fun topic. I realized at some point that there is a discrepancy between what's allowed on the file system, and "Windows" itself (or, more exactly, the programs running on Windows and using its APIs to communicate with said file system. In this case, NTFS, totally allows for "illegal" characters such as < > : " | ? * etc... pretty much everything except / and \, and \0, I think. This makes for funny situations, where sometimes Windows programs cannot deal with that. At best, they can't read, write or rename them... at worse they'll crash, which is always fun.
unnouinceput 4y ago
Ahhh, the memories of naming my folder AUX and then having my Uni professors with administrative privileges not being able to access my files on that folder. Drove them crazy because they thought there is the secret of the Universe hidden there and they demanded I let them see what's inside, when in reality I only wanted to show off and had nothing at all. Novel Netware on top of DOS, year is 1994 - good times.
QuercusMax 4y ago
I knew windows filesystem layout was super bonkers when I had to explain to fellow devs that on a 64-bit machine, you put the 32-bit libraries in SysWow64, and the 64-bit libraries in system32.
ambicapter 4y ago
This might be some run-of-the-mill weirdness, but I was using a Microsoft file globbing library recently (https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.filesystemglobbing?view=dotnet-plat-ext-7.0 https://learn.microsoft.com/en-us/dotnet/api/microsoft.exten...) and it handed back filepaths with forward slashes (instead of double-backslash), even on Windows (and in .Net Framework). I don't know if this is a library someone developed for .Net Core and forgot that it also was going to be used in .Net Framework? Anyways, another reason I don't like the occasional dip into Windows I have to do at my job (which is 80% mac/linux).
planetafro 4y ago
Was Dave Gahan left off purposely?
layer8 4y ago
The part about trailing periods and spaces being disallowed isn’t quite correct. On an NTFS drive, you can actually do echo > \\?\C:\path\to\file. and echo > "\\?\C:\path\to\file " Similarly, files with such names can be created with Cygwin.
xiwenc 4y ago
Recently we had a bug in an electron app that couldnt call exec on a filepath that contains a space in windows. After 2 days of researching and trying, i still don’t have a solution. I reckon adding nodejs to the fantastic world of windows file IO does not make it any easier. Gonna give it another try this weekend.
deleted 4y ago
[deleted]
sam1r 4y ago
Wow, incredible write up. Not only did I learn a lot - but it was also the perfect product pitch. The “and this is why fileside exists” transition at the end - perfect. No “sign up” or anything. Awesome!
dmarinus 4y ago
GitHub Actions in 2023 has issues with checking out a repository with 256ish path names on Windows runners. whups
Dwedit 4y ago
Here is an excellent article about Paths on Win32: https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html https://googleprojectzero.blogspot.com/2016/02/the-definitiv...
simonsarris 4y ago
One of these tripped me up, the "Disallowed Names" section has a bunch of not-too-wordlike names, except one. When I made carefulwords.com, which I made because I wanted a thesaurus where you could just write eg https://carefulwords.com/book https://carefulwords.com/book for "book" and get the results, I found out the hard way that you cannot make a file named "con" on Windows. Or "con.html", or any file extension. You can try to force this, make it via a script, but then programs like Git will hang when they come across it. So in my thesaurus the actual page is /con-word.html and I just have it rewrite to /con
- ChrisSD 4y ago
  This has actually changed in Windows 11. You can use "con.html" without fear. "con" is still a bit of a problem. ".\con" will work but not a bare "con".
kelsolaar 4y ago
The 260 characters limit has been the bane of my existence. Even though it can be disabled in the registry there are gazillions software built against old APIs that will still not work. You also get really odd bugs when that plague hits you.
ChrisSD 4y ago
I think what's missing from this discussion is an emphasis on how layered Windows paths are. The Win32 paths are like an emulation layer. They parse the given path and produce a kernel path. Win32 implements all the weird history you know and love as well as things like `.` and `..`. You can use the `\\?\` prefix to escape this parsing and pass paths to the kernel. The NT kernel has paths like `\Device\HarddiskVolume2\path\to\file`. NT paths are much simpler. There are no restrictions on paths except that they can't contain empty components (notably they can contain nul). At this layer, `.` and `..` are legit filenames. However, it's the filesystem driver ultimately says what's a valid filename and what isn't.
- easton 4y ago
  You know, I’ve never tried. Can you use the device paths in explorer without assigning a drive letter?
  - ChrisSD 4y ago
    No, not really. It only supports "normal" win32 paths. You would have to mount it to a drive letter or directory.
fisian 4y ago
> Disallowed characters > < > ” / \ | ? * Never allowed A friend once somehow crated a file called <HTML>. I don't know how, but he also couldn't delete or do anything with it.
- kps 4y ago
  They should have fixed that when they went to long file names. It's ridiculous that you can't name a file with its contents' actual title. Random example: http://doi.org/10.1145/327070.327153 http://doi.org/10.1145/327070.327153
- pixl97 4y ago
  There are two possibilities here... Connecting from another operating system that allowed names like that to be corrected to a Windows share. There are a few other possibilities where you boot to Linux using a FS driver for NTFS that allows you to create illegal file names. And/or odd things like WSL/Cygwin.
- MattPalmer1086 4y ago
  A long time ago I found a hidden directory on my file server that some student had created to store their pirated software. This was back in the days of DOS and Novell NetWare. Turns out you could create files with "illegal" (and invisible) characters in the filename. The standard OS utilities would not allow them, but the underlying file system did not care. So you could write a short program to do it. I had to write a utility just to delete it.
fifticon 4y ago
the worst thing about windows paths is how unintegrated it all is. You can navigate into all kinds of weird paths in the win32 com shell (file explorer), which is itself possibly the pinnacle of executed design MS ever achieved. But those paths you build.. You can't copy them to the clipboard, you can't serialize them, you can't move them between various tools, and particularly not to the command prompt nor to socalled Power"shell. If there ever was a continent of independent fiefdoms, windows is it :-/. If you don't know what i am talking about, try navigating to your android phone's image folder in file explorer. Next, then try to USE that path in powershell or cmd, to copy those image files.. good luck. there must certainly have been some moron in charge to make SURE things couldnt interoperate on windows.. in spite of them having explorers design.
proactivesvcs 4y ago
It'd be nice if Microsoft read this list and adjusted their software, like perhaps File Explorer, to be able to read and write this data. Or at least delete it.
sim7c00 4y ago
you can create files and folders with illegal names like '.' or '..' with python. comedy ensues if you try to recursively go into . directory and explorer crashesh -_- . despite windows having rules, appearently not everything needs to play by it despite giving these expectations
noAnswer 4y ago
There is also this can of worms regarding translations. For example C:\Users will be shown as C:\Benutzer on a German system but still be C:\Users on the FS. "C:\Program Files (x86)" will show up as "C:\Programme (x86)" BUT "C:\Program Files" will stay the same (non translated). (I forgot how this topic is called though and on what layer it takes places.) Windows XP already could do it, but didn't for the most part. I remember, like ~18 years ago, I was a sysadmin. One user out of 50 got his "My Doc/Pictures/etc." in English but for me on the File-Server it was all in German. Very confusing.
- ygra 4y ago
  It's the shell (i.e. Explorer) showing a localized name. Although I think in Windows XP it was still baked into the language you installed Windows with and stuck with that.
andrewstuart 4y ago
Ultimately, the thing that really distinguishes Linux from Windows is file paths.
quickthrower2 4y ago
Possibly the most annoying thing about windows w.r.t. this topic is you need a large physical C drive to accomodate future windows bloat. It is very hard to get anything installed to put data on another physical drive. It is impossible to extend C to another physical drive. Realistically you need a 1TB SSD/NVMe as your primary drive. So if you get a laptop you then usually need to get a high end one to offer you that.
chaostheory 4y ago
Starting with Windows 10, it also supports Nix style file paths too
1letterunixname 4y ago
On Ruby, '/' and '\' function identically, presumably due to normalization underneath. File.join(too_long_identifier, some_other_long_identifier).gsub('/', '\\') is often useless and nauseating to read when "#{dir}/#{filename}" is portable enough for most of Ruby's File APIs.
lerax 4y ago
there is nothing more disgraceful, disgusting and unpleasant as Windows file paths
sedatk 4y ago
"On any Unix-derived system, a path is an admirably simple thing: if it starts with a /, it’s a path. Not so on Windows." Then, the author goes on to explain that every path on Windows starts with "\\" instead. :)
javier2 4y ago
Can this tool help me identify duplicates in my photo library?
raydiatian 4y ago
Cus Windows fuckin sucks from the subcutaneous layer down to it’s bone marrow.
teo_zero 4y ago
Amiga used to have the best file-path convention I've ever worked with: drive:dir/file where driver was not a letter like in Windows but rather the drive's label, so you could have Work:Pictures/HN.jpg. To go up to the parent directory you had to use an additional slash. So /xxx was the equivalent of Windows ..\xxx, and you could add more slashes to go further up in the directory tree: Work:a/b/c/d////file was the same as Work:a//file. Drives could be "virtual" ones, similar to "bind" mount points in Unix, that could be associated with multiple positions. E.g. you could assign both System:Libs and Data:MyLibs to the virtual drive LIBS: so that LIBS:xxx would match a file called xxx in either directory. Files used to have a comment field to store kind of extended attributes, but was seldom used IIRC. Wildcards were quite unique, I think ? was like regexp . meaning any char, and # like * but prefixed meaning any number of the _following_ char, so that #? would match anything. I'm sure there were other niceties I can't remember right now.

435 points by sharevari 4y ago

deleted 4y ago

[deleted]

In UNC paths you can append “$NOCSC$” to the hostname to force the client to bypass the “Offline Files” cache. (There are probably other wild undocumented bits like this one hiding in other places in the Windows stack.)

mdaniel 4y ago
Do you happen to have a source where you learned that? I'm always interested in "teaching myself to fish"
- c0nsumer 4y ago
  Lots of this stuff we just find out while working various deep MS cases, and then the info just leaks out.
- EvanAnderson 4y ago
  I don't recall. Like the other reply to you says, these get leaked in support, etc. I'll also run "strings" or even Ghidra on closed-source binaries when I'm troubleshooting issues. There's usually good fun to be had from Microsoft binaries doing that. I've discovered undocumented debugging switches, registry entries, etc. (In version 10.0.19041.985 of cscsvc.dll in Windows 10 I'm seeing the string "If you hit this breakpoint, send debugger remote to BrianAu." Presumably that's "Brian Aust", referenced in a chat[0] re: Offline Files.) [0] https://techcommunity.microsoft.com/t5/storage-at-microsoft/offline-files-chat-july-10-2007/ba-p/423395 https://techcommunity.microsoft.com/t5/storage-at-microsoft/...

softfalcon 4y ago

After years of working exclusively on Windows, I took a job that required me to build file management, except now, on macOS and Linux (along with Windows). All I can say is, this article is the tip of the ice berg on Windows I/O weirdness. You really don't realize how strange it is until you are actively comparing it to an equivalent implementation on the two other competing operating systems day-to-day. Each of them has their quirks, but I think Windows takes the cake for "out there" hacks you can do to get things running. I sometimes like to ponder what the business case behind all of them was, then I Google it and find the real reasons are wilder than most ideas I can imagine. Fun stuff!

ChuckNorris89 4y ago
From my experience as an EE, working with serial ports is much nicer on Windows (COM1, COM2, etc.) than on Linux where serial ports are abstracted to files like /dev/ttyACM0 and has a lot more gotchas. PowerShell is also quite a powerful alternative to Bash/Mingw, although it came out much later. Windows might do some things differently than UNIX-like OSs, but it does them really well.
- klysm 4y ago
  I’ve found the assignment of com ports in windows really annoying
- mikub 4y ago
  I just started with using serial ports on windows while doing some Raspberry Pico hobby projects. Something that I find strange is that every new device gets assigned a new comport, I mean let's say I do this for a while one day I will have a comport 100, 200 and so on. Is that right, or does it somehow reset the comports?
  - zwieback 4y ago
    That's how it works and generally it's to the user's advantage. We often set specific parameters based on the device's serial number so getting the same COM port is nice, sometimes the devices are so simple that you cannot query its serial number. Sometimes I'll do a "blank slate" and delete all my accumulated COM ports in Device Manager (need to enable "Show Hidden Devices").

e4m2 4y ago

A lower-level, more security oriented look at some of the same issues: https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html https://googleprojectzero.blogspot.com/2016/02/the-definitiv...

nicetryguy 4y ago

As a related anecdote, you NEVER want to put whitespace in filepaths. Ever. Tons of programs will just break, even in 2023, even Microsoft ones (looking at you Powershell), and i'd imagine forevermore. Your life will end up as a series of awful MYPATH~1\ kludges. You have been warned. EDIT: There seems to be different camps here and perhaps a generational divide. Maybe the kids haven't (or never will be) burned with this one, but i've seen too much, wasted too many hours, wrote too many workarounds, and will forever remain #TeamNoWhiteSpace.

eru 4y ago
Tools in the Unix world aren't too robust in that regard either. Gnu Make is particularly egregious.
- Joker_vD 4y ago
  There is also gnu patch: the filenames in the unified patch file format can't contain spaces and can't be quoted so... just don't use spaces, I guess.
  - mdaniel 4y ago
    how old is your copy of patch? $ cat > my.patch <<PATCH diff --git a/alpha beta/charlie b/alpha beta/charlie new file mode 100644 index 0000000..3b18e51 --- /dev/null +++ b/alpha beta/charlie @@ -0,0 +1 @@ +hello world PATCH $ patch -p1 < my.patch patching file 'alpha beta/charlie' $ patch --version GNU patch 2.7.6
dr-detroit 4y ago
[dead]
ygra 4y ago
Curious about PowerShell misbehaving there. Care to share any details? Typically you never want to handle paths to files as strings, or at least resolve them to File/DirectoryInfo objects as soon as possible. But the only real blunder regarding paths in PowerShell I'm aware of is the Path parameter to various cmdlets and having [] in file names (which is why LiteralPath always exists alongside Path parameters).

c0nsumer 4y ago

This is a great article and really illustrates just how hard Windows works to be backwards compatible. Lots of these (eg: the COM/LPT stuff) could be dropped and wouldn't affect most people either way, but for those things depending on it, it would be a profoundly breaking change.

neerajsi 4y ago
Newest versions of Windows do let you use these names.
- c0nsumer 4y ago
  How new? 'echo foo > COM1' returns 'The system cannot find the file specified.' on Windows 11. (Machine doesn't have a COM1; if this wasn't being redirected to the port, it'd have gone into a file of that name.)
  - ChrisSD 4y ago
    You can now use ".\COM1" or "COM1.txt" but not a bar COM1.
    - c0nsumer 4y ago
      Thanks! That's good to know.

tragomaskhalos 4y ago

No coverage of this nonsense would be complete without also mentioning that CON, AUX and PRN and a couple of others are verboten as filenames in Win. Although apparently you can defeat this via e.g. \\?\C:\con

dr-detroit 4y ago
[dead]
loudgas 4y ago
The article mentions this in the Disallowed Names section.
jeroenhd 4y ago
One of the early stupid annoying teenager programs I wrote was a tool that would spam your desktop with CON.001, CON.002, and so on through the \\?\ trick. Windows explorer could not delete the file. You have to specify the \\?\ path to get the delete call to work, but that didn't work well with cmd.exe's `del` command. I've since used these files to create directories that can't be deleted by automated cleanups and such, like a special folder in %TEMP% that one program needed but didn't create on its own.

dataflow 4y ago

The (second-)worst offense I'm aware of here is that alternate data stream names can have otherwise special characters in them, like backslashes. So if you (for example) want to strip off the last path component, you technically cannot do this by just stripping everything after the last backslash. In fact this probably isn't the worst thing - it's even worse than this. Because you first need to strip off the prefix that represent the volume (like C:\) before you can look for a colon. But the prefix can be something like \\.\C:\ or \\.\HarddiskVolume2\, or even \\?\GLOBALROOT\DosDevices\HarddiskVolume2\. Or it can be any mount point inside another volume! (Remember that feature inside Disk Management?) Moreover you can't even assume the colon and alternate data streams are even a thing on the file system - it's an NTFS feature. So you gotta query the file system name first. And if the file system is something else with its own special syntax you don't know, then in general you can't find the file name and strip the last component at all. All of which I think means it's impossible to figure out the prefix length without performing syscalls on the target system, and that the answer might vary if the mounts change at run time.

flangola7 4y ago
What is an alternate data stream?
- dataflow 4y ago
  C:\foo has a default (primary) data stream; the name of that stream is empty, so it's omitted entirely when writing the name. But the file can also have C:\foo:bar on NTFS. It's a different stream that's part of the original. (Look up "NTFS ADS" or just "NTFS streams".) These are often used to store information tied to a file that shouldn't affect the file contents.
  - KMag 4y ago
    In the late 1990s, there was a bug in MS IIS where if you requested http://example.com/page.php http://example.com/page.php , it would execute the PHP script, but if you requested http://example.com/page.php http://example.com/page.php: , it would give you the PHP source code. Even more than today, it was common to hard-code database connection details, including passwords, into the source code.

mdaniel 4y ago

> UNC paths have a peculiar way of indicating the drive letter, we must use $ instead of :. I don't believe that's true, I am almost positive they're SMB shares, just like any other, but are created by the system, which is why "accessing drives in this way will only work if you’re logged in as an administrator."

kunwon1 4y ago
The dollar sign indicates that the share is 'hidden' and can't be enumerated by traditional means. The C$ share is created by default and provides root level access to the system drive, and is locked down by default for this reason you are correct that they are just SMB shares like any other. They can be removed, though many management processes across different applications assume that those shares will be present

banana_giraffe 4y ago

The bit about "UNC Paths" is a bit simplified. The "$" shares are administrative shares. They're created by default, you can delete or disable them (though, if you delete them, they'll be recreated on a reboot). You can also add normal users to them. It should also be noted that while the single driver letter ones are automatically created, the "$" at the end just marks them as hidden. You can create your own hidden shares if you ever want to.

masfuerte 4y ago
You can also change the permissions on them so you don't need to be an admin to access them.

zwieback 4y ago

Needs mention of auto-generated short file names (8.3 alias, typically with ~ in them) on volumes that don't support long names

Someone 4y ago
Not only there. Windows originally created them on disks that supported long names, too. That was necessary to support the use case where an older OS tried to read the disk (could happen because the user rebooted into an old DOS, for example, or if an external disk was moved to a different computer) https://en.wikipedia.org/wiki/8.3_filename#VFAT_and_computer-generated_8.3_filenames https://en.wikipedia.org/wiki/8.3_filename#VFAT_and_computer...: “VFAT, a variant of FAT with an extended directory format, was introduced in Windows 95 and Windows NT 3.5. It allowed mixed-case Unicode long filenames (LFNs) in addition to classic 8.3 names by using multiple 32-byte directory entry records for long filenames (in such a way that only one will be recognised by old 8.3 system software as a valid directory entry). To maintain backward-compatibility with legacy applications (on DOS and Windows 3.1), on FAT and VFAT filesystems an 8.3 filename is automatically generated for every LFN, through which the file can still be renamed, deleted or opened, although the generated name (e.g. OVI3KV~N) may show little similarity to the original. On NTFS filesystems the generation of 8.3 filenames can be turned off. The 8.3 filename can be obtained using the Kernel32.dll function GetShortPathName“
- zwieback 4y ago
  Right. Back in the 90s I worked on a network server to allow AppleTalk clients into DOS or OS/2 based networks. The Mac users enjoyed their filename freedom but the PC clients had trouble with the super-weird 8.3 short names. You couldn't really tell what the Mac filenames were. The other direction worked great, though, DOS filenames always worked on the Mac side of the network.

eduction 4y ago

The fact these paths are considered at all "weird" just underlines how much we live in a Unix world. Filesystem paths used to all be weird in the sense that there was more OS diversity. I'm sure some people here remember that classic MacOS paths used colon as the separator: Hard Drive:My Folder:My Document VMS (designed by the same person as Windows NT by the way) had paths that looked like this (per Wikipedia): NODE"accountname password"::device:[directory.subdirectory]filename.type;ver RSTS/E had [project,user] in the filename. Multics paths: >dir1>dir2>dir3>filename Apple Lisa used dashes as the separator. Etc.

jeroenhd 4y ago
As someone who grew up with Windows, I don't think these paths are that weird at all. Drive letter working directories just make sense, for example. The weirdest part is the (edit: HFS) compatibility mode (file.ext:substream). One fun surprise is that because of codepage reasons, the Windows will use ¥ as a path separator in Japanese. In Korean, it's ₩. These characters represent U+005C, which is \ in Latin-compatible character sets.
- eduction 4y ago
  that's amazing. and point taken that Windows is probably extra weird due to its longevity, evolution, and backward compatibility.
  - deleted 4y ago
    [deleted]
  - PaulDavisThe1st 4y ago
    Windows is younger than Unix, and Unix filesystem evolved has "evolved less" due to getting it right the first time, removing backward compatibility issues.
- naikrovek 4y ago
  that's not a compatibility thing, is it? it's just the alternate streams feature that NTFS implemented.

throwaway09223 4y ago

The bit about allowing / as a path separator is one of my favorite bits of DOS/Windows trivia. As a unix guy it's fun to give a windows person a path with the slashes wrong like "z:/foo/bar", being corrected for a unix-ism, then having it actually work! In practice I think the biggest problem with using forward slashes on Windows is confusing programs which expect "/" to indicate program switches. The non-uniformity of shell parsing is also a big unix/win design difference.

naikrovek 4y ago
yeah almost no one knows that forward slashes have been acceptable as path separators since (I wanna say) Win95. perhaps MSDOS.
- throwaway09223 4y ago
  It actually predates MSDOS, and I believe dates back to PCDOS 2 when support for directories was first added.
- zwieback 4y ago
  I'd qualify that as "almost no non-programmers know". Forward slashes are so useful in languages that use \ as an escape sequence that most programmers do know this.
- layer8 4y ago
  It doesn’t work everywhere. For example tab completion in cmd.exe doesn’t work for a path containing forward slashes (even when quoted), because forward slash is the prefix character for command-line options.
  - naikrovek 4y ago
    right but that's a cmd.exe thing, not a Windows thing. Windows supports it, CMD doesn't. programs that you run from a CMD prompt support other options flag syntaxes, so it's just a cmd.exe feature. CMD.exe is its own thing with its own backwards compatibility requirements and the case could be made that cmd.exe is "Windows" as much as anything else is, so I get it.

cryptonector 4y ago

> Say you, for whatever incomprehensible reason, need to access a file named .., which would normally be resolved to the parent directory during normalisation, you can do so via a literal device path. Oh no. No. Windows allows files to be named `..`?!

Karellen 4y ago
> Windows allows files to be named `..`?! Maybe not? Under unix, if you create a symlink to a directory, e.g. `~/syslogs` is a symlink to `/var/log`, then `..` can be used to traverse the "true" parent directory. So `~/syslogs/../lib` will traverse `/var/log/..` and refer to `/var/lib`, not to `~/lib`. However, a "normalising" path interpreter will just take something like `~/syslogs/../lib` and change it to `~/lib` without consulting the filesystem. Given that (AIUI) Windows has supported symlinks for a while now (?), it's possible that files called `..` aren't actually allowed, but the ability to access `..` is still necessary. (Notably, the article does point out that filenames ending in `.` are disallowed - which should exclude `..` as a name one can give a file.)
jagged-chisel 4y ago
Seems like that violates the "can't end in a period" rule
- chungy 4y ago
  Just using Cygwin will show that file names can indeed end in periods (and spaces). The article is very much restraining itself to the standard limitations imposed by the Win32 API, but not what the operating system actually allows. Case sensitivity has always been a thing, since Windows NT 3.1, for example; the "forbidden" characters are not so forbidden with the right file access APIs.
Jap2-0 4y ago
I can't find a way to create one - so if Windows allows that, it's begrudgingly at best. You can put `..` earlier in the file name, though.
4y ago

usrusr 4y ago

It's not really a new version of Windows if it does not introduce a new variation of universal path addressing to end all variations of universal path addressing.

NelsonMinar 4y ago

Add \\wsl$ to the list of "weird paths". This one fits in the general schema the article is about but has the special meaning of "your files in the Linux VM hiding in the Windows box" (aka WSL). It's running as a Plan9 network filesystem.

jeroenhd 4y ago
Is it a weird path? It's just a simulated computer with hostname wsl$. It looks like this is just a network share server running on the loopback interface. Similarly, \\wsl.localhost\ will also show you your WSL files. The dollar sign has some weird special meaning to the Windows security system (trusted account, local computer, etc., in AD it's related to this mess: https://techcommunity.microsoft.com/t5/core-infrastructure-and-security/the-mysterious-case-of-the-a-k-a-dollar-sign-character-in-hybrid/ba-p/768326 https://techcommunity.microsoft.com/t5/core-infrastructure-a...) but it's essentially just a standard Windows UNC paths.
- pixl97 4y ago
  $ sign just really means "hide this", though it's commonly used on administrative shares.

helloooooooo 4y ago

This is a better explainer: https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html?m=1 https://googleprojectzero.blogspot.com/2016/02/the-definitiv...

0xbadcafebee 4y ago

If they could slowly start adopting Unix file paths and slowly phase out Windows file paths, I think more people would start to use Windows. I would love to want to use Windows. It's a platform that has first class hardware support and paid support, and it's designed (well, in theory) with users and application platforms in mind. And it actually has a bunch of advanced features that Linux and Mac doesn't have.

wvenable 4y ago
Why would it matter? 99% of these are effectively edge cases and feature-wise they're pretty equivalent.
naikrovek 4y ago
people don't choose to use Windows or not based on the file and path syntax.
chasil 4y ago
Windows was built with a POSIX layer; it already does this. "Broad software compatibility was initially achieved with support for several API 'personalities', including Windows API, POSIX, and OS/2 APIs – the latter two were phased out starting with Windows XP." https://en.wikipedia.org/wiki/Windows_NT https://en.wikipedia.org/wiki/Windows_NT

jaclaz 4y ago

Another lesser known fact: The volume id string (what you get with mountvol) is - at least up to Windows 10[1] a UUID version 1 according to RFC 4122, i.e. time and node based: https://www.famkruithof.net/guid-uuid-make.html https://www.famkruithof.net/guid-uuid-make.html https://www.famkruithof.net/guid-uuid-timebased.html https://www.famkruithof.net/guid-uuid-timebased.html Since windows creates the UUID the first time it "sees" a volume, and - usually - uses the network card MAC as node, by decoding the UUID you can get the MAC address of the PC and the time the volume was seen (this can be useful for forensics, expecially with removable devices and to verify there has been no manipulation of the MountedDevices in the Registry). [1]possibly windows 11 changed that, or at least the UUID's shown in the article are type 4

mixmastamyk 4y ago

The period as the final character limit has gotten me when copying albums to a fat32 drive. Turns out a few album names end in a period. :-/

naikrovek 4y ago

eh some of these are slightly wrong but yeah it's weird. MSDOS included some compatibility for CP\M and everything since has maintained compatibility with the version prior, except for a few exceptions. so even today we have compatibility built into Windows for things that don't exist anymore in any real capacity. Microsoft is very serious about backwards compatibility.

pjmlp 4y ago
Old Microsoft was very serious, young blood Microsoft not so much.
- naikrovek 4y ago
  I am not sure where you are coming from with that comment. Microsoft hasn't broken backwards compatibility on anything that I'm aware of in some time.
  - pjmlp 4y ago
    .NET Framework to .NET Core transition, Xamarin.Forms to MAUI, XNA, the multiple rewrites on the WinRT platform since Windows 8, Windows 8 to 10 users pace drivers framework, .NET Native, C++/CX, Win2D, WinRT for XBox replaced by WinGDK, .NET Sharepoint Framework via a JavaScript one,
    - naikrovek 4y ago
      exactly zero of those things break backwards compatibility. programs written using ANY of those technologies run on Windows 11 unmodified, and they will for Windows 12, too. backwards compatibility means new OS versions can run programs written for older OS versions. backwards compatibility promises do not prevent you from coming up with new ways to write programs.
      - pjmlp 4y ago
        Except when the old way no longer works, the bugs don't get fixed,tooling disappears from Visual Studio. I am on the Microsoft ecosystem since MS-DOS 3.3.

dark-star 4y ago

So many things wrong with this article. Some things that I noticed by skimming over it: > UNC paths can also be used to access local drives in a similar way: > \\127.0.0.1\C$\Users\Alan Wilder > UNC paths have a peculiar way of indicating the drive letter, we must use $ instead of :. This is actually incorrect... He's actually accessing some random share that has no real connection to a drive. Yes, sometimes (quite often), the C$ share corresponds to the C: drive's root, but this is by no means given, as one can easily either delete the C$ share, or have it pointing to somewhere else entirely > When the current directory is accessed via a UNC path, a current drive-relative path is interpreted relative to the current root share, say \\Earth\Asia. This is also wrong. There is no "current directory" on an UNC share (which can easily be shown by trying to open a command prompt on a UNC share, it will show an error and start you somewhere on C:\users), and the example he gives just tries to access the share "Asia" on the server "Earth" > Less commonly used, paths specifying a drive without a backslash, e.g. E:Kreuzberg, are interpreted relative to the current directory of that drive. This really only makes sense in the context of the command line shell, which keeps track of a current working directory for each drive. Also wrong, it's not the command line shell that keeps track of the current directories, it's the Windows kernel itself. But I agree that such a scenario is quite useless as you can never be quite sure on what CWD you are on a given drive > For the most part, : is also banned. However, there is an exotic exception in the form of NTFS alternate data streams. Yeah, well, surprise: the ":" is not part of the file name, it's just a separator between filename and stream name. This is like saying that "you cannot have \ characters in a file name, but in directory names it is allowed". No, it's not. It's a separator

ChrisSD 4y ago
> There is no "current directory" on an UNC share SetCurrentDirectory allows setting the current directory to a UNC share. https://learn.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setcurrentdirectory https://learn.microsoft.com/en-us/windows/win32/api/winbase/... > Also wrong, it's not the command line shell that keeps track of the current directories, it's the Windows kernel itself. But I agree that such a scenario is quite useless as you can never be quite sure on what CWD you are on a given drive Not for a long time. It's set as a special (hidden) environment variable like `=C:=C:\current\directory`. https://devblogs.microsoft.com/oldnewthing/20100506-00/?p=14133 https://devblogs.microsoft.com/oldnewthing/20100506-00/?p=14...

janvdberg 4y ago

These path shenanigans get even trickier when invoking WSL or Cygwin. Here's an alias from my Cygwin .bashrc (which took me way too long to figure out) where both the *nix and Windows style paths are invoked: alias ms='/cygdrive/c/Windows/System32/OpenSSH/ssh.exe -A -i 'C:\Users\me\.ssh\mm-id_rsa' me@myserver'

mynameisvlad 4y ago
I believe it’s because it’s executing a Windows program (ssh.exe) located in Cygwin’s mount for your C: drive and that program therefore expects Windows-style paths.
- janvdberg 4y ago
  I know now :)
Calzifer 4y ago
Or MSYS2. Other tricky variants is using Windows commands (which typically use '/' as option marker), e.g. $ tasklist /V ERROR: argument/option invalid - "C:/msys64/V" or the time I wanted to use xmlstarlet and any xpath expression was interpreted as Windows path :( At least for MSYS2 the environment variable MSYS2_ARG_CONV_EXC can be used to prevent conversion. https://www.msys2.org/docs/filesystem-paths/#process-arguments https://www.msys2.org/docs/filesystem-paths/#process-argumen...

usefulcat 4y ago

> That’s what sticking to a policy of total backwards compatibility for several decades gets you! Kind of? Surely unix file path conventions are of a similar age (if not older), yet somehow they seem to exhibit much less weirdness..

userbinator 4y ago
That's because the Unix convention is far simpler: everything except / and \0 (the null byte) is allowed in filenames, which are also case-sensitive (exact byte match).

Calzifer 4y ago

One thing that make Windows paths wired is that Windows API, NTFS and most Windows tools have different restrictions on file paths. NTFS would accept almost anything. The Windows API (I think of the old Win32 one) would apply most restrictions the article mentions. But for example not the normalization part. A filename can end with a space, no problem. That lead me once to a minor bug in .NET Framework. One of the path related functions (I think it was Directory.move) did not correctly apply this normalization and could produce directories with trailing whitespace. Good luck removing/fixing those in Windows Explorer.

pixl97 4y ago
>NTFS would accept almost anything. So for the longest time Adobe software had random bugs where it would create a series of folders name "Application Data" repeating recursively 3000+ characters deep. Yea, that was fun to try to delete.
- proactivesvcs 4y ago
  Sounds like it wasn't handling junctions correctly. I wonder what obscure/ancient code caused that.

tpmx 4y ago

This got me to try out out the Fileside app (an Explorer/Finder alternative) from the blog post author. It's available for Windows and Mac. I think this is an interesting space with room for innovation. Fileside starts out with a grid of four directories: Home, Documents, Desktop and Downloads. You can customize and name new grid layouts that are shown in a sidebar for quick switching. This seems like a neat idea for specific recurring manual workflows. It's doesn't seem to be targetted to the minimalist crowd. Directory entries beginning with a dot are visible (but greyed out). Full Unix-style permissions are shown for each entry, etc. It looks like it's Electron-based and implemented in a javascript SPA framework. It doesn't use the default system font (SF Pro) on Mac. A bunch of other things also don't look or behave as you expect. The font weight in the size column maps to each file's relative size. All the way from very thin to very bold. Kind of cute. The path completion seems pretty good - as could be guessed from the blog post. I think this app sometimes confuses power with details/verbosity. There are some gold nuggets in there though.

antx 4y ago

"Legal" characters is a fun topic. I realized at some point that there is a discrepancy between what's allowed on the file system, and "Windows" itself (or, more exactly, the programs running on Windows and using its APIs to communicate with said file system. In this case, NTFS, totally allows for "illegal" characters such as < > : " | ? * etc... pretty much everything except / and \, and \0, I think. This makes for funny situations, where sometimes Windows programs cannot deal with that. At best, they can't read, write or rename them... at worse they'll crash, which is always fun.

unnouinceput 4y ago

Ahhh, the memories of naming my folder AUX and then having my Uni professors with administrative privileges not being able to access my files on that folder. Drove them crazy because they thought there is the secret of the Universe hidden there and they demanded I let them see what's inside, when in reality I only wanted to show off and had nothing at all. Novel Netware on top of DOS, year is 1994 - good times.

QuercusMax 4y ago

I knew windows filesystem layout was super bonkers when I had to explain to fellow devs that on a 64-bit machine, you put the 32-bit libraries in SysWow64, and the 64-bit libraries in system32.

ambicapter 4y ago

This might be some run-of-the-mill weirdness, but I was using a Microsoft file globbing library recently (https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.filesystemglobbing?view=dotnet-plat-ext-7.0 https://learn.microsoft.com/en-us/dotnet/api/microsoft.exten...) and it handed back filepaths with forward slashes (instead of double-backslash), even on Windows (and in .Net Framework). I don't know if this is a library someone developed for .Net Core and forgot that it also was going to be used in .Net Framework? Anyways, another reason I don't like the occasional dip into Windows I have to do at my job (which is 80% mac/linux).

planetafro 4y ago

Was Dave Gahan left off purposely?

layer8 4y ago

The part about trailing periods and spaces being disallowed isn’t quite correct. On an NTFS drive, you can actually do echo > \\?\C:\path\to\file. and echo > "\\?\C:\path\to\file " Similarly, files with such names can be created with Cygwin.

xiwenc 4y ago

Recently we had a bug in an electron app that couldnt call exec on a filepath that contains a space in windows. After 2 days of researching and trying, i still don’t have a solution. I reckon adding nodejs to the fantastic world of windows file IO does not make it any easier. Gonna give it another try this weekend.

deleted 4y ago

[deleted]

sam1r 4y ago

Wow, incredible write up. Not only did I learn a lot - but it was also the perfect product pitch. The “and this is why fileside exists” transition at the end - perfect. No “sign up” or anything. Awesome!

dmarinus 4y ago

GitHub Actions in 2023 has issues with checking out a repository with 256ish path names on Windows runners. whups

Dwedit 4y ago

Here is an excellent article about Paths on Win32: https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html https://googleprojectzero.blogspot.com/2016/02/the-definitiv...

simonsarris 4y ago

One of these tripped me up, the "Disallowed Names" section has a bunch of not-too-wordlike names, except one. When I made carefulwords.com, which I made because I wanted a thesaurus where you could just write eg https://carefulwords.com/book https://carefulwords.com/book for "book" and get the results, I found out the hard way that you cannot make a file named "con" on Windows. Or "con.html", or any file extension. You can try to force this, make it via a script, but then programs like Git will hang when they come across it. So in my thesaurus the actual page is /con-word.html and I just have it rewrite to /con

ChrisSD 4y ago
This has actually changed in Windows 11. You can use "con.html" without fear. "con" is still a bit of a problem. ".\con" will work but not a bare "con".

kelsolaar 4y ago

The 260 characters limit has been the bane of my existence. Even though it can be disabled in the registry there are gazillions software built against old APIs that will still not work. You also get really odd bugs when that plague hits you.

ChrisSD 4y ago

I think what's missing from this discussion is an emphasis on how layered Windows paths are. The Win32 paths are like an emulation layer. They parse the given path and produce a kernel path. Win32 implements all the weird history you know and love as well as things like `.` and `..`. You can use the `\\?\` prefix to escape this parsing and pass paths to the kernel. The NT kernel has paths like `\Device\HarddiskVolume2\path\to\file`. NT paths are much simpler. There are no restrictions on paths except that they can't contain empty components (notably they can contain nul). At this layer, `.` and `..` are legit filenames. However, it's the filesystem driver ultimately says what's a valid filename and what isn't.

easton 4y ago
You know, I’ve never tried. Can you use the device paths in explorer without assigning a drive letter?
- ChrisSD 4y ago
  No, not really. It only supports "normal" win32 paths. You would have to mount it to a drive letter or directory.

fisian 4y ago

> Disallowed characters > < > ” / \ | ? * Never allowed A friend once somehow crated a file called <HTML>. I don't know how, but he also couldn't delete or do anything with it.

kps 4y ago
They should have fixed that when they went to long file names. It's ridiculous that you can't name a file with its contents' actual title. Random example: http://doi.org/10.1145/327070.327153 http://doi.org/10.1145/327070.327153
pixl97 4y ago
There are two possibilities here... Connecting from another operating system that allowed names like that to be corrected to a Windows share. There are a few other possibilities where you boot to Linux using a FS driver for NTFS that allows you to create illegal file names. And/or odd things like WSL/Cygwin.
MattPalmer1086 4y ago
A long time ago I found a hidden directory on my file server that some student had created to store their pirated software. This was back in the days of DOS and Novell NetWare. Turns out you could create files with "illegal" (and invisible) characters in the filename. The standard OS utilities would not allow them, but the underlying file system did not care. So you could write a short program to do it. I had to write a utility just to delete it.

fifticon 4y ago

the worst thing about windows paths is how unintegrated it all is. You can navigate into all kinds of weird paths in the win32 com shell (file explorer), which is itself possibly the pinnacle of executed design MS ever achieved. But those paths you build.. You can't copy them to the clipboard, you can't serialize them, you can't move them between various tools, and particularly not to the command prompt nor to socalled Power"shell. If there ever was a continent of independent fiefdoms, windows is it :-/. If you don't know what i am talking about, try navigating to your android phone's image folder in file explorer. Next, then try to USE that path in powershell or cmd, to copy those image files.. good luck. there must certainly have been some moron in charge to make SURE things couldnt interoperate on windows.. in spite of them having explorers design.

proactivesvcs 4y ago

It'd be nice if Microsoft read this list and adjusted their software, like perhaps File Explorer, to be able to read and write this data. Or at least delete it.

sim7c00 4y ago

you can create files and folders with illegal names like '.' or '..' with python. comedy ensues if you try to recursively go into . directory and explorer crashesh -_- . despite windows having rules, appearently not everything needs to play by it despite giving these expectations

noAnswer 4y ago

There is also this can of worms regarding translations. For example C:\Users will be shown as C:\Benutzer on a German system but still be C:\Users on the FS. "C:\Program Files (x86)" will show up as "C:\Programme (x86)" BUT "C:\Program Files" will stay the same (non translated). (I forgot how this topic is called though and on what layer it takes places.) Windows XP already could do it, but didn't for the most part. I remember, like ~18 years ago, I was a sysadmin. One user out of 50 got his "My Doc/Pictures/etc." in English but for me on the File-Server it was all in German. Very confusing.

ygra 4y ago
It's the shell (i.e. Explorer) showing a localized name. Although I think in Windows XP it was still baked into the language you installed Windows with and stuck with that.

andrewstuart 4y ago

Ultimately, the thing that really distinguishes Linux from Windows is file paths.

quickthrower2 4y ago

Possibly the most annoying thing about windows w.r.t. this topic is you need a large physical C drive to accomodate future windows bloat. It is very hard to get anything installed to put data on another physical drive. It is impossible to extend C to another physical drive. Realistically you need a 1TB SSD/NVMe as your primary drive. So if you get a laptop you then usually need to get a high end one to offer you that.

chaostheory 4y ago

Starting with Windows 10, it also supports Nix style file paths too

1letterunixname 4y ago

On Ruby, '/' and '\' function identically, presumably due to normalization underneath. File.join(too_long_identifier, some_other_long_identifier).gsub('/', '\\') is often useless and nauseating to read when "#{dir}/#{filename}" is portable enough for most of Ruby's File APIs.

lerax 4y ago

there is nothing more disgraceful, disgusting and unpleasant as Windows file paths

sedatk 4y ago

"On any Unix-derived system, a path is an admirably simple thing: if it starts with a /, it’s a path. Not so on Windows." Then, the author goes on to explain that every path on Windows starts with "\\" instead. :)

javier2 4y ago

Can this tool help me identify duplicates in my photo library?

raydiatian 4y ago

Cus Windows fuckin sucks from the subcutaneous layer down to it’s bone marrow.

teo_zero 4y ago

Amiga used to have the best file-path convention I've ever worked with: drive:dir/file where driver was not a letter like in Windows but rather the drive's label, so you could have Work:Pictures/HN.jpg. To go up to the parent directory you had to use an additional slash. So /xxx was the equivalent of Windows ..\xxx, and you could add more slashes to go further up in the directory tree: Work:a/b/c/d////file was the same as Work:a//file. Drives could be "virtual" ones, similar to "bind" mount points in Unix, that could be associated with multiple positions. E.g. you could assign both System:Libs and Data:MyLibs to the virtual drive LIBS: so that LIBS:xxx would match a file called xxx in either directory. Files used to have a comment field to store kind of extended attributes, but was seldom used IIRC. Wildcards were quite unique, I think ? was like regexp . meaning any char, and # like * but prefixed meaning any number of the _following_ char, so that #? would match anything. I'm sure there were other niceties I can't remember right now.