5 ms·

See here for some notes from the WSL team why certain filesystem operations that would be fast on Linux are slow on Windows: https://github.com/microsoft/WSL/i

• nivenhuh 5y ago
  Thanks for the links -- one pro-tip that stood out to me was to use the D: drive (because it's likely to have less filter drivers attached). "Windows's IO stack is extensible, allowing filter drivers to attach to volumes and intercept IO requests before the file system sees them. This is used for numerous things, including virus scanning, compression, encryption, file virtualization, things like OneDrive's files on demand feature, gathering pre-fetching data to speed up app startup, and much more. Even a clean install of Windows will have a number of filters present, particularly on the system volume (so if you have a D: drive or partition, I recommend using that instead, since it likely has fewer filters attached). Filters are involved in many IO operations, most notably creating/opening files."
• hughrr 5y ago
  I'm going to have to complain about this because in real world use these are far less of a problem than MFT contention on sub-900 byte files generated by typical unix environments. Particularly things that are lockfile heavy and VCSs etc are the most painful. The WSL thing reads like an excuse here. If you actually go and look at what's happening it's small files which are the damage multiplier. I think the real issue is that you shouldn't mix the two operating system paradigms and it's far better to just run Linux in a VM and benefit from the near native performance at the cost of a tiny bit of inconvenience. It's not a bad option when you consider the remote IDE capabilities that VScode gives you, which is the one product they're doing 100% right.
  • deleted 5y ago
    [deleted]
  • rossy 5y ago
    I'm sorry, but I find that hard to believe when the linked posts are from a Microsoft employee, who says repeatedly that the problem isn't a simple as "NTFS is slow," that they spent a release optimising NTFS, they've gotten rid of all the low hanging fruit and so on. I don't see why they'd be making excuses for NTFS when the underlying problems seem to be much more fundamental to Windows, which is much worse. You say you discovered the MFT contention issue when you were working on Subversion a decade ago, but these posts are from 2018, so I'm inclined to believe that the MFT contention issue has been mitigated and the real issues really are things like filter drivers, like they say. I'm also not sure there's two paradigms here, if by that you mean there are workloads more suited to Linux and workloads more suited to Windows. Are there any file system operations that are actually faster on Windows? I'm still inclined to believe, like they say, that "file operations in Windows are more expensive than in Linux," even if large files are less impacted than small files. I really hope Microsoft can improve this, rather than just throwing it under the rug and saying to use Linux VMs, even if they have to make drastic changes like deprecating filesystem filter drivers completely. I think a lot of us depend on the performance of software that was designed for fast filesystems and ported to Windows (not just Git, either.) I also think a lot of tasks fundamentally use a lot of small files. After all, what is source code?
    • hughrr 5y ago
      No it’s still MFT contention. Someone at my company independently discovered the same problem recently with git on WSL. MSFT don’t listen to or care about their customers findings from experience. That’s why canned our partner status. In this case they haven’t touched NTFS at all - that’s out right rubbish. I pointed out the paradigm mismatch already. Run unix loads on unix not NT.