4 ms·
> Marking the temporary files as FILE_ATTRIBUTE_TEMPORARY could improve things, without having to go into significant Windows kernel changes. Having used Cygwi
by snagglegaggle 7y ago
> Marking the temporary files as FILE_ATTRIBUTE_TEMPORARY could improve things, without having to go into significant Windows kernel changes.
Having used Cygwin and MinGW (and less so WSL) NTFS is probably the main factor here. Especially when using Cygwin program compilation is very slow not just due to process creation but file access, potentially on disk.
I could see checkpointing contributing, but having used backups/file history on Windows server, you will see CPU use irregularities that I think are similar in cause to this but you do not see CPU use irregularities as bad as described in the article.
The pathological behavior of NTFS with many files is easy to prove once you encounter it.[0] This would be exposed to the kernel as well and is likely holding up NtfsCheckpointVolume. At least in my experience the problem goes deeper, and how NTFS structures are handled also contributes; for example, trying to enumerate and copy files in certain ways is extremely slow even if the files are easily enumerable.
You can say that if disabling checkpointing gets rid of noticeable slowdown it is the right thing to do, but there are people who will rightly not want to disable it.
[0] https://stackoverflow.com/questions/197162/ntfs-performance-and-large-volumes-of-files-and-directories https://stackoverflow.com/questions/197162/ntfs-performance-...
- pault 7y agoTime the difference between an npm install, with all its thousands of tiny files, on NTFS and ext4. It's excruciating.
- cpach 7y agoJust copying Emacs for Windows from one folder to another takes surprisingly long time :)
- Filligree 7y agoI've definitely noticed this behaviour. It's excruciating. There's a ZFS driver for Windows now. I haven't tried it yet, but I think I will. Can't be any worse.