4 ms·
The reason is due to file system filters, of which Windows Defender is always there. There is a significant delay from Defender when performing CloseFile()[0].
by nullindividual 2y ago
The reason is due to file system filters, of which Windows Defender is always there. There is a significant delay from Defender when performing CloseFile()[0].
> As I was looking at the raw system calls related to I/O, something immediately popped out: CloseFile() operations were frequently taking 1-5 milliseconds whereas other operations like opening, reading, and writing files only took 1-5 microseconds. That's a 1000x difference!
This is why DevDrive was introduced[1]. You can either have Defender operate in async mode (default) or remove it entirely from the volume at your own risk.
The performance issue isn't related to sync or async I/O.
[0] https://gregoryszorc.com/blog/2015/10/22/append-i/o-performance-on-windows/ https://gregoryszorc.com/blog/2015/10/22/append-i/o-performa...
[1] https://devblogs.microsoft.com/visualstudio/devdrive/ https://devblogs.microsoft.com/visualstudio/devdrive/
- cyberax 2y agoWindows FS stack is still _way_ slower than Linux. Filesystem operations have to create IRPs and submit them for execution through a generic mechanism. These IRPs can get filtered and modified in-flight, providing quite a bit of overall flexibility. In Linux, filesystem paths are super-optimized, with all the filtering (e.g. for SELinux) special-cased if needed. But even still, Windows also had to cheat to avoid completely cratering the performance, there's a shortcut called "FastIO": https://learn.microsoft.com/en-us/windows-hardware/drivers/ifs/registering-fast-i-o-dispatch-routines https://learn.microsoft.com/en-us/windows-hardware/drivers/i... I wrote a filesystem for Windows around 25 years ago, and I still remember how I implemented all the required prototypes and everything in Explorer worked. But notepad.exe was just showing me empty data. It took me several days to find a note tucked into MSDN that you need to implement FastIO for memory mapped files to work (which Notepad.exe used).
- SoothingSorbet 2y agoThat's interesting, why would notepad.exe use mmapped files?
- cyberax 2y agoProbably to save space when opening large files?
- nullindividual 2y agoBut it simply isn't slower than Linux. Robert Collins explains that performance is just as good as Linux and the performance loss on Windows is due to file system filters (Defender)[0]. This is what DevDrive intends (and does) fix. [0] https://youtu.be/qbKGw8MQ0i8?t=1759 https://youtu.be/qbKGw8MQ0i8?t=1759
- cyberax 2y agoThe last time I did FS tests was admittedly around 4 years ago, but back then Windows was several times slower on pure FS performance benchmarks (creating/listing/deleting large number of directories and files). It used to be _much_ slower, like orders of magnitude slower, especially for directories with a large number of files.
- nullindividual 2y agoYou're not seeing "pure" FS performance. You're seeing all of the abstractions between you and the file system. To get more of the abstractions out of the way, you want DevDrive. And don't use Explorer.exe as a test bed which has shims and hooks and god knows what else.
- cyberax 2y agoI was testing the performance using plain C++ code.
- lproven 2y agoThese days there is a full native Btrfs implementation for Windows. https://github.com/maharmstone/btrfs https://github.com/maharmstone/btrfs It's complete enough you can boot and run Windows from Btrfs. https://lilysthings.org/blog/windows-on-btrfs/ https://lilysthings.org/blog/windows-on-btrfs/ I wonder how many of these limitations affect this, notably performance. If WSL1 could natively talk to Btrfs without the performance-sapping translation to NTFS, would that resolve the poor Git performance etc?