2 ms·

The issue isn't NTFS as far as I understand (based on what the WSL team themselves have explained). The problem is that the NT kernel is simply slow at opening

The issue isn't NTFS as far as I understand (based on what the WSL team themselves have explained). The problem is that the NT kernel is simply slow at opening files. Windows has transactional NTFS but it's deprecated and hardly used. The slowness can't be fixed because the open codepath goes via a lot of different filter drivers that can hook the open sequence, combined with the fact that there's no equivalent of a UNIX dentry cache because file path parsing is done by the FS driver and not the upper layers of the kernel. Even if the default filters were fixed to be made as fast as possible - which is hard because they're things like virus scanners - third party filter drivers are common and would tank performance again. It's a pity because Windows would benefit from faster file IO in general but it seems like they gave up on improving things as being too hard.

WorldMaker 4y ago
What I mean here by "transaction" semantics is not "transactional NTFS" (or the other distributed transaction engines that replaced it) but as a short hand for all the various different ways that file locking mechanics and file consistency guarantees are very different in NT/Windows/NTFS than in the POSIX "inode" model. All of that has a lot of complex moving parts (filter drivers are indeed one part of that complex dance both affecting and affected by Windows' file "transaction" semantics). "Transaction" is a useful analogical word here for all of that complexity because how a mini-version of the CAP theorem trade-off space can be seen to apply to file systems. Windows heavily, heavily favors Consistency above all else in file operations. Consistency checks of course slow down file opening (and other operations too). POSIX heavily favors Availability above all else and will among other things partition logical files across multiple physical inodes to do that. Neither approach to "file transactions" is "better" or "perfect", they are different trade-offs. They both have their strengths and weaknesses. Using tools designed for one is always going to have some problems operating in the other. POSIX tools are always going look at Windows as "slow file IO" because it doesn't hustle for availability. Windows tools are always going to look at POSIX as willfully dangerous when it comes to file consistency. At the end of the day these filesystem stacks are always going to be different tools for different jobs.

WorldMaker 4y ago

What I mean here by "transaction" semantics is not "transactional NTFS" (or the other distributed transaction engines that replaced it) but as a short hand for all the various different ways that file locking mechanics and file consistency guarantees are very different in NT/Windows/NTFS than in the POSIX "inode" model. All of that has a lot of complex moving parts (filter drivers are indeed one part of that complex dance both affecting and affected by Windows' file "transaction" semantics). "Transaction" is a useful analogical word here for all of that complexity because how a mini-version of the CAP theorem trade-off space can be seen to apply to file systems. Windows heavily, heavily favors Consistency above all else in file operations. Consistency checks of course slow down file opening (and other operations too). POSIX heavily favors Availability above all else and will among other things partition logical files across multiple physical inodes to do that. Neither approach to "file transactions" is "better" or "perfect", they are different trade-offs. They both have their strengths and weaknesses. Using tools designed for one is always going to have some problems operating in the other. POSIX tools are always going look at Windows as "slow file IO" because it doesn't hustle for availability. Windows tools are always going to look at POSIX as willfully dangerous when it comes to file consistency. At the end of the day these filesystem stacks are always going to be different tools for different jobs.