15 ms·

Windows file access performance compared to Linux

518 points by eatonphil 8y ago

kuroguro 8y ago
Wouldn't holding project files in an ext partition fix this? Or is that not supported yet?
- ronsor 8y ago
  AFAIK Windows 10 doesn't support extN (N = {2,3,4}) partitions
- anticensor 8y ago
  Use ReactOS driver if that works. If not, file a bug report on https://jira.reactos.org https://jira.reactos.org.
- Dylan16807 8y ago
  The way to really fix it would be a filesystem on a loopback device that stays inside linux territory, but that's not supported yet.
- int_19h 8y ago
  Keep in mind that Linux filesystems are all implemented in the kernel (FUSE aside), and WSL doesn't run a Linux kernel - it just emulates syscalls for userland to work. So there's no ext drivers in there, or any other standard Linux FS drivers.
qwerty456127 8y ago
> A Win32 path like C:\dir\file gets translated to an NT path like \??\C:\dir\file, where \??\C: is a symlink in Object Manager to a device object like \Device\HarddiskVolume4 Cool! Can the "NT paths" be used directly on Windows? As far as I know the ordinary Win32/NTFS system doesn't even have symlinks and this feels quite a handicap.
- xpaulbettsx 8y ago
  Yes they can, though one problems is that because NT paths are less restrictive than Win32 paths, you can create files / directories that you then have trouble doing things with using standard tools
  - blattimwind 8y ago
    Notably the Windows shell does not support longlong file names, like, at all. The Windows shell and Win32 APIs are also easily confused by files using the NTFS POSIX namespace, which may have \ in their name.
- jstimpfle 8y ago
  I'm not a super expert but I think that is the case and you even need them if you want to use some advanced features like paths that are longer than MAX_PATH or these virtual mount kinds of things. But IMO it's ugly as hell and just begging for compatibility problems. So don't go there, at least not to get super long paths. 260 bytes should be enough for anybody.
  - qwerty456127 8y ago
    > 260 bytes should be enough for anybody It happens to be not enough just so damn often.
    - jstimpfle 8y ago
      How did you manage? I've never hit the limit in like 20 years. Maybe 260 bytes means only 130 2-byte characters (I don't even know) and that I might find a little restrictive maybe. In any case I'm tempted to say, just fix your damn paths (if possible). I guess at 80 characters it starts to get unreadable anyway.
      - qwerty456127 8y ago
        Of course I have only encountered filenames this long a couple times but as for full path like C:\something\something this is very easy. E.g. "How did you manage? I've never hit the limit in like 20 years." is 63 characters and it still looks like a reasonable title for a book. You may have written or downloaded a book and stored it in a folder named this way in multiple formats (i.e. PDF, EPUB etc) and this can already make something like "C:\Documents and Settings\John Doe\My Documents\My Projects\Books\How did you manage - I've never hit the limit in like 20 years\How did you manage - I've never hit the limit in like 20 years.epub" and this already is 199 characters. Just 60 characters left and 60 characters is as short as "C:\Documents and Settings\John Doe\My Documents", and you might in fact have your book file named a longer way like "John Doe - How did you manage? I've never hit the limit in like 20 years. 2nd edition draft". This is a bit clumsy example but it demonstrates how easily reachable the limit actually is in the real life. This actually forced me to store my data in c:\a\b\c... (where a, b, and c actually were 1-4-letter acronyms) instead of C:\Documents and Settings\John Doe\My Documents\
marsrover 8y ago
I develop on Windows 10 because I’m a .NET developer and really like the idea of WSL. But the more I use it the more frustrated I become by its file access performance. I started off using it for Git but now I just use Git bash in Poweshell (which also annoys me with its slowness). I haven’t developed on an actual *NIX machine in years but recently I deployed something to my DO VPS and it “built” (php composer) in what felt like 1/100 of the time it was taking on my computer, whether running the Windows binaries in Powershell/CMD or the Linux binaries in WSL. Although I will say WSL is slower. In fact, it was so fast that I’m about to call Microcenter and see about picking up a Thinkpad to install Linux on.
- tyingq 8y ago
  This helped me: https://github.com/Microsoft/WSL/issues/1932#issuecomment-407855346 https://github.com/Microsoft/WSL/issues/1932#issuecomment-40... Microsoft made some change a year or so ago that brought more of WSL into Windows Defender's view.
  - Strom 8y ago
    Windows Defender is an abomination. It makes file operations on regular Windows also extremely slow. For example unpacking the Go 1.11 zip which is 8700 files takes a second with my PCIe SSD with Windows Defender disabled. Enable it and the extraction time rises to several minutes.
    - tyingq 8y ago
      I wonder if their "real time protection" is making a network request every time you open a new file.
- Joeri 8y ago
  A major performance improvement is adding the working folders for my coding projects to the exclusion list of whatever antivirus solution is running. On low-specced machines I disable AV entirely, because I feel these days it is mostly snake oil anyway with zero days being commonplace.
quelsolaar 8y ago
I think this is a very good example of how windows is different in its goals and designs from Linux. I have a feeling this is the reason Linux has had a hard time catching on, on the desktop. Its easy to complain about a slow filesystem, but Microsoft lives in a different world, where other priorities exist. For someone building a server running a database serving loads of people Linux is a no brainer. You can pick the parts you want, shrink wrap it and fire it up without any fat. On a desktop machine, you want to be able to update drivers in the background without rebooting, you want virus scanners, and you want to have driver ready the moment the user plug's in a new device. Both Windows and Linux is for the most part very well engineered, but with very different priorities.
- platz 8y ago
  As a c# Dev, I am constantly annoyed that Windows updates and sometimes installs require reboots or stopping all user activity, while I've never had to reboot or block during an upgrade on ubuntu
  - gh02t 8y ago
    To be fair, a lot of Linux updates require a reboot or at least a logout to properly take effect, too. Windows is just very aggressive about forcing you to upgrade and reboot, which does undeniably have security benefits when you consider that Windows has a lot of non-technical users and a huge attack surface. At least they have relaxed it a bit, the frequent forced reboots caused me some serious problems on a Windows machine I had controlling a CNC machine.
    - snazz 8y ago
      This is true. Fedora now only has a Reboot and Update button in the GNOME Software GUI because some software like Firefox and some GNOME components crash if you update them while they're running (although this seems to happen more often with Wayland than Xorg for some reason). At least Linux and the BSDs give you a choice whether to do offline or online updates.
nickjj 8y ago
I remember this issue (I commented in it a few years ago). On the bright side, WSL as a development environment is no longer slow when it comes to real world usage. I've been using it for full time web app development for the last year and even made a video about my entire set up and configuration a week ago[0]. For example with WSL you can easily get <= 100ms code reloads even through a Docker for Windows volume on really big applications with thousands of files, such as a Rails app. Even compiling ~200kb of SCSS through a few Webpack loaders (inside of Docker with a volume) takes ~600ms with Webpack's watcher. It's glorious. I haven't had a single performance issue with a bunch of different Rails, Flask, Node, Phoenix and Jekyll apps (with and without Webpack). This is with all source code being mounted in from a spinning disk drive too, so it can get a lot faster. So while maybe in a micro-benchmark, the WSL file system might be an order of magnitude slower (or worse) than native Linux, it doesn't really seem to matter much when you use it in practice. [0]: https://www.youtube.com/watch?v=5gu8wWX3Ob4 https://www.youtube.com/watch?v=5gu8wWX3Ob4
- cma 8y ago
  Git runs really slow on it. It runs much faster when turning off Windows defender (but still slow).
  - nickjj 8y ago
    Are you talking about running it from the command line? I use Git almost every day and have it installed directly inside of WSL. It's not slow for any projects I've worked on, but it's worth mentioning I'm not dealing with code bases with millions of lines of code and many tens of thousands of files. Most of the code bases I work on have ~100kb LOC or less (most are much less), and hundreds or low thousands of files. Grepping through 50,000+ files through a WSL mount on a non-SSD is pretty slow, but I haven't tried doing that on a native Linux system in a long time so I can't really say if that slowness is mostly WSL or grepping through a lot of files in general.
    - cma 8y ago
      Its a project with several million lines of code. If grep is hitting 100% utilization and isn't just file system bound, and you are dealing with ascii stuff, you can speed it up a lot of times by prepending 'LANG=C', so that it doesn't have to deal with unicode.
CodeArtisan 8y ago
NTOS (Windows NT's kernel) was originally designed to be backward compatible with Windows (DOS era), OS/2 and POSIX. Designing such a kernel was considered a performance at the time but probably cost its fair share of bloat that kept growing with the years. Also, it's not surprising that Linux is optimized for operating on files since the UNIX philosophy is about considering everything as a file, something that Dave Cutler (NTOS main designer) was known for criticizing[1]. I pray daily that more of my fellow-programmers may find the means of freeing themselves from the curse of compatibility. (Dijkstra) [1] https://en.wikipedia.org/wiki/Dave_Cutler#Attitude_towards_Unix https://en.wikipedia.org/wiki/Dave_Cutler#Attitude_towards_U...
mehrdadn 8y ago
Not sure if I'm being naive, but couldn't they make it so that if there are no filters present on a given volume, then the entire filter stack is bypassed and the directory nodes are served from a cache managed by the file system at the root of the volume? That way developers who really want performance can make sure they have no filter drivers installed on that volume.
- tracker1 8y ago
  If they really wanted to, they could mount an ext4 fs as a single blob on disk directly into WSL. That won't really help if you want to run a GUI editor in windows against that identical directory though.
  - mehrdadn 8y ago
    Any idea why they haven't responded to people's requests for FUSE? It's the second-most requested feature and should enable applications like this.
    - zmj 8y ago
      They have: https://docs.microsoft.com/en-us/windows/desktop/projfs/projected-file-system https://docs.microsoft.com/en-us/windows/desktop/projfs/proj...
      - mehrdadn 8y ago
        That looks awesome (thanks for sharing) but it seems to be a different thing? People are looking for a way to run existing Linux FUSE in SL, not develop new virtual file system implementations. They haven't responded to that request at all. https://wpdev.uservoice.com/forums/266908-command-prompt-console-windows-subsystem-for-l/suggestions/13522845-add-fuse-filesystem-in-userspace-support-in-wsl https://wpdev.uservoice.com/forums/266908-command-prompt-con...
        int_19h 8y ago
Solar19 8y ago
This is interesting, and to some extent, expected. I'd expect that emulating one OS on top of another is going to have performance challenges and constraints in general. But the deep dive into aspects of Windows and Linux I/O reminded me that I'd love to see a new generation of clean sheet operating systems. Not another Unix-like OS. Not Windows. Something actually new and awesome. It's been a long time since Unix/BSD, Linux, and Windows NT/XP were introduced. A few years ago, Microsoft developed a new operating system called Midori, but sadly they killed it. MS, Apple, and Google are each sitting on Carl Sagan money – billions and billions in cash. Would it hurt MS to spend a billion to build an awesome, clean-sheet OS? They could still support and update Windows 10 for as many years as they deemed optimal for their business, while also offering the new OS. Would it cost more than a billion dollars to assemble an elite team of a few hundred people to build a new OS? Would Apple, Microsoft, or Google notice the billion dollar cost? Or even two billion? If you think there's no point in a new OS, oh I think there's a lot of room for improvement right now. For one thing, we really ought to have Instant Computing in 2018. Just about everything we do on a computer should be instant, like 200 ms, maybe 100 ms. Opening an application and having it be fully ready for further action should be instant. Everything we do in an application should be instant, except for things like video transcoding or compressing a 1 GiB folder. OSes today are way, way too slow, which comes back to the file access performance issue on Windows. Even a Mac using a PCIe SSD fails to be instant for all sorts of tasks. They're all too damn slow. We also need a fundamentally new security model. There's no way data should be leaving a user's computer as freely and opaquely as is the case now with all consumer OSes. Users should have much more control and insight into data leaving their machine. A good OS should also differentiate actual human users from software – the user model on nix is inadequate. Computers should be able to know when an action or authorization was physically committed by a human. That's just basic, and we don't have it. And I'm just scratching the surface of how much better security could be on general purpose OSes. There's so much more we could do.
- pas 8y ago
  Google is doing something with that money: https://en.wikipedia.org/wiki/Google_Fuchsia https://en.wikipedia.org/wiki/Google_Fuchsia
setquk 8y ago
I can’t see MFT contention mentioned once. That’s what absolutely and totally destroys small file write performance. This affects source operations, WSL, compilers, file storage, everything. And that’s so architecturally tied to the guts of NT you can’t fix it without pushing a new NTFS revision out. Which is risky and expensive. Which is incidentally why no one at Microsoft even seems to mention it I suspect and just chips at trivial issues around the edges. Bad show. NTFS is fundamentally broken. Go fix it. Edit: my experience comes from nearly two decades of trying to squeeze the last bit of juice out of windows unsuccessfully. My conclusion is don’t bother. Ext4, ZFS and APFS are at least an order of magnitude more productive and this is a measurable gain.
- Zardoz84 8y ago
  Perhaps we didn't read the same article. What it says that the root of problem is the Windows IO subsystem architecture. Change NTFS for anything and you will get the same problem.
  - setquk 8y ago
    But that’s not the case. The root cause is the MFT and NTFS architecture. People fail to mention that because the problem is harder to fix. It’s almost that there is a “do not speak bad of NTFS” going on. You can demonstrate this by using a third party file system driver for NT. when NTFS on its own is eliminated the performance is much much better. This is a neat little differential analysis which is conclusive. I can’t remember the product I used when I evaluated this about 8 years ago unfortunately.
mey 8y ago
I still scracth my head as to why doing a kernel update of Ubuntu running under Hyper-V on a spinning disk is so horrifically slow. If I migrate the VM while it's running to SSD, let it finish, then migrate it back to spinning, it's faster then just letting it update in place. This is my device desktop, otherwise I would put in on SSD full time. The above comments make me think something like the stat calls may be the issue and moving the running vm to SSD hides the problem. It obviously isn't raw disk rate at the point.
- rusk 8y ago
  Has to be some kind of caching issue ...
- AaronFriel 8y ago
  Is the Ubuntu VM's / on a fixed or dynamic VHD? The former provides (modulo filesystem fragmentation) linear addressing of virtual LBAs to physical LBAs, and the latter can be heavily fragmented and result in sequential read/write operations becoming random access, which on a HDD kills performance. My advice for anyone running VMs is if they're on HDDs or they're highly tuned for disk usage (databases for example), use fixed VHDs, otherwise use dynamic.
  - mey 8y ago
    Thanks for the suggestion. The disk is a dynamic VHDX, migrated it to fixed vhdx and will experiment.
- gerdesj 8y ago
  I still scracth my head as to why doing a kernel update of Ubuntu running under Hyper-V on a spinning disk is so horrifically slow. Define slow please. For a laugh I picked a random VM (VMWare) at work and ran (I did apt update first): # time apt upgrade ... 82 to upgrade, 5 to newly install, 0 to remove and 0 not to upgrade ... real 6m16.015s user 2m38.936s sys 0m55.216s The updates included two client server DB engines (Clickhouse and Postgresql) the fileserving thingie (Samba) and a few other bits. The reboot takes about 90 seconds before the webby interface appears for rspamd.
ezoe 8y ago
An interesting comment from insider. We all knows from various benchmarks that Microsoft Windows' filesystem access performance was way too worse than Linux kernel not only WSL but also Win32 Subsystem too. Also, process creating performance is worse on Windows. I wonder if it is also the case of "Death by thousand cuts"
bytematic 8y ago
I would love a QT-esque rust library that utilizes components and maybe even scss or css for styling
Daniel_sk 8y ago
The build speed difference for Android (Gradle build system) between Windows and Linux is extremely noticeable especially for large projects, where you can see even almost 50% faster builds.
reubenmorais 8y ago
dyt
stevefan1999 8y ago
Perhaps this explains why Yarn/NPM was slow on WSL too?
ulkesh 8y ago
Windows performance is abysmal. Instead of beating around the bush, they should just state that it is the combination of everything that makes Windows dog slow compared to either Mac or Linux. Someone said it on that thread, but Windows really should be a DE on Linux now instead of being its own OS.
edoo 8y ago
The entire post negates the first statement that it isn't easy to just say "NTFS is slow".
Abishek_Muthian 8y ago
On similar lines, one of the usual gripe with windows is the horrible file copy estimates, Raymond Chen wrote an article on it to explain why it is[1]. [1]: https://blogs.msdn.microsoft.com/oldnewthing/20040106-00/?p=41193/ https://blogs.msdn.microsoft.com/oldnewthing/20040106-00/?p=...
- blntechie 8y ago
  The Old New Thing is a fantastic blog especially if you develop in Windows. I used to read it religiously until 5 years back and stopped reading about time when Google Reader was retired and somehow didn't setup in my new flow of reading blogs. Thanks for reminding me about this blog and article again.
yellowapple 8y ago
As another data point here: I had to add a TKinter-based loading screen to a PyQt5 app I wrote for work because the app - which takes half a second to start up on Linux - takes nearly a minute on Windows the first time around (and multiple dozens of seconds on subsequent runs) and I know firsthand that my end-users are unlikely to wait that long unless they can see something loading. I suspect it has to do with the sheer number of file accesses for each of the Python libraries and underlying DLLs and such.
tomohawk 8y ago
tldr: discussion is about why disk access under WSL is slower than under Linux, mostly due to the different design constraints of Linux vs Windows for file systems.
herf 8y ago
I spent many years optimizing "stat-like" APIs for Picasa - Windows just feels very different than Linux once you're benchmarking. It turns out Windows/SMB is very good at "give me all metadata over the wire for a directory" and not so fast at single file stat performance. On a high-latency network (e.g. Wi-Fi) the Windows approach is faster, but on a local disk (e.g., compiling code), Linux stat is faster.
- avar 8y ago
  I'm mostly clueless about Windows, so bear with me, but that makes no sense to me. If SMB has some "give me stat info for all stuff in a directory" API call then that's obviously faster over the network since it eliminates N roundtrips, but I'd still expect a Linux SMB host to beat a Windows SMB host at that since FS operations are faster, the Linux host would also understand that protocol. Unless what you mean is that Windows has some kernel-level "stat N" interface, so it beats Linux by avoiding the syscall overhead, or having a FS that's more optimized for that use-case. But then that would also be faster when using a SMB mount on localhost, and whether it's over a high-latency network wouldn't matter (actually that would amortize some of the benefits).
  - a1369209993 8y ago
    I think the idea is that you're accessing files sparsely and/or randomly. With the linux approach you avoid translating (from disk representation to syscall representation) metadata you don't need, and the in-memory disk cache saves having to re-read it (and some filesystems require a seek for each directory entry to read the inode data structure, which can also be avoided it you don't care about that particular stat). With the windows approach, the kernel knows you want multiple files from the same directory, so it can send a (slightly more expensive) bulk stat request, using only one round trip[0]. On linux, the kernel doesn't know whether you're grabbing a.txt,b.txt,... (single directory-wide stat) or foo/.get,bar/.git,... (multiple single stats that could be pipelined) or just a single file, so it makes sense to use the cheapest request initially. If it then sees another stat in the same directory, it might make a bulk request, but that still incurred a extra round trip, and may have added useless processing overhead if you only needed two files. TLDR: Access to distant memory is faster if assuptions can be made about your access patterns, access to local memory is faster if you access less of the local memory. 0: I'm aware of protocol-induced round trips, but I don't think it effects the reasoning.
dustinmoorenet 8y ago
The issue seems to be with how Windows handles file system operations. They allow filters to operate over each request. These filters are like hardware drivers in that they are created and maintained by third parties. So, MS has no real ability to change how things work because they don't have absolute control in the same way that Linux does over the operations (Linux device drivers are part of the kernel)
giancarlostoro 8y ago
> The NT file system API is designed around handles, not paths. Almost any operation requires opening the file first, which can be expensive. Even things that on the Win32 level seem to be a single call (e.g. DeleteFile) actually open and close the file under the hood. One of our biggest performance optimizations for DrvFs which we did several releases ago was the introduction of a new API that allows us to query file information without having to open it first. Ouch that sounds painful... Is this why deleting gigs worth of files takes a bit? I could of sworn it's not a huge difference on Linux, at least when using the GUI, maybe when doing straight rm it's quicker.
- mirimir 8y ago
  > I could of sworn it's not a huge difference on Linux, at least when using the GUI, maybe when doing straight rm it's quicker. Using Linux file management GUIs can be a disaster. Some years ago, I was massaging many GiB of text, using a crude hack, with scripts that did grep, sed, awk, etc. And I had many working directories with >10^4 tiny files each (maybe 10^5 even). In terminal, everything was fine. But opening a GUI would swap out so much that the box would freeze. I don't know why. I just learned to avoid doing it.
  - giancarlostoro 8y ago
    Heh, that's fine, I don't mind the terminal or the GUI, as long as you're rationally using either and aren't blaming either for your problems. I see a lot of "I only use the terminal" type of people who screw up GUIs and the reverse can be said, GUI users who screw up the terminal, it's all about RTFM'ing sometimes, or just reading what's right in front of you.
    - mirimir 8y ago
      it was this issue, I think: https://askubuntu.com/questions/892549/nautilus-so-slow-if-there-are-so-many-files-folder-in-an-opened-directory https://askubuntu.com/questions/892549/nautilus-so-slow-if-t...
- mehrdadn 8y ago
Thaxll 8y ago
At least windows doesn't freeze your whole desktop under heavy IO. Edit: I'm getting downvoted by people that never used Linux probably.
- zlynx 8y ago
  I haven't noticed Linux doing that in the last few years. But honestly, that could also be the NVMe drives.
- eMSF 8y ago
  It doesn't? Oh right, you're bullshitting me. Case in point: I recently tried unzipping the Boost library on an up-to-date Windows 10, and after trying to move the frozen progress window after a minute, the whole desktop promptly crashed. I have to say, the experience is better than it used to be, because at least the taskbar reappeared on its own. (Decompression succeeded on the second attempt after leaving the whole computer well alone ... but it certainly took its time even on a high-end desktop computer.)
  - dang 8y ago
    Could you please leave personal swipes out of your comments here? They have a degrading effect on discussion and evoke worse from others. Your comment would be just fine without the second sentence. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- Zardoz84 8y ago
  I work and play every day over Linux and I never saw the desktop freezing.
  - kiwijamo 8y ago
    I’ve attempted to use Linux and desktop freezing is the norm even on machines that run fine under windows. Admittedly the machines might be underpowered but that does call into question the commonly held belief that desktop Linux is better for low spec machines.
jclay 8y ago
Thanks for sharing, that was an interesting read. I know the IO performance has always been the main gripe with WSL. It makes me think more broadly about some of the trade offs with the discussions that happen on Github Issues. It’s great that insights like this get pulled out in the discussions, and this could serve as excellent documentation. However, the discoverability will be difficult and it always takes some weeding through the low quality comments to piece the insights from contributors together. I wonder how Github (and Microsoft) could better handle this. They could allow for flagging in-depth comments like this into a pinned section and they could be updated collaboratively in the future (Wiki?). It also feels like a reputation system could help to motivate healthier discussions and could bury low quality “me too” comments and gripes. It’s not so bad in the linked example but I often encounter many rude comments aimed against volunteer OSS developers.
- Analemma_ 8y ago
  This particular post is pretty famous if you've been following the development of WSL. It's constantly linked to and referenced from other GitHub issues regarding WSL performance. So I think GitHub's current system is succeeding in that regard, although there are so many good points raised here that I wish it could get turned into a standalone blog post.
jclay 8y ago
They mention that file performance decreases with the numbers of filters that are attached to the NTFS drive. Is there a way to list the filters to determine which ones (and how many) are applied on your system?
- AbacusAvenger 8y ago
  I had the same question. Apparently there are a couple of commands to look at: fltmc filters That shows the filters loaded, and presumably the ones with "num instances" of 0 aren't actually in use on any volumes. You can figure out what each of the filters is by looking at the output of: sc query type=driver To figure out what the ones I saw in fltmc were, I used this: sc query type=driver | grep -A1 -i -e cldflt -e wcifs -e storqos -e filecrypt -e luafv -e npsvctrig -e wof -e fileinfo
tahoemph999 8y ago
Interesting this article never said "poorly architected". The conclusion that the issue is peanut buttered points at that. Instead of looking into the system for things to optimize is there any proposal or initiative to rework it at a higher level?
- wbl 8y ago
  Windows is different. Linux puts effort into making the Unix style of IO work fast.
- tedunangst 8y ago
  Proposal from an earlier comment: > Noting, as an aside, that it isn't really all that necessary for MSFT to work on the project, because I gather there are at least 684 FOSS NT kernel developers available, and qualified, and willing, to volunteer their time to work on projects like that. I assume that's why all those people upvoted, anyway. With a team that size, stabilizing WinBtrfs will take no time at all.

518 points by eatonphil 8y ago

kuroguro 8y ago

Wouldn't holding project files in an ext partition fix this? Or is that not supported yet?

ronsor 8y ago
AFAIK Windows 10 doesn't support extN (N = {2,3,4}) partitions
anticensor 8y ago
Use ReactOS driver if that works. If not, file a bug report on https://jira.reactos.org https://jira.reactos.org.
Dylan16807 8y ago
The way to really fix it would be a filesystem on a loopback device that stays inside linux territory, but that's not supported yet.
int_19h 8y ago
Keep in mind that Linux filesystems are all implemented in the kernel (FUSE aside), and WSL doesn't run a Linux kernel - it just emulates syscalls for userland to work. So there's no ext drivers in there, or any other standard Linux FS drivers.

qwerty456127 8y ago

> A Win32 path like C:\dir\file gets translated to an NT path like \??\C:\dir\file, where \??\C: is a symlink in Object Manager to a device object like \Device\HarddiskVolume4 Cool! Can the "NT paths" be used directly on Windows? As far as I know the ordinary Win32/NTFS system doesn't even have symlinks and this feels quite a handicap.

xpaulbettsx 8y ago
Yes they can, though one problems is that because NT paths are less restrictive than Win32 paths, you can create files / directories that you then have trouble doing things with using standard tools
- blattimwind 8y ago
  Notably the Windows shell does not support longlong file names, like, at all. The Windows shell and Win32 APIs are also easily confused by files using the NTFS POSIX namespace, which may have \ in their name.
jstimpfle 8y ago
I'm not a super expert but I think that is the case and you even need them if you want to use some advanced features like paths that are longer than MAX_PATH or these virtual mount kinds of things. But IMO it's ugly as hell and just begging for compatibility problems. So don't go there, at least not to get super long paths. 260 bytes should be enough for anybody.
- qwerty456127 8y ago
  > 260 bytes should be enough for anybody It happens to be not enough just so damn often.
  - jstimpfle 8y ago
    How did you manage? I've never hit the limit in like 20 years. Maybe 260 bytes means only 130 2-byte characters (I don't even know) and that I might find a little restrictive maybe. In any case I'm tempted to say, just fix your damn paths (if possible). I guess at 80 characters it starts to get unreadable anyway.
    - qwerty456127 8y ago
      Of course I have only encountered filenames this long a couple times but as for full path like C:\something\something this is very easy. E.g. "How did you manage? I've never hit the limit in like 20 years." is 63 characters and it still looks like a reasonable title for a book. You may have written or downloaded a book and stored it in a folder named this way in multiple formats (i.e. PDF, EPUB etc) and this can already make something like "C:\Documents and Settings\John Doe\My Documents\My Projects\Books\How did you manage - I've never hit the limit in like 20 years\How did you manage - I've never hit the limit in like 20 years.epub" and this already is 199 characters. Just 60 characters left and 60 characters is as short as "C:\Documents and Settings\John Doe\My Documents", and you might in fact have your book file named a longer way like "John Doe - How did you manage? I've never hit the limit in like 20 years. 2nd edition draft". This is a bit clumsy example but it demonstrates how easily reachable the limit actually is in the real life. This actually forced me to store my data in c:\a\b\c... (where a, b, and c actually were 1-4-letter acronyms) instead of C:\Documents and Settings\John Doe\My Documents\

marsrover 8y ago

I develop on Windows 10 because I’m a .NET developer and really like the idea of WSL. But the more I use it the more frustrated I become by its file access performance. I started off using it for Git but now I just use Git bash in Poweshell (which also annoys me with its slowness). I haven’t developed on an actual *NIX machine in years but recently I deployed something to my DO VPS and it “built” (php composer) in what felt like 1/100 of the time it was taking on my computer, whether running the Windows binaries in Powershell/CMD or the Linux binaries in WSL. Although I will say WSL is slower. In fact, it was so fast that I’m about to call Microcenter and see about picking up a Thinkpad to install Linux on.

tyingq 8y ago
This helped me: https://github.com/Microsoft/WSL/issues/1932#issuecomment-407855346 https://github.com/Microsoft/WSL/issues/1932#issuecomment-40... Microsoft made some change a year or so ago that brought more of WSL into Windows Defender's view.
- Strom 8y ago
  Windows Defender is an abomination. It makes file operations on regular Windows also extremely slow. For example unpacking the Go 1.11 zip which is 8700 files takes a second with my PCIe SSD with Windows Defender disabled. Enable it and the extraction time rises to several minutes.
  - tyingq 8y ago
    I wonder if their "real time protection" is making a network request every time you open a new file.
Joeri 8y ago
A major performance improvement is adding the working folders for my coding projects to the exclusion list of whatever antivirus solution is running. On low-specced machines I disable AV entirely, because I feel these days it is mostly snake oil anyway with zero days being commonplace.

quelsolaar 8y ago

I think this is a very good example of how windows is different in its goals and designs from Linux. I have a feeling this is the reason Linux has had a hard time catching on, on the desktop. Its easy to complain about a slow filesystem, but Microsoft lives in a different world, where other priorities exist. For someone building a server running a database serving loads of people Linux is a no brainer. You can pick the parts you want, shrink wrap it and fire it up without any fat. On a desktop machine, you want to be able to update drivers in the background without rebooting, you want virus scanners, and you want to have driver ready the moment the user plug's in a new device. Both Windows and Linux is for the most part very well engineered, but with very different priorities.

platz 8y ago
As a c# Dev, I am constantly annoyed that Windows updates and sometimes installs require reboots or stopping all user activity, while I've never had to reboot or block during an upgrade on ubuntu
- gh02t 8y ago
  To be fair, a lot of Linux updates require a reboot or at least a logout to properly take effect, too. Windows is just very aggressive about forcing you to upgrade and reboot, which does undeniably have security benefits when you consider that Windows has a lot of non-technical users and a huge attack surface. At least they have relaxed it a bit, the frequent forced reboots caused me some serious problems on a Windows machine I had controlling a CNC machine.
  - snazz 8y ago
    This is true. Fedora now only has a Reboot and Update button in the GNOME Software GUI because some software like Firefox and some GNOME components crash if you update them while they're running (although this seems to happen more often with Wayland than Xorg for some reason). At least Linux and the BSDs give you a choice whether to do offline or online updates.

nickjj 8y ago

I remember this issue (I commented in it a few years ago). On the bright side, WSL as a development environment is no longer slow when it comes to real world usage. I've been using it for full time web app development for the last year and even made a video about my entire set up and configuration a week ago[0]. For example with WSL you can easily get <= 100ms code reloads even through a Docker for Windows volume on really big applications with thousands of files, such as a Rails app. Even compiling ~200kb of SCSS through a few Webpack loaders (inside of Docker with a volume) takes ~600ms with Webpack's watcher. It's glorious. I haven't had a single performance issue with a bunch of different Rails, Flask, Node, Phoenix and Jekyll apps (with and without Webpack). This is with all source code being mounted in from a spinning disk drive too, so it can get a lot faster. So while maybe in a micro-benchmark, the WSL file system might be an order of magnitude slower (or worse) than native Linux, it doesn't really seem to matter much when you use it in practice. [0]: https://www.youtube.com/watch?v=5gu8wWX3Ob4 https://www.youtube.com/watch?v=5gu8wWX3Ob4

cma 8y ago
Git runs really slow on it. It runs much faster when turning off Windows defender (but still slow).
- nickjj 8y ago
  Are you talking about running it from the command line? I use Git almost every day and have it installed directly inside of WSL. It's not slow for any projects I've worked on, but it's worth mentioning I'm not dealing with code bases with millions of lines of code and many tens of thousands of files. Most of the code bases I work on have ~100kb LOC or less (most are much less), and hundreds or low thousands of files. Grepping through 50,000+ files through a WSL mount on a non-SSD is pretty slow, but I haven't tried doing that on a native Linux system in a long time so I can't really say if that slowness is mostly WSL or grepping through a lot of files in general.
  - cma 8y ago
    Its a project with several million lines of code. If grep is hitting 100% utilization and isn't just file system bound, and you are dealing with ascii stuff, you can speed it up a lot of times by prepending 'LANG=C', so that it doesn't have to deal with unicode.

CodeArtisan 8y ago

NTOS (Windows NT's kernel) was originally designed to be backward compatible with Windows (DOS era), OS/2 and POSIX. Designing such a kernel was considered a performance at the time but probably cost its fair share of bloat that kept growing with the years. Also, it's not surprising that Linux is optimized for operating on files since the UNIX philosophy is about considering everything as a file, something that Dave Cutler (NTOS main designer) was known for criticizing[1]. I pray daily that more of my fellow-programmers may find the means of freeing themselves from the curse of compatibility. (Dijkstra) [1] https://en.wikipedia.org/wiki/Dave_Cutler#Attitude_towards_Unix https://en.wikipedia.org/wiki/Dave_Cutler#Attitude_towards_U...

mehrdadn 8y ago

Not sure if I'm being naive, but couldn't they make it so that if there are no filters present on a given volume, then the entire filter stack is bypassed and the directory nodes are served from a cache managed by the file system at the root of the volume? That way developers who really want performance can make sure they have no filter drivers installed on that volume.

tracker1 8y ago
If they really wanted to, they could mount an ext4 fs as a single blob on disk directly into WSL. That won't really help if you want to run a GUI editor in windows against that identical directory though.
- mehrdadn 8y ago
  Any idea why they haven't responded to people's requests for FUSE? It's the second-most requested feature and should enable applications like this.
  - zmj 8y ago
    They have: https://docs.microsoft.com/en-us/windows/desktop/projfs/projected-file-system https://docs.microsoft.com/en-us/windows/desktop/projfs/proj...
    - mehrdadn 8y ago
      That looks awesome (thanks for sharing) but it seems to be a different thing? People are looking for a way to run existing Linux FUSE in SL, not develop new virtual file system implementations. They haven't responded to that request at all. https://wpdev.uservoice.com/forums/266908-command-prompt-console-windows-subsystem-for-l/suggestions/13522845-add-fuse-filesystem-in-userspace-support-in-wsl https://wpdev.uservoice.com/forums/266908-command-prompt-con...
      - int_19h 8y ago

Solar19 8y ago

This is interesting, and to some extent, expected. I'd expect that emulating one OS on top of another is going to have performance challenges and constraints in general. But the deep dive into aspects of Windows and Linux I/O reminded me that I'd love to see a new generation of clean sheet operating systems. Not another Unix-like OS. Not Windows. Something actually new and awesome. It's been a long time since Unix/BSD, Linux, and Windows NT/XP were introduced. A few years ago, Microsoft developed a new operating system called Midori, but sadly they killed it. MS, Apple, and Google are each sitting on Carl Sagan money – billions and billions in cash. Would it hurt MS to spend a billion to build an awesome, clean-sheet OS? They could still support and update Windows 10 for as many years as they deemed optimal for their business, while also offering the new OS. Would it cost more than a billion dollars to assemble an elite team of a few hundred people to build a new OS? Would Apple, Microsoft, or Google notice the billion dollar cost? Or even two billion? If you think there's no point in a new OS, oh I think there's a lot of room for improvement right now. For one thing, we really ought to have Instant Computing in 2018. Just about everything we do on a computer should be instant, like 200 ms, maybe 100 ms. Opening an application and having it be fully ready for further action should be instant. Everything we do in an application should be instant, except for things like video transcoding or compressing a 1 GiB folder. OSes today are way, way too slow, which comes back to the file access performance issue on Windows. Even a Mac using a PCIe SSD fails to be instant for all sorts of tasks. They're all too damn slow. We also need a fundamentally new security model. There's no way data should be leaving a user's computer as freely and opaquely as is the case now with all consumer OSes. Users should have much more control and insight into data leaving their machine. A good OS should also differentiate actual human users from software – the user model on nix is inadequate. Computers should be able to know when an action or authorization was physically committed by a human. That's just basic, and we don't have it. And I'm just scratching the surface of how much better security could be on general purpose OSes. There's so much more we could do.

pas 8y ago
Google is doing something with that money: https://en.wikipedia.org/wiki/Google_Fuchsia https://en.wikipedia.org/wiki/Google_Fuchsia

setquk 8y ago

I can’t see MFT contention mentioned once. That’s what absolutely and totally destroys small file write performance. This affects source operations, WSL, compilers, file storage, everything. And that’s so architecturally tied to the guts of NT you can’t fix it without pushing a new NTFS revision out. Which is risky and expensive. Which is incidentally why no one at Microsoft even seems to mention it I suspect and just chips at trivial issues around the edges. Bad show. NTFS is fundamentally broken. Go fix it. Edit: my experience comes from nearly two decades of trying to squeeze the last bit of juice out of windows unsuccessfully. My conclusion is don’t bother. Ext4, ZFS and APFS are at least an order of magnitude more productive and this is a measurable gain.

Zardoz84 8y ago
Perhaps we didn't read the same article. What it says that the root of problem is the Windows IO subsystem architecture. Change NTFS for anything and you will get the same problem.
- setquk 8y ago
  But that’s not the case. The root cause is the MFT and NTFS architecture. People fail to mention that because the problem is harder to fix. It’s almost that there is a “do not speak bad of NTFS” going on. You can demonstrate this by using a third party file system driver for NT. when NTFS on its own is eliminated the performance is much much better. This is a neat little differential analysis which is conclusive. I can’t remember the product I used when I evaluated this about 8 years ago unfortunately.

mey 8y ago

I still scracth my head as to why doing a kernel update of Ubuntu running under Hyper-V on a spinning disk is so horrifically slow. If I migrate the VM while it's running to SSD, let it finish, then migrate it back to spinning, it's faster then just letting it update in place. This is my device desktop, otherwise I would put in on SSD full time. The above comments make me think something like the stat calls may be the issue and moving the running vm to SSD hides the problem. It obviously isn't raw disk rate at the point.

rusk 8y ago
Has to be some kind of caching issue ...
AaronFriel 8y ago
Is the Ubuntu VM's / on a fixed or dynamic VHD? The former provides (modulo filesystem fragmentation) linear addressing of virtual LBAs to physical LBAs, and the latter can be heavily fragmented and result in sequential read/write operations becoming random access, which on a HDD kills performance. My advice for anyone running VMs is if they're on HDDs or they're highly tuned for disk usage (databases for example), use fixed VHDs, otherwise use dynamic.
- mey 8y ago
  Thanks for the suggestion. The disk is a dynamic VHDX, migrated it to fixed vhdx and will experiment.
gerdesj 8y ago
I still scracth my head as to why doing a kernel update of Ubuntu running under Hyper-V on a spinning disk is so horrifically slow. Define slow please. For a laugh I picked a random VM (VMWare) at work and ran (I did apt update first): # time apt upgrade ... 82 to upgrade, 5 to newly install, 0 to remove and 0 not to upgrade ... real 6m16.015s user 2m38.936s sys 0m55.216s The updates included two client server DB engines (Clickhouse and Postgresql) the fileserving thingie (Samba) and a few other bits. The reboot takes about 90 seconds before the webby interface appears for rspamd.

ezoe 8y ago

An interesting comment from insider. We all knows from various benchmarks that Microsoft Windows' filesystem access performance was way too worse than Linux kernel not only WSL but also Win32 Subsystem too. Also, process creating performance is worse on Windows. I wonder if it is also the case of "Death by thousand cuts"

bytematic 8y ago

I would love a QT-esque rust library that utilizes components and maybe even scss or css for styling

Daniel_sk 8y ago

The build speed difference for Android (Gradle build system) between Windows and Linux is extremely noticeable especially for large projects, where you can see even almost 50% faster builds.

reubenmorais 8y ago

dyt

stevefan1999 8y ago

Perhaps this explains why Yarn/NPM was slow on WSL too?

ulkesh 8y ago

Windows performance is abysmal. Instead of beating around the bush, they should just state that it is the combination of everything that makes Windows dog slow compared to either Mac or Linux. Someone said it on that thread, but Windows really should be a DE on Linux now instead of being its own OS.

edoo 8y ago

The entire post negates the first statement that it isn't easy to just say "NTFS is slow".

Abishek_Muthian 8y ago

On similar lines, one of the usual gripe with windows is the horrible file copy estimates, Raymond Chen wrote an article on it to explain why it is[1]. [1]: https://blogs.msdn.microsoft.com/oldnewthing/20040106-00/?p=41193/ https://blogs.msdn.microsoft.com/oldnewthing/20040106-00/?p=...

blntechie 8y ago
The Old New Thing is a fantastic blog especially if you develop in Windows. I used to read it religiously until 5 years back and stopped reading about time when Google Reader was retired and somehow didn't setup in my new flow of reading blogs. Thanks for reminding me about this blog and article again.

yellowapple 8y ago

As another data point here: I had to add a TKinter-based loading screen to a PyQt5 app I wrote for work because the app - which takes half a second to start up on Linux - takes nearly a minute on Windows the first time around (and multiple dozens of seconds on subsequent runs) and I know firsthand that my end-users are unlikely to wait that long unless they can see something loading. I suspect it has to do with the sheer number of file accesses for each of the Python libraries and underlying DLLs and such.

tomohawk 8y ago

tldr: discussion is about why disk access under WSL is slower than under Linux, mostly due to the different design constraints of Linux vs Windows for file systems.

herf 8y ago

I spent many years optimizing "stat-like" APIs for Picasa - Windows just feels very different than Linux once you're benchmarking. It turns out Windows/SMB is very good at "give me all metadata over the wire for a directory" and not so fast at single file stat performance. On a high-latency network (e.g. Wi-Fi) the Windows approach is faster, but on a local disk (e.g., compiling code), Linux stat is faster.

avar 8y ago
I'm mostly clueless about Windows, so bear with me, but that makes no sense to me. If SMB has some "give me stat info for all stuff in a directory" API call then that's obviously faster over the network since it eliminates N roundtrips, but I'd still expect a Linux SMB host to beat a Windows SMB host at that since FS operations are faster, the Linux host would also understand that protocol. Unless what you mean is that Windows has some kernel-level "stat N" interface, so it beats Linux by avoiding the syscall overhead, or having a FS that's more optimized for that use-case. But then that would also be faster when using a SMB mount on localhost, and whether it's over a high-latency network wouldn't matter (actually that would amortize some of the benefits).
- a1369209993 8y ago
  I think the idea is that you're accessing files sparsely and/or randomly. With the linux approach you avoid translating (from disk representation to syscall representation) metadata you don't need, and the in-memory disk cache saves having to re-read it (and some filesystems require a seek for each directory entry to read the inode data structure, which can also be avoided it you don't care about that particular stat). With the windows approach, the kernel knows you want multiple files from the same directory, so it can send a (slightly more expensive) bulk stat request, using only one round trip[0]. On linux, the kernel doesn't know whether you're grabbing a.txt,b.txt,... (single directory-wide stat) or foo/.get,bar/.git,... (multiple single stats that could be pipelined) or just a single file, so it makes sense to use the cheapest request initially. If it then sees another stat in the same directory, it might make a bulk request, but that still incurred a extra round trip, and may have added useless processing overhead if you only needed two files. TLDR: Access to distant memory is faster if assuptions can be made about your access patterns, access to local memory is faster if you access less of the local memory. 0: I'm aware of protocol-induced round trips, but I don't think it effects the reasoning.

dustinmoorenet 8y ago

The issue seems to be with how Windows handles file system operations. They allow filters to operate over each request. These filters are like hardware drivers in that they are created and maintained by third parties. So, MS has no real ability to change how things work because they don't have absolute control in the same way that Linux does over the operations (Linux device drivers are part of the kernel)

giancarlostoro 8y ago

> The NT file system API is designed around handles, not paths. Almost any operation requires opening the file first, which can be expensive. Even things that on the Win32 level seem to be a single call (e.g. DeleteFile) actually open and close the file under the hood. One of our biggest performance optimizations for DrvFs which we did several releases ago was the introduction of a new API that allows us to query file information without having to open it first. Ouch that sounds painful... Is this why deleting gigs worth of files takes a bit? I could of sworn it's not a huge difference on Linux, at least when using the GUI, maybe when doing straight rm it's quicker.

mirimir 8y ago
> I could of sworn it's not a huge difference on Linux, at least when using the GUI, maybe when doing straight rm it's quicker. Using Linux file management GUIs can be a disaster. Some years ago, I was massaging many GiB of text, using a crude hack, with scripts that did grep, sed, awk, etc. And I had many working directories with >10^4 tiny files each (maybe 10^5 even). In terminal, everything was fine. But opening a GUI would swap out so much that the box would freeze. I don't know why. I just learned to avoid doing it.
- giancarlostoro 8y ago
  Heh, that's fine, I don't mind the terminal or the GUI, as long as you're rationally using either and aren't blaming either for your problems. I see a lot of "I only use the terminal" type of people who screw up GUIs and the reverse can be said, GUI users who screw up the terminal, it's all about RTFM'ing sometimes, or just reading what's right in front of you.
  - mirimir 8y ago
    it was this issue, I think: https://askubuntu.com/questions/892549/nautilus-so-slow-if-there-are-so-many-files-folder-in-an-opened-directory https://askubuntu.com/questions/892549/nautilus-so-slow-if-t...
mehrdadn 8y ago

Thaxll 8y ago

At least windows doesn't freeze your whole desktop under heavy IO. Edit: I'm getting downvoted by people that never used Linux probably.

zlynx 8y ago
I haven't noticed Linux doing that in the last few years. But honestly, that could also be the NVMe drives.
eMSF 8y ago
It doesn't? Oh right, you're bullshitting me. Case in point: I recently tried unzipping the Boost library on an up-to-date Windows 10, and after trying to move the frozen progress window after a minute, the whole desktop promptly crashed. I have to say, the experience is better than it used to be, because at least the taskbar reappeared on its own. (Decompression succeeded on the second attempt after leaving the whole computer well alone ... but it certainly took its time even on a high-end desktop computer.)
- dang 8y ago
  Could you please leave personal swipes out of your comments here? They have a degrading effect on discussion and evoke worse from others. Your comment would be just fine without the second sentence. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
Zardoz84 8y ago
I work and play every day over Linux and I never saw the desktop freezing.
- kiwijamo 8y ago
  I’ve attempted to use Linux and desktop freezing is the norm even on machines that run fine under windows. Admittedly the machines might be underpowered but that does call into question the commonly held belief that desktop Linux is better for low spec machines.

jclay 8y ago

Thanks for sharing, that was an interesting read. I know the IO performance has always been the main gripe with WSL. It makes me think more broadly about some of the trade offs with the discussions that happen on Github Issues. It’s great that insights like this get pulled out in the discussions, and this could serve as excellent documentation. However, the discoverability will be difficult and it always takes some weeding through the low quality comments to piece the insights from contributors together. I wonder how Github (and Microsoft) could better handle this. They could allow for flagging in-depth comments like this into a pinned section and they could be updated collaboratively in the future (Wiki?). It also feels like a reputation system could help to motivate healthier discussions and could bury low quality “me too” comments and gripes. It’s not so bad in the linked example but I often encounter many rude comments aimed against volunteer OSS developers.

Analemma_ 8y ago
This particular post is pretty famous if you've been following the development of WSL. It's constantly linked to and referenced from other GitHub issues regarding WSL performance. So I think GitHub's current system is succeeding in that regard, although there are so many good points raised here that I wish it could get turned into a standalone blog post.

jclay 8y ago

They mention that file performance decreases with the numbers of filters that are attached to the NTFS drive. Is there a way to list the filters to determine which ones (and how many) are applied on your system?

AbacusAvenger 8y ago
I had the same question. Apparently there are a couple of commands to look at: fltmc filters That shows the filters loaded, and presumably the ones with "num instances" of 0 aren't actually in use on any volumes. You can figure out what each of the filters is by looking at the output of: sc query type=driver To figure out what the ones I saw in fltmc were, I used this: sc query type=driver | grep -A1 -i -e cldflt -e wcifs -e storqos -e filecrypt -e luafv -e npsvctrig -e wof -e fileinfo

tahoemph999 8y ago

Interesting this article never said "poorly architected". The conclusion that the issue is peanut buttered points at that. Instead of looking into the system for things to optimize is there any proposal or initiative to rework it at a higher level?

wbl 8y ago
Windows is different. Linux puts effort into making the Unix style of IO work fast.
tedunangst 8y ago
Proposal from an earlier comment: > Noting, as an aside, that it isn't really all that necessary for MSFT to work on the project, because I gather there are at least 684 FOSS NT kernel developers available, and qualified, and willing, to volunteer their time to work on projects like that. I assume that's why all those people upvoted, anyway. With a team that size, stabilizing WinBtrfs will take no time at all.