9 ms·

> I thought it was a thin translation layer between the kernels? WSL1 was like that, but it was severely handicapped by NT filesystem performance, especially f

> I thought it was a thin translation layer between the kernels? WSL1 was like that, but it was severely handicapped by NT filesystem performance, especially for small files or metadata operations. There are a lot of small operations that are more than 2 orders of magnitude slower on Windows than on Linux, and a lot of Linux software just expects them to be fast, so WSL1 is not a great experience for a lot of stuff.

mehrdadn 6y ago
Note that the performance didn't really have anything to do with NTFS. This is trivial to prove if you just try a different filesystem.
- close04 6y ago
  WSL 1 and WSL 2 can coexist and be used according to what needs one's scenarios present. I imagine MS collected enough requirements to also justify a full Linux VM. There will be some slowness somewhere, the 2 versions now let you choose on which side of the boundary to place it. https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq
  - mehrdadn 6y ago
    They could've instead worked on speeding up the I/O subsystem and sped up Windows altogether. I refuse to believe it's impossible. The more plausible reasons to me seem to be (a) they have to work independently from the I/O subsystem team(s) for whatever reason (maybe they didn't care enough to cooperate), or (b) it's too tricky to get right and prone to bugs, so they preferred not to have to deal with it and risk breaking I/O.
    - close04 6y ago
      I/O subsystem performance is just one aspect. WSL 1 is missing a lot of other stuff probably because the effort to implement it all was to great. MS would have to duplicate a lot of the effort already put in by Linux developers. So they went the route of a full Linux VM. And no matter how much MS improves WSL 1 there would always be something missing that enough users want/need. For example I care more about the network related stuff that wasn't implemented (NETLINK_ROUTE\RTM_GETROUTE, AF_PACKET family) because this excluded nmap.
  - shawnz 6y ago
    WSL 2 can't coexist with other virtual machine software though (unless it leverages Hyper-V)
    - sebazzz 6y ago
      Good chance that your other VM software can't co-exist anyway. Many other technologies, Windows Defender among others, leverage virtualization through Hyper-V.
      - shawnz 6y ago
        Hyper-V is not installed by default on Windows and I have not been required to install it for anything except WSL 2 (and Hyper-V itself). I think Docker on Windows also uses it but I don't use that. Windows Defender has never indicated a requirement for Hyper-V to me.
        mehrdadn 6y ago
        I think by Windows Defender they might've been thinking of the Sandbox?
        sebazzz 6y ago
        It is not anywhere in the Windows features dialog mentioned explicitly but many features install an hypervisor under the covers. Not Hyper-V itself but a hypervisor which inhibits other VM hypervisors from functioning.
        shawnz 6y ago
        If you are talking about "Windows Hypervisor Platform", that indeed has Hyper-V as a dependency.
        rb12345 6y ago
        CredentialGuard uses Hyper-V, so it could be that.
    - blackoil 6y ago
- cesarb 6y ago
  Yes, this long issue comment explains how the true cause is the design of the Windows IO subsystem: https://github.com/microsoft/WSL/issues/873#issuecomment-425272829 https://github.com/microsoft/WSL/issues/873#issuecomment-425...
  - mehrdadn 6y ago
    Yeah, and I'm not sure I really buy everything they say there either. For example, "unfortunately that means sending more operations down to the file system" seems a little tough to swallow. There's no reason it has to be that way for file systems they control, which I dare say is the case for almost all their users. They could just implement separate APIs that optimize the cases where all sides of the equation are well-known, and only fall back (as they mention) when there's an unknown somewhere. (Not that I'm suggesting these are easy. Just commenting on the technical premises they're providing.) Also they said "That doesn't mean we've given up either", but you really have to do some mental gymnastics to convince yourself that moving to a VM wasn't "giving up" on solving these problems in WSL1.
- Lammy 6y ago
  To be fair, I read that as "NT (kernel) filesystem performance" and didn't assume it meant NT File System as in NTFS.
  - mehrdadn 6y ago
    I also specifically worded it as "note that" to preempt this comment about the slight ambiguity, but I guess that didn't really help... But in any case, I think the component you want to refer to in that case it the "I/O subsystem".
shawnz 6y ago
Personally I never had a problem with filesystem performance on WSL 1. I certainly don't think it was so bad that the whole thing needed to be scrapped in favor of a virtual machine.
- mehrdadn 6y ago
  Try doing git filter-branch. ;)
- jrockway 6y ago
  Windows does something weird with the filesystem that I have never figured out. In a msys2 shell, git is incredibly incredibly slow. I actually don't even bother running git in msys2 anymore; I just use the native git in that directory. (I use msys2 to cross-compile AVR stuff.) This seems to be a known issue, but nobody knows how to fix it. (It's some UAC thing, apparently.) I imagine this is what WSL 1 used, and it was similarly slow.
  - zlynx 6y ago
    As far as I know it's a fundamental kernel architecture decision that Microsoft can't just "fix." From the original NT microkernel design, filesystem access is all asynchronous messaging, and it can be intercepted and modified by drivers at many points. A lot of software takes advantage of that. But it's also why caching isn't as effective as it could be, since you can't rely on all those unknown layers returning the same result each time. Plus a lot of Windows systems are running an AV which likes to check to see if you should be allowed to read any particular file in case it might be dangerous.
    - WorldMaker 6y ago
      Also, NTFS has very different (multi-file) ACID guarantees and transaction semantics compared to most POSIX file systems. A lot of performance differences between Windows and POSIX file APIs make some sense in a NoSQL versus SQL database analogy: they make very different guarantees about what the "atomic commit" level is, what is stored and how it is stored, what is locked and how and when it is locked. They both have great uses and there are trade-offs to both approaches, you can't always rely on the performance of an application built for one running performatively on top of the other.

mehrdadn 6y ago

Note that the performance didn't really have anything to do with NTFS. This is trivial to prove if you just try a different filesystem.

close04 6y ago
WSL 1 and WSL 2 can coexist and be used according to what needs one's scenarios present. I imagine MS collected enough requirements to also justify a full Linux VM. There will be some slowness somewhere, the 2 versions now let you choose on which side of the boundary to place it. https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq
- mehrdadn 6y ago
  They could've instead worked on speeding up the I/O subsystem and sped up Windows altogether. I refuse to believe it's impossible. The more plausible reasons to me seem to be (a) they have to work independently from the I/O subsystem team(s) for whatever reason (maybe they didn't care enough to cooperate), or (b) it's too tricky to get right and prone to bugs, so they preferred not to have to deal with it and risk breaking I/O.
  - close04 6y ago
    I/O subsystem performance is just one aspect. WSL 1 is missing a lot of other stuff probably because the effort to implement it all was to great. MS would have to duplicate a lot of the effort already put in by Linux developers. So they went the route of a full Linux VM. And no matter how much MS improves WSL 1 there would always be something missing that enough users want/need. For example I care more about the network related stuff that wasn't implemented (NETLINK_ROUTE\RTM_GETROUTE, AF_PACKET family) because this excluded nmap.
- shawnz 6y ago
  WSL 2 can't coexist with other virtual machine software though (unless it leverages Hyper-V)
  - sebazzz 6y ago
    Good chance that your other VM software can't co-exist anyway. Many other technologies, Windows Defender among others, leverage virtualization through Hyper-V.
    - shawnz 6y ago
      Hyper-V is not installed by default on Windows and I have not been required to install it for anything except WSL 2 (and Hyper-V itself). I think Docker on Windows also uses it but I don't use that. Windows Defender has never indicated a requirement for Hyper-V to me.
      - mehrdadn 6y ago
        I think by Windows Defender they might've been thinking of the Sandbox?
      - sebazzz 6y ago
        It is not anywhere in the Windows features dialog mentioned explicitly but many features install an hypervisor under the covers. Not Hyper-V itself but a hypervisor which inhibits other VM hypervisors from functioning.
        shawnz 6y ago
        If you are talking about "Windows Hypervisor Platform", that indeed has Hyper-V as a dependency.
      - rb12345 6y ago
        CredentialGuard uses Hyper-V, so it could be that.
  - blackoil 6y ago
cesarb 6y ago
Yes, this long issue comment explains how the true cause is the design of the Windows IO subsystem: https://github.com/microsoft/WSL/issues/873#issuecomment-425272829 https://github.com/microsoft/WSL/issues/873#issuecomment-425...
- mehrdadn 6y ago
  Yeah, and I'm not sure I really buy everything they say there either. For example, "unfortunately that means sending more operations down to the file system" seems a little tough to swallow. There's no reason it has to be that way for file systems they control, which I dare say is the case for almost all their users. They could just implement separate APIs that optimize the cases where all sides of the equation are well-known, and only fall back (as they mention) when there's an unknown somewhere. (Not that I'm suggesting these are easy. Just commenting on the technical premises they're providing.) Also they said "That doesn't mean we've given up either", but you really have to do some mental gymnastics to convince yourself that moving to a VM wasn't "giving up" on solving these problems in WSL1.
Lammy 6y ago
To be fair, I read that as "NT (kernel) filesystem performance" and didn't assume it meant NT File System as in NTFS.
- mehrdadn 6y ago
  I also specifically worded it as "note that" to preempt this comment about the slight ambiguity, but I guess that didn't really help... But in any case, I think the component you want to refer to in that case it the "I/O subsystem".

shawnz 6y ago

Personally I never had a problem with filesystem performance on WSL 1. I certainly don't think it was so bad that the whole thing needed to be scrapped in favor of a virtual machine.

mehrdadn 6y ago
Try doing git filter-branch. ;)
jrockway 6y ago
Windows does something weird with the filesystem that I have never figured out. In a msys2 shell, git is incredibly incredibly slow. I actually don't even bother running git in msys2 anymore; I just use the native git in that directory. (I use msys2 to cross-compile AVR stuff.) This seems to be a known issue, but nobody knows how to fix it. (It's some UAC thing, apparently.) I imagine this is what WSL 1 used, and it was similarly slow.
- zlynx 6y ago
  As far as I know it's a fundamental kernel architecture decision that Microsoft can't just "fix." From the original NT microkernel design, filesystem access is all asynchronous messaging, and it can be intercepted and modified by drivers at many points. A lot of software takes advantage of that. But it's also why caching isn't as effective as it could be, since you can't rely on all those unknown layers returning the same result each time. Plus a lot of Windows systems are running an AV which likes to check to see if you should be allowed to read any particular file in case it might be dangerous.
  - WorldMaker 6y ago
    Also, NTFS has very different (multi-file) ACID guarantees and transaction semantics compared to most POSIX file systems. A lot of performance differences between Windows and POSIX file APIs make some sense in a NoSQL versus SQL database analogy: they make very different guarantees about what the "atomic commit" level is, what is stored and how it is stored, what is locked and how and when it is locked. They both have great uses and there are trade-offs to both approaches, you can't always rely on the performance of an application built for one running performatively on top of the other.