10 ms·
Contra opinion: the VM option reduced the service interface between Windows and Linux to a single kernel implementation and a few drivers, rather than every pos
by ralph87 6y ago
Contra opinion: the VM option reduced the service interface between Windows and Linux to a single kernel implementation and a few drivers, rather than every possible userspace program ever written. It's an amazing and obvious trade off. My inner architecture astronaut appreciates all the ideas in this post, but I've been trying to kill that guy for over a decade now. The bottom line is WSLv1 design SUCKED precisely because it tried to cross-breed 2 extremely complex semantically incompatible systems.
Example: Linux local filesystem performance mostly derives from the dentry cache. That cache keeps a parsed representation of whatever the filesystem knows appears on disk. The dentry cache is crucial to pretty much any filesystem system call that does not already involve an open file, and IIRC many that also do. Problem is, that same cache in WSL must be subverted because Linux is not the only thing that can mutate the NTFS filesystem - any Windows program could as well. This one fundamentally unfixable problem alone is probably 80% the reason WSL1 IO perf sucked - because the design absolutely required it.
Solutions are rip out a core piece of kernel functionality, and in the process basically taking over ownership and maintenance for ALL code anywhere in the kernel assuming the existence of said cache, engineer something that is somehow better, and support this in perpetuity, including any semantic mismatches that turn up much later that were never designed for
The idea of merged ps output where Windows binaries would show up in the Linux /proc. How would you even begin to implement that without confusing EVERY process management tool ever written that interfaced with /proc? What about /proc/.../environ? On UNIX that has no character set. On Windows it is Unicode.
A trillion problems like this made WSL1 a beautiful nightmare. Glad it was tried and from watching the tickets, that team bled heroically trying to make it all work, but ultimately, I'm also glad it's gone, because the replacement is infinitely easier to engineer, maintain, and use, and that team earns its quarterly bonus much more easily. Everyone wins, except the astronauts, but as experience teaches us they never do.
- sedatk 6y agoI agree with the sentiment but, some scenarios have become orders of magnitude more complicated on WSL2 like connecting to a daemon on Windows or vice versa. I understand clear cut security boundaries and separate network interfaces, but it's extremely hard to get them running smoothly now. Everything was on localhost on WSL1. Memory usage has also gone bonkers with the VM approach, causing unnecessary overhead for casual users who don't need performance.
- whoisburbansky 6y agoI apologize if I'm missing something here, but if casual users don't care about performance, what's the overhead here?
- deleted 6y ago[deleted]
- neurostimulant 6y agoIIRC WSL 2 use 50% of your memory in Windows, or 8GB (whichever is smaller) by default.
- chaostheory 6y agoI have 32 GB. Hardware is cheap for Windows. imo That's one of its main perks vs Apple
- ASalazarMX 6y agoIs that an upper bound? My Ubuntu 20.04 console sessions on WSL2 consume a couple tens of Mb.
- Karunamon 6y agoLook for a 'vmmemory' entry in your process list, it'll likely be in the hundreds of megs, if not a gig or so.
- ASalazarMX 6y agoI don't have that process while running WSL2 in my machine. I have vm-agent and vm-agent-daemon, but I'm using a virtual desktop, so this might or not be related to WSL2. Both of these processes consume less than 10 Mb combined. In my other comment [0] I referenced a link from Microsoft that says RAM is dynamically allocated and reclaimed when freed. 0. https://news.ycombinator.com/item?id=25161769 https://news.ycombinator.com/item?id=25161769
- devit 6y agoThe actual advantage of working on something like WSL1 is that it would ensure that the NT kernel is as capable as the Linux kernel (Linux is of course the best designed and implemented among the non-realtime, not provably correct and not secure kernels). For instance, the complaints about I/O performance are because the NT kernel has a worse implementation, so they should have improved it for both Win32 and Linux apps instead of giving up. (your reasoning about the dentry cache makes no sense though since such a cache would be implemented by the NT kernel and certainly not by WSL1, so there's no difference between Linux and Windows programs)
- meddlepal 6y ago> Linux is of course the best designed and implemented among the non-realtime, not provably correct and not secure kernels Citation needed.
- hapless 6y agoAs I understand it, it's not really possible to "improve" Win32 I/O performance -- both the fundamental I/O APIs and the NTFS on-disk storage format make high performance infeasible. Not without either abandoning all extant FS drivers, or abandoning NTFS compatibility, anyway. Edit: Here are the WSL 1.x's team members original comments on this subject. It sounds like a deeply intractable problem. https://github.com/microsoft/WSL/issues/873#issuecomment-424914762 https://github.com/microsoft/WSL/issues/873#issuecomment-424... https://github.com/microsoft/WSL/issues/873#issuecomment-425272829 https://github.com/microsoft/WSL/issues/873#issuecomment-425... The second link explains why a dentry cache just isn't feasible. The short version is that the NT IO APIs seem to have been very, very ill-considered. It's just not possible to make it fast. Even win32 operations are extremely slow, so win32 applications go out of their way to avoid doing any file I/O. Linux applications were not written with those constraints in mind.
- dblohm7 6y agoAmusingly enough, I just watched a talk about this topic earlier today: https://www.youtube.com/watch?v=qbKGw8MQ0i8 https://www.youtube.com/watch?v=qbKGw8MQ0i8
- j0057 6y agoNTFS performance leaves much to be desired on Windows too, it really hurts for workloads dealing with lots of small files, such as programming.
- jhallenworld 6y agoI was wondering if Wine on Ubuntu is faster than Windows native. I found this OSBench near the end of this page (look for "Test:Create Files"): https://www.phoronix.com/scan.php?page=article&item=wine-ubuntu1804-win10&num=4 https://www.phoronix.com/scan.php?page=article&item=wine-ubu... Ubuntu native is much faster. It's odd that Wine is so slow.. I wonder why?
- userbinator 6y agoLinux and Unix in general seems to love huge writeback caching, which is great for speed but horrible for consistency and reliability to power failure and such; on the other hand, Windows flushes the caches more often, providing greater reliability but without as much speed. That's been my experience, in any case; doing lots of small file operations barely causes any disk activity in Linux, but far more in Windows. Moreover, abruptly cutting power in the middle of that would likely result in far more writes lost on Linux than on Windows.
- spacenick88 6y agoLinux/Unix just trust that if you want something persisted with certainty you'll do an fsync, if you do it will absolutely guarantee you're not losing that. It will absolutely make sure that your filesystem doesn't get corrupted by a power loss but if you didn't fsync your write you had no place believing it was persisted. Doing that IMHO matches real world use much better. If I do a compile and get a power outage I don't care if some object files get lost. If I do an INSERT in a DB I do care a lot but the DB knows that and will fsync before telling me it succeeded. So making sync explicit gives you both great performance and flexibility.
- 6y ago
- alerighi 6y agoCorrect, but... this is not WSL. This is a VM. Let's see this from the opposite point of view: WSL1 is to WSL2 what WINE is to a VM running Windows. Two completely different approaches. The thing that would have allowed the WSL1 to do was to really integrate between Windows and Linux: imagine writing running a Linux command, piping its output in a Windows command to pipe it again in a Linux command. Imagine sending a signal to a Linux process from a Windows process. Or imagine accessing physical hardware from the Linux subsystem. With the WSL2 is impossible to even use a serial port! WSL1 had performance problems. That thing about the filesystem performance could have been the occasion to optimize NTFS and even to add to the NT kernel the support for other filesystem, natively. Or why not add to the NT kernel other Linux functionality, to make them available both in the Linux and Windows subsystems. Of course not everything would have been possible to map on the NT kernel. That is fine. But for most application really the WSL1 was usable. Maybe the real problem with the WSL was only having called it "Widnows subsystem for Linux". They should really have made WSP: Windows Subsystem for POSIX. Drop the binary compatibility with Linux requirement and just make it an environment in which you can compile code that uses the POSIX API and interface with Windows. Not having binary compatibility with Linux is not a big deal, thing about it but is what macOS does and nobody seems to care, and users would have created a package manager just like there is brew in macOS to easily install software.
- moron4hire 6y agoWindows Subsystem for POSIX was already tried. It didn't take off.
- josephcsible 6y agoIsn't the only reason that it didn't take off because it was so incomplete?
- sgerenser 6y agoIncomplete and nobody really knew it existed.
- 6y ago
- brendangregg 6y agoGood points; in addition, I can only imagine the headache of trying to support the latest eBPF, io_uring, WireGuard, and other advanced kernel features in WSL1. I imagine a lot of newer features would return ENOSUPP from the kernel, so WSL1 basically becomes a weird fork of an old Linux kernel. (Weird because it lacks a large community, and old because the ENOSUPP's downgrade what's available.) In WSL2 everything should work (depending on the CONFIG options and how much Microsoft has changed).
- wahern 6y agoYour point about the dentry cache seems like a non-sequitur. The cache isn't the bottleneck; the bottleneck is that NTFS is a crappy filesystem. Yes, I've read this comment: https://github.com/microsoft/WSL/issues/873#issuecomment-425272829 https://github.com/microsoft/WSL/issues/873#issuecomment-425... The argument about Windows not having a single, central dentry cache doesn't hold water. For one thing, there's no reason to think that a two-level cache would significantly decrease performance. More importantly, the fact that on Windows most VFS work occurs in the filesystem driver itself only proves that Windows could have implemented an EXT4 driver with better dentry and other semantics and permitted WSL1 environments to keep most of its files on an EXT4 volume, which is what happens with WSL2, anyhow. WSL1 could have been better in every way. It would have required them to track a moving target, but they already accomplished semantic parity with seemingly minimal resources. Improving performance was certainly possible, and keeping up would have required significantly less work on its own. At the end of the day, I think the reason WSL1 was canned is obvious: the decision to ship WSL1 was probably accidental. Why accidental? Because anyone with the slightest business acumen would realize that a WSL1 environment with feature and performance parity to open source Linux would mean there'd be little reason to target Windows' native environment at all for new software development. And while Microsoft has done relatively well for itself with Azure and other ventures, its revenue still principally derives from Windows and Office, and the last thing Microsoft needs is to accelerate movement away from those products. WSL2 means that Microsoft can keep its Linux environment a second-class citizen without being blamed for it, while still providing all the convenience necessary for doing Linux server application development locally. Integration with the native Windows environment will be handicapped by design and excused as an insurmountable consequence of a VM architecture. For example, with WSL1 and some obvious and straight-forward improvements it would have been trivial to run a Linux-built Electron app with identical performance, responsiveness, and behavior (not to mention look & feel, given the web-like UI). With WSL2 Microsoft will likely only ever provide GUI integration over RDP, which will never have the same responsiveness (not because it's theoretically impossible, but because nobody would ever demand it, and in any event it would require tremendously more effort than the comparable WSL1 approach).
- for_xyz 6y ago> the bottleneck is that NTFS is a crappy filesystem. Care to explain why do you think NTFS is crap? From my experience it's usually bad assumptions about files under windows and every application or library tries to stick to posix interface when dealing with files (open, read/write, close) which tends to block for longer periods on windows than on linux counterparts which results in significant perfomance loss . Linux first software will always outperform windows implementation and Windows first software will outperform Linux implementations unless you provide separate code paths to properly handle underlying OS architecture and assumptions. On Windows closing the file handle is extremely costly operation due AV checks and file content indexing [1] [1] https://www.youtube.com/watch?v=qbKGw8MQ0i8 https://www.youtube.com/watch?v=qbKGw8MQ0i8
- kabes 6y agoWhat are the trillion other problems? Honest question, some I heard this a lot, but in reality I only ever saw people complain about the speed of the filesystem. So that would just mean 1 (hard) thing to fix.
- ralph87 6y agoThe network stack integration was less than transparent too. Random places all over the system where e.g. a magical ioctl() just didn't work properly. Who really wants to spend their life reimplementing every last baroque detail of that stuff?
- utxaa 6y agonot just the dentry cache: https://github.com/microsoft/WSL/issues/873#issuecomment-425272829 https://github.com/microsoft/WSL/issues/873#issuecomment-425...