4 ms·

> No. Microsoft tried to bridge the Linux APIs to NT APIs in WSL1 just like WINE does for Windows APIs to Linux APIs. Uhh, I think you miss read my post. I agr

> No. Microsoft tried to bridge the Linux APIs to NT APIs in WSL1 just like WINE does for Windows APIs to Linux APIs. Uhh, I think you miss read my post. I agree with you WSL1 was a dead end. It tried to put Linux on top of NT. I believe MS will try to put Win32, etc on top of Native Linux APIs (removing NT entirely). WINE and Proton have effectively done 90% and MSFT will have a way simpler time of getting it to 100% given they dont have to worry about their own copyright. MSFT also has the financial motivation to stop developing the NT Kernel, so it’s valuable for them to invest into this “WINE++”.

skissane 4y ago
I think it is unlikely even Microsoft would succeed in replacing the NT kernel with the Linux kernel. NT has features which Linux kernel devs have actively opposed including – such as alternate data stream support in its VFS/IFS layer, and a stable device driver ABI. So even supposing – and I'd be rather surprised if it were to ever actually happen – Microsoft were to release the NT kernel under a GPL-compatible license (thereby eliminating many of the legal issues), they still might find it a great struggle to get the necessary features upstreamed. Important parts of Windows – e.g. synchronisation primitives – cannot be emulated on top of the Linux syscall API as it currently exists (with accuracy and high performance), due to gaps in functionality. See https://lore.kernel.org/lkml/f4cc1a38-1441-62f8-47e4-0c67f5ad1d43@codeweavers.com/ https://lore.kernel.org/lkml/f4cc1a38-1441-62f8-47e4-0c67f5a... for some of the gory details. I note that LKML message appears to have received zero replies, which is not an encouraging sign for any progress on that issue. > MSFT also has the financial motivation to stop developing the NT Kernel It would be a huge investment to extend the Linux kernel to be able to support a 100% correct and equal performance emulation of the NT API, and to port that API to run on top of it. Even with such a huge investment, there would be no guarantee of success (what if Linus refuses to upstream the required features?). And, if it legally requires open-sourcing large parts of the existing Windows kernel (due to GPL compatibility), that would greatly improve the viability of Wine/ReactOS/etc, threatening Microsoft's existing Windows revenue stream. The financial case for what you are proposing is far less clear than you think it is.
- yazaddaruvala 4y ago
  Love your response. Im sure you’re right about all of those details if a strict translation was needed. I just don’t believe it needs to be so strict, and as such that it is not that difficult these days. Apple’s Rosetta 2 and Valve’s Proton have shown that these kind of emulations can be extremely effective. When you control the OS code, and are willing to be extremely hacky for the medium term, anything is possible. If I were to lead this effort I’d do: 1. Try my best porting the Win32 APIs, Direct3d, to Linux. I’d convert many of them. Use Office, the Windows window manager, Visual Studio, etc as test suites. This can be deployed to users where there is no performance regression. 2. Update any non-legacy applications to use more of WSL2 especially were incompatible with the new Win32 APIs. eg Office, Electron, Chrome, Visual Studio. 3. Build a binary parser like Rosetta that detects use of any incompatible Windows APIs. Mark these binaries as “legacy” (this parsing only needs to be run once per binary). Run legacy binary processes through a hyper visor and slimmed down snapshot of Windows. 4. Investigate more traspiling of binary code like Rosetta, but for Windows APIs to WSL APIs. 5. Announce it and state the intention to migrate. Many modern apps are Electron anyways. Actively supported apps will slowly migrate to more performant APIs. Legacy apps were not designed for today's hardware anyways and it would work just fine with a little performance degradation but overall still a performance win compared to when the app was released.
  - skissane 4y ago
    > Apple’s Rosetta 2 and Valve’s Proton have shown that these kind of emulations can be extremely effective. Rosetta and Rosetta 2 are CPU emulation with the same underlying OS – so quite a different problem from what WINE/Proton/etc address. And something Microsoft actually already has anyway – their implementation is less impressive than Apple's in terms of performance, although it can do some interesting things which Apple's can't – in particular, mix emulated x86-64 and native ARM code in a single binary image and process (Arm64EC) – Apple fat binaries are two completely separate binaries concatenated together into a single file, only one of which is actually loaded; Windows Arm64EC binaries mix x86-64 and Aarch64 code in a single binary image, so one function can be native and the other emulated. A program can be native Aarch64, but still be able to load legacy x86-64 plugin DLLs, with some performance penalty incurred for the later [0] – something Rosetta or Rosetta 2 can't do, although Classic MacOS had similar support for mixing emulated 68k and native PPC code in a single process. Valve's Proton is just a fork of Wine, and it works great for a carefully curated subset of games. However, games are a relatively narrow category of applications, there are lots of things other categories of applications will do which games never will. > 3. Build a binary parser like Rosetta that detects use of any incompatible Windows APIs Sometimes, the problem is not with the APIs themselves, but particular patterns of using them. A straightforward emulation of a Win32 API under Linux will behave the same for the vast majority of cases, but in rare edge cases will behave differently. Inspecting a binary to detect what APIs it uses won't tell you whether it is one of the minority of APIs which depends on one of those edge cases. Sometimes the developers themselves won't even know, because it is not unheard of for developers to do weird stupid things by accident rather than intention, yet it just so happens they work, and they don't even realise they've done something weird and stupid. The biggest problem with your proposal, is will the huge dollar investment it would require of Microsoft, actually be worth it for them? Even if (maybe) saves money in the long run, it will cost a lot more in the short-to-medium term. And the fact is, Windows licensing revenue is still massive enough to more than pay for what Microsoft is spending on Windows development, so there is no financial pressure for them to save money in the long-run – and migrating Windows to Linux would make it easier for their customers to migrate away from Windows, hence risking that revenue stream. Why take such a big risk with one of their cash cows for such an unclear benefit for them? [0] https://devblogs.microsoft.com/windows-music-dev/load-x64-plug-ins-like-vsts-from-your-arm-code-using-arm64ec/ https://devblogs.microsoft.com/windows-music-dev/load-x64-pl...

skissane 4y ago

I think it is unlikely even Microsoft would succeed in replacing the NT kernel with the Linux kernel. NT has features which Linux kernel devs have actively opposed including – such as alternate data stream support in its VFS/IFS layer, and a stable device driver ABI. So even supposing – and I'd be rather surprised if it were to ever actually happen – Microsoft were to release the NT kernel under a GPL-compatible license (thereby eliminating many of the legal issues), they still might find it a great struggle to get the necessary features upstreamed. Important parts of Windows – e.g. synchronisation primitives – cannot be emulated on top of the Linux syscall API as it currently exists (with accuracy and high performance), due to gaps in functionality. See https://lore.kernel.org/lkml/f4cc1a38-1441-62f8-47e4-0c67f5ad1d43@codeweavers.com/ https://lore.kernel.org/lkml/f4cc1a38-1441-62f8-47e4-0c67f5a... for some of the gory details. I note that LKML message appears to have received zero replies, which is not an encouraging sign for any progress on that issue. > MSFT also has the financial motivation to stop developing the NT Kernel It would be a huge investment to extend the Linux kernel to be able to support a 100% correct and equal performance emulation of the NT API, and to port that API to run on top of it. Even with such a huge investment, there would be no guarantee of success (what if Linus refuses to upstream the required features?). And, if it legally requires open-sourcing large parts of the existing Windows kernel (due to GPL compatibility), that would greatly improve the viability of Wine/ReactOS/etc, threatening Microsoft's existing Windows revenue stream. The financial case for what you are proposing is far less clear than you think it is.

yazaddaruvala 4y ago
Love your response. Im sure you’re right about all of those details if a strict translation was needed. I just don’t believe it needs to be so strict, and as such that it is not that difficult these days. Apple’s Rosetta 2 and Valve’s Proton have shown that these kind of emulations can be extremely effective. When you control the OS code, and are willing to be extremely hacky for the medium term, anything is possible. If I were to lead this effort I’d do: 1. Try my best porting the Win32 APIs, Direct3d, to Linux. I’d convert many of them. Use Office, the Windows window manager, Visual Studio, etc as test suites. This can be deployed to users where there is no performance regression. 2. Update any non-legacy applications to use more of WSL2 especially were incompatible with the new Win32 APIs. eg Office, Electron, Chrome, Visual Studio. 3. Build a binary parser like Rosetta that detects use of any incompatible Windows APIs. Mark these binaries as “legacy” (this parsing only needs to be run once per binary). Run legacy binary processes through a hyper visor and slimmed down snapshot of Windows. 4. Investigate more traspiling of binary code like Rosetta, but for Windows APIs to WSL APIs. 5. Announce it and state the intention to migrate. Many modern apps are Electron anyways. Actively supported apps will slowly migrate to more performant APIs. Legacy apps were not designed for today's hardware anyways and it would work just fine with a little performance degradation but overall still a performance win compared to when the app was released.
- skissane 4y ago
  > Apple’s Rosetta 2 and Valve’s Proton have shown that these kind of emulations can be extremely effective. Rosetta and Rosetta 2 are CPU emulation with the same underlying OS – so quite a different problem from what WINE/Proton/etc address. And something Microsoft actually already has anyway – their implementation is less impressive than Apple's in terms of performance, although it can do some interesting things which Apple's can't – in particular, mix emulated x86-64 and native ARM code in a single binary image and process (Arm64EC) – Apple fat binaries are two completely separate binaries concatenated together into a single file, only one of which is actually loaded; Windows Arm64EC binaries mix x86-64 and Aarch64 code in a single binary image, so one function can be native and the other emulated. A program can be native Aarch64, but still be able to load legacy x86-64 plugin DLLs, with some performance penalty incurred for the later [0] – something Rosetta or Rosetta 2 can't do, although Classic MacOS had similar support for mixing emulated 68k and native PPC code in a single process. Valve's Proton is just a fork of Wine, and it works great for a carefully curated subset of games. However, games are a relatively narrow category of applications, there are lots of things other categories of applications will do which games never will. > 3. Build a binary parser like Rosetta that detects use of any incompatible Windows APIs Sometimes, the problem is not with the APIs themselves, but particular patterns of using them. A straightforward emulation of a Win32 API under Linux will behave the same for the vast majority of cases, but in rare edge cases will behave differently. Inspecting a binary to detect what APIs it uses won't tell you whether it is one of the minority of APIs which depends on one of those edge cases. Sometimes the developers themselves won't even know, because it is not unheard of for developers to do weird stupid things by accident rather than intention, yet it just so happens they work, and they don't even realise they've done something weird and stupid. The biggest problem with your proposal, is will the huge dollar investment it would require of Microsoft, actually be worth it for them? Even if (maybe) saves money in the long run, it will cost a lot more in the short-to-medium term. And the fact is, Windows licensing revenue is still massive enough to more than pay for what Microsoft is spending on Windows development, so there is no financial pressure for them to save money in the long-run – and migrating Windows to Linux would make it easier for their customers to migrate away from Windows, hence risking that revenue stream. Why take such a big risk with one of their cash cows for such an unclear benefit for them? [0] https://devblogs.microsoft.com/windows-music-dev/load-x64-plug-ins-like-vsts-from-your-arm-code-using-arm64ec/ https://devblogs.microsoft.com/windows-music-dev/load-x64-pl...