5 ms·

This is the old Embrace, Extend and Extinguish all over again. Now you can write Linux software that uses Microsoft APIs, but will only work with WSL2. There i

This is the old Embrace, Extend and Extinguish all over again. Now you can write Linux software that uses Microsoft APIs, but will only work with WSL2. There is no way to run DirectML on a bare-metal system. You'll always need Windows in the background. Microsoft noticed the Machine Learning / GPU compute ecosystem has moved to Linux, now they attempt to tie it back to Windows.

ryukafalz 6y ago
Yeah, I’ll grant that they’ve done a lot of work on Linux recently, but this seems pretty blatant: > libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows.
- zozbot234 6y ago
  Doesn't Wine/Proton support running DirectX on the ordinary Linux stack anyway? What stops folks from implementing compatible .so binaries that hook into the existing Wine to support this?
aseipp 6y ago
Meh, it's just more proprietary middleware, which is nothing unusual for the Machine Learning crowd. They don't care. Everyone will still use PyTorch or TensorFlow (or just CUDA) and whether or not it directly talks with nvidia's Linux driver stack or through DirectML to the WDDM stack or through CUDA to the WDDM stack isn't relevant for most people, with one exception: training performance. The vast majority of actual large scale training and inference happens on headless servers anyway and even if it did run on Windows Server you'd just pass a PCIe card directly through to a Linux VM with Hyper-V, you already had that for a while. I see this as a response to one of WSL's most requested features by developers. "Machine Learning Support" has been a day 1 bug report, and in practice it means "Run TensorFlow/Pytorch" which in practice means "Support CUDA". Frankly if they only did that, it'd be fine (AMD DL support doesn't exist meaningfully), so the fact they went with this whole DirectML thing, allowing generic DirectX 12 compatible GPUs to work, is a surprise. It's more goodwill, in other words, for researchers/developers doing local work. Nobody is going to use these APIs directly, and those who do just need them purely as "device middleware" for the underlying hardware stack, so some generic thing can sit on top. The fact that they're working on Mesa to bring OpenGL to WSL applications, by way of WDDM/DXGI, is more evidence of this, I think, because it will not just benefit ML apps. "Embrace, Extend, Extinguish" implies that Microsoft wants to subvert things and attract people to their APIs, over time, so they can do a 180. But they don't want Linux users to use DirectX APIs on Linux. They don't have time to make that happen. They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. DirectX only is a means here, not an end. It's the same reason they went from "crazy ersatz clone of the Linux kernel inside the Windows Kernel that works 85% of the time and fails otherwise" (WSL1) to "Run ordinary Linux in a hypervisor and call it a day" (WSL2). Yeah, the first version makes nerds on Hacker News happy because it's a cool hack, but the second version actually means your Linux software will run as is, and it works for users, which is more important. They need OpenGL, and TensorFlow, and whatever, to run on WSL as they exist today to attract people. Not in some theoretical future where everyone is using DirectX APIs on Linux and then Microsoft does a backflip and says "gotcha!" It will actually be hilarious if TensorFlow-on-Linux-on-DirectML-via-WDDM could actually provide a viable mechanism to use AMD GPUs for Deep Learning, while keeping the same high level software stack.
- ta17711771 6y ago
  Scientific technology users need to see the big picture of what they're enabling...hell, everyone does.
- themacguffinman 6y ago
  > They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. This is exactly on course for EEE. EEE isn't about increasing DirectX adoption on regular Linux, it's about using DirectX as a wedge to increase Microsoft product adoption (ie. Windows, Azure and co.) and marginalize regular Linux. By marginalizing regular Linux, they also marginalize every competitor that supports regular Linux. The theoretical future moves in stages. Today it's "only Windows can support DirectX on Linux! But don't worry, it's still Linux, just this one extension." Then it's "DirectX on Linux now works best with Windows Machine Learning Foundation or some crap like that, only on Windows!" Then it's "Windows Machine Learning Foundation is now only on Windows Professional Plus" Then it's "WSL enjoys deep integration with Visual Studio, only on Windows!" Then it's "Azure ML is the best way to run DirectX on Linux in the cloud! You see, the Azure Advantage (TM) is that only we know how to run DirectX on Linux effectively, because only we can see and change the code." At some point, you stopped using open source tooling, stopped using an open source OS, and stopped being compatible with open source Linux providers like AWS & GCP. Microsoft practically owns your entire stack. "Gotcha!"
vkaku 6y ago
You always have the Wine project, and one could contribute to such a project if DirectML is the way one wants to run ML on a system - if they choose to ignore the other APIs to use the GPU.
colejohnson66 6y ago
I keep seeing this EEE resentment here, and I don’t understand it. It’s the same claim whether it’s Linux, git, etc. How would Microsoft even “extinguish” Linux, git, etc?
- pantalaimon 6y ago
  All the Machine Learning tools are on Linux. Microsoft wants to stop the developer bleed, so they offer a way to run those tools on WSL, but using their own API. Now tools may emerge that only work with the DirectML API, but not on native Linux.
- themacguffinman 6y ago
  The strategy's three phases are [1]: 1. Embrace: Development of software substantially compatible with a competing product, or implementing a public standard. 2. Extend: Addition and promotion of features not supported by the competing product or part of the standard, creating interoperability problems for customers who try to use the "simple" standard. 3. Extinguish: When extensions become a de facto standard because of their dominant market share, they marginalize competitors that do not or cannot support the new extensions. DirectX bindings are phase 2: encourage users and programs to depend on a Microsoft-exclusive variant of standard Linux (ie. WSL), touting compatibility with standard Linux. When enough users and programs depend on these Microsoft-exclusive features, Microsoft can start phase 3: break compatibility with standard Linux and marginalize it because it is now more popular. [1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...

ryukafalz 6y ago

Yeah, I’ll grant that they’ve done a lot of work on Linux recently, but this seems pretty blatant: > libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows.

zozbot234 6y ago
Doesn't Wine/Proton support running DirectX on the ordinary Linux stack anyway? What stops folks from implementing compatible .so binaries that hook into the existing Wine to support this?

aseipp 6y ago

Meh, it's just more proprietary middleware, which is nothing unusual for the Machine Learning crowd. They don't care. Everyone will still use PyTorch or TensorFlow (or just CUDA) and whether or not it directly talks with nvidia's Linux driver stack or through DirectML to the WDDM stack or through CUDA to the WDDM stack isn't relevant for most people, with one exception: training performance. The vast majority of actual large scale training and inference happens on headless servers anyway and even if it did run on Windows Server you'd just pass a PCIe card directly through to a Linux VM with Hyper-V, you already had that for a while. I see this as a response to one of WSL's most requested features by developers. "Machine Learning Support" has been a day 1 bug report, and in practice it means "Run TensorFlow/Pytorch" which in practice means "Support CUDA". Frankly if they only did that, it'd be fine (AMD DL support doesn't exist meaningfully), so the fact they went with this whole DirectML thing, allowing generic DirectX 12 compatible GPUs to work, is a surprise. It's more goodwill, in other words, for researchers/developers doing local work. Nobody is going to use these APIs directly, and those who do just need them purely as "device middleware" for the underlying hardware stack, so some generic thing can sit on top. The fact that they're working on Mesa to bring OpenGL to WSL applications, by way of WDDM/DXGI, is more evidence of this, I think, because it will not just benefit ML apps. "Embrace, Extend, Extinguish" implies that Microsoft wants to subvert things and attract people to their APIs, over time, so they can do a 180. But they don't want Linux users to use DirectX APIs on Linux. They don't have time to make that happen. They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. DirectX only is a means here, not an end. It's the same reason they went from "crazy ersatz clone of the Linux kernel inside the Windows Kernel that works 85% of the time and fails otherwise" (WSL1) to "Run ordinary Linux in a hypervisor and call it a day" (WSL2). Yeah, the first version makes nerds on Hacker News happy because it's a cool hack, but the second version actually means your Linux software will run as is, and it works for users, which is more important. They need OpenGL, and TensorFlow, and whatever, to run on WSL as they exist today to attract people. Not in some theoretical future where everyone is using DirectX APIs on Linux and then Microsoft does a backflip and says "gotcha!" It will actually be hilarious if TensorFlow-on-Linux-on-DirectML-via-WDDM could actually provide a viable mechanism to use AMD GPUs for Deep Learning, while keeping the same high level software stack.

ta17711771 6y ago
Scientific technology users need to see the big picture of what they're enabling...hell, everyone does.
themacguffinman 6y ago
> They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. This is exactly on course for EEE. EEE isn't about increasing DirectX adoption on regular Linux, it's about using DirectX as a wedge to increase Microsoft product adoption (ie. Windows, Azure and co.) and marginalize regular Linux. By marginalizing regular Linux, they also marginalize every competitor that supports regular Linux. The theoretical future moves in stages. Today it's "only Windows can support DirectX on Linux! But don't worry, it's still Linux, just this one extension." Then it's "DirectX on Linux now works best with Windows Machine Learning Foundation or some crap like that, only on Windows!" Then it's "Windows Machine Learning Foundation is now only on Windows Professional Plus" Then it's "WSL enjoys deep integration with Visual Studio, only on Windows!" Then it's "Azure ML is the best way to run DirectX on Linux in the cloud! You see, the Azure Advantage (TM) is that only we know how to run DirectX on Linux effectively, because only we can see and change the code." At some point, you stopped using open source tooling, stopped using an open source OS, and stopped being compatible with open source Linux providers like AWS & GCP. Microsoft practically owns your entire stack. "Gotcha!"

vkaku 6y ago

You always have the Wine project, and one could contribute to such a project if DirectML is the way one wants to run ML on a system - if they choose to ignore the other APIs to use the GPU.

colejohnson66 6y ago

I keep seeing this EEE resentment here, and I don’t understand it. It’s the same claim whether it’s Linux, git, etc. How would Microsoft even “extinguish” Linux, git, etc?

pantalaimon 6y ago
All the Machine Learning tools are on Linux. Microsoft wants to stop the developer bleed, so they offer a way to run those tools on WSL, but using their own API. Now tools may emerge that only work with the DirectML API, but not on native Linux.
themacguffinman 6y ago
The strategy's three phases are [1]: 1. Embrace: Development of software substantially compatible with a competing product, or implementing a public standard. 2. Extend: Addition and promotion of features not supported by the competing product or part of the standard, creating interoperability problems for customers who try to use the "simple" standard. 3. Extinguish: When extensions become a de facto standard because of their dominant market share, they marginalize competitors that do not or cannot support the new extensions. DirectX bindings are phase 2: encourage users and programs to depend on a Microsoft-exclusive variant of standard Linux (ie. WSL), touting compatibility with standard Linux. When enough users and programs depend on these Microsoft-exclusive features, Microsoft can start phase 3: break compatibility with standard Linux and marginalize it because it is now more popular. [1] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...