4 ms·

DirectX is coming to the Windows Subsystem for Linux

66 points by localhost 6y ago

cannedslime 6y ago
Im a little confused what the impact of this is... We will finally have 3D acceleration on WSL? Does this mean I can "finally" train TensorFlow stuff on WSL?
- localhost 6y ago
  Yes, deep learning can now use your local GPU. I've been waiting for this a long time, as it means that I can run this stuff reliably on a laptop with an NVidia GPU now. It's not clear when this will hit the Insiders Fast Ring builds yet though ... can't arrive fast enough for me.
- pjmlp 6y ago
  Yes, it exposes DirectX bindings directly in WSL.
spystath 6y ago
The relevant bits are available in the WSL2 fork of the linux kernel. I know MS is probably doing that for access to GPU hardware for ML/AI training but I'm wondering (and hope) if this would indirectly benefit the DirectX implementation efforts in Wine.
- pantalaimon 6y ago
  Unlikely, the kernel module is just a shim to redirect the DirectX calls to the host DirectX driver. You will always need WSL for this to work.
pantalaimon 6y ago
This is the old Embrace, Extend and Extinguish all over again. Now you can write Linux software that uses Microsoft APIs, but will only work with WSL2. There is no way to run DirectML on a bare-metal system. You'll always need Windows in the background. Microsoft noticed the Machine Learning / GPU compute ecosystem has moved to Linux, now they attempt to tie it back to Windows.
- ryukafalz 6y ago
  Yeah, I’ll grant that they’ve done a lot of work on Linux recently, but this seems pretty blatant: > libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows.
  - zozbot234 6y ago
    Doesn't Wine/Proton support running DirectX on the ordinary Linux stack anyway? What stops folks from implementing compatible .so binaries that hook into the existing Wine to support this?
- aseipp 6y ago
  Meh, it's just more proprietary middleware, which is nothing unusual for the Machine Learning crowd. They don't care. Everyone will still use PyTorch or TensorFlow (or just CUDA) and whether or not it directly talks with nvidia's Linux driver stack or through DirectML to the WDDM stack or through CUDA to the WDDM stack isn't relevant for most people, with one exception: training performance. The vast majority of actual large scale training and inference happens on headless servers anyway and even if it did run on Windows Server you'd just pass a PCIe card directly through to a Linux VM with Hyper-V, you already had that for a while. I see this as a response to one of WSL's most requested features by developers. "Machine Learning Support" has been a day 1 bug report, and in practice it means "Run TensorFlow/Pytorch" which in practice means "Support CUDA". Frankly if they only did that, it'd be fine (AMD DL support doesn't exist meaningfully), so the fact they went with this whole DirectML thing, allowing generic DirectX 12 compatible GPUs to work, is a surprise. It's more goodwill, in other words, for researchers/developers doing local work. Nobody is going to use these APIs directly, and those who do just need them purely as "device middleware" for the underlying hardware stack, so some generic thing can sit on top. The fact that they're working on Mesa to bring OpenGL to WSL applications, by way of WDDM/DXGI, is more evidence of this, I think, because it will not just benefit ML apps. "Embrace, Extend, Extinguish" implies that Microsoft wants to subvert things and attract people to their APIs, over time, so they can do a 180. But they don't want Linux users to use DirectX APIs on Linux. They don't have time to make that happen. They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. DirectX only is a means here, not an end. It's the same reason they went from "crazy ersatz clone of the Linux kernel inside the Windows Kernel that works 85% of the time and fails otherwise" (WSL1) to "Run ordinary Linux in a hypervisor and call it a day" (WSL2). Yeah, the first version makes nerds on Hacker News happy because it's a cool hack, but the second version actually means your Linux software will run as is, and it works for users, which is more important. They need OpenGL, and TensorFlow, and whatever, to run on WSL as they exist today to attract people. Not in some theoretical future where everyone is using DirectX APIs on Linux and then Microsoft does a backflip and says "gotcha!" It will actually be hilarious if TensorFlow-on-Linux-on-DirectML-via-WDDM could actually provide a viable mechanism to use AMD GPUs for Deep Learning, while keeping the same high level software stack.
dzonga 6y ago
EEE, slowly the Linux desktop | os environment will be a thing of the past. something you won't run directly but interact via the windows api. and your only other choice will be mac os on arm. the biggest mistake the linux ecosystem did, was being so fragmented. if they had rallied around one desktop env, then lots of people would've migrated. I personally use a t490 on ubuntu and a 2013 macbook. but even I I'm attempted to try out WSL even though, I haven't used windows in over a decade
timvisee 6y ago
This allows DirectX usage on Linux. The catch is that it only works on a Windows host. This is for WSL specifically, running in Hyper-V on a Windows host. It basically forwards DirectX calls to the Windows kernel through paravirtualization.
bmh 6y ago
The headline should be "CUDA support on WSL2", but that's only mentioned at the bottom of the article. And having all this stuff work out of the box, I can see some people choosing the WSL path over native Linux simply because there are less apt-gets involved to get a PyTorch model training.

66 points by localhost 6y ago

Im a little confused what the impact of this is... We will finally have 3D acceleration on WSL? Does this mean I can "finally" train TensorFlow stuff on WSL?

localhost 6y ago
Yes, deep learning can now use your local GPU. I've been waiting for this a long time, as it means that I can run this stuff reliably on a laptop with an NVidia GPU now. It's not clear when this will hit the Insiders Fast Ring builds yet though ... can't arrive fast enough for me.
pjmlp 6y ago
Yes, it exposes DirectX bindings directly in WSL.

spystath 6y ago

The relevant bits are available in the WSL2 fork of the linux kernel. I know MS is probably doing that for access to GPU hardware for ML/AI training but I'm wondering (and hope) if this would indirectly benefit the DirectX implementation efforts in Wine.

pantalaimon 6y ago
Unlikely, the kernel module is just a shim to redirect the DirectX calls to the host DirectX driver. You will always need WSL for this to work.

pantalaimon 6y ago

This is the old Embrace, Extend and Extinguish all over again. Now you can write Linux software that uses Microsoft APIs, but will only work with WSL2. There is no way to run DirectML on a bare-metal system. You'll always need Windows in the background. Microsoft noticed the Machine Learning / GPU compute ecosystem has moved to Linux, now they attempt to tie it back to Windows.

ryukafalz 6y ago
Yeah, I’ll grant that they’ve done a lot of work on Linux recently, but this seems pretty blatant: > libd3d12.so and libdxcore.so are closed source, pre-compiled user mode binaries that ship as part of Windows.
- zozbot234 6y ago
  Doesn't Wine/Proton support running DirectX on the ordinary Linux stack anyway? What stops folks from implementing compatible .so binaries that hook into the existing Wine to support this?
aseipp 6y ago
Meh, it's just more proprietary middleware, which is nothing unusual for the Machine Learning crowd. They don't care. Everyone will still use PyTorch or TensorFlow (or just CUDA) and whether or not it directly talks with nvidia's Linux driver stack or through DirectML to the WDDM stack or through CUDA to the WDDM stack isn't relevant for most people, with one exception: training performance. The vast majority of actual large scale training and inference happens on headless servers anyway and even if it did run on Windows Server you'd just pass a PCIe card directly through to a Linux VM with Hyper-V, you already had that for a while. I see this as a response to one of WSL's most requested features by developers. "Machine Learning Support" has been a day 1 bug report, and in practice it means "Run TensorFlow/Pytorch" which in practice means "Support CUDA". Frankly if they only did that, it'd be fine (AMD DL support doesn't exist meaningfully), so the fact they went with this whole DirectML thing, allowing generic DirectX 12 compatible GPUs to work, is a surprise. It's more goodwill, in other words, for researchers/developers doing local work. Nobody is going to use these APIs directly, and those who do just need them purely as "device middleware" for the underlying hardware stack, so some generic thing can sit on top. The fact that they're working on Mesa to bring OpenGL to WSL applications, by way of WDDM/DXGI, is more evidence of this, I think, because it will not just benefit ML apps. "Embrace, Extend, Extinguish" implies that Microsoft wants to subvert things and attract people to their APIs, over time, so they can do a 180. But they don't want Linux users to use DirectX APIs on Linux. They don't have time to make that happen. They want Linux users to use Linux APIs, but run them on WSL (on Windows). And those Linux users expect normal Linux software to work, out of the box, all the time. Totally different things. DirectX only is a means here, not an end. It's the same reason they went from "crazy ersatz clone of the Linux kernel inside the Windows Kernel that works 85% of the time and fails otherwise" (WSL1) to "Run ordinary Linux in a hypervisor and call it a day" (WSL2). Yeah, the first version makes nerds on Hacker News happy because it's a cool hack, but the second version actually means your Linux software will run as is, and it works for users, which is more important. They need OpenGL, and TensorFlow, and whatever, to run on WSL as they exist today to attract people. Not in some theoretical future where everyone is using DirectX APIs on Linux and then Microsoft does a backflip and says "gotcha!" It will actually be hilarious if TensorFlow-on-Linux-on-DirectML-via-WDDM could actually provide a viable mechanism to use AMD GPUs for Deep Learning, while keeping the same high level software stack.

dzonga 6y ago

EEE, slowly the Linux desktop | os environment will be a thing of the past. something you won't run directly but interact via the windows api. and your only other choice will be mac os on arm. the biggest mistake the linux ecosystem did, was being so fragmented. if they had rallied around one desktop env, then lots of people would've migrated. I personally use a t490 on ubuntu and a 2013 macbook. but even I I'm attempted to try out WSL even though, I haven't used windows in over a decade

timvisee 6y ago

This allows DirectX usage on Linux. The catch is that it only works on a Windows host. This is for WSL specifically, running in Hyper-V on a Windows host. It basically forwards DirectX calls to the Windows kernel through paravirtualization.

bmh 6y ago

The headline should be "CUDA support on WSL2", but that's only mentioned at the bottom of the article. And having all this stuff work out of the box, I can see some people choosing the WSL path over native Linux simply because there are less apt-gets involved to get a PyTorch model training.