5 ms·

For GPU accelerated ML in Win11 (PyTorch w/ CUDA) - is WSL enough nowadays, or still dual boot to Linux?

sxp 4y ago
I was able to get Stable Diffusion and other similar ML systems working in WSL2 on Win11. There might be performance differences between WSL and a native system. I haven't benchmarked AI in different configs, but my main use case is Rust+wgpu and there were noticeable performance differences because the GPU driver exposed by WSL didn't have as many features as the GPU driver used on Windows. I also had problems with other APIs on WSL such as Optix.
- arprocter 4y ago
  fwiw I have Stable Diffusion working in WSL2 on Windows 10
- rfoo 4y ago
  For my CUDA workload WSL2 (or really, Hyper-V GPU-PV) is about 75% native performance. Isn't too bad, I still prefer running non-virtualized Linux for anything longer than overnight. OTOH, run Linux on the metal and Windows in KVM works just better.
lostmsu 4y ago
I've been using PyTorch + CUDA on native Windows without much problems for a few years now.
- deepsquirrelnet 4y ago
  I’ve also been doing native windows without too much issue. I have run into difficulty setting up libraries like detectron though. For the record, I loathe working in the windows terminal. But for people like me, conda is a lifesaver. The m2-base packages bring a pretty good number of tools into the terminal and significantly resurrect its usability. I’m a little tempted to try wsl again with this improvement, but my last experience wasn’t great with it and I’ve gotten comfortable enough without it.
  - lostmsu 4y ago
    CUDA worked in WSL since WSLg just started. I used it, as you mentioned, for a few PyTorch modules, that were Linux-only. Slight pain to setup, but one time only.
pridkett 4y ago
It works well enough for me to not worry about dual booting. Windows tends to claim some of the GPU memory - about 1.5GB on my 3090. If I do anything performance sensitive I’ll still do it in the cloud - but it has made my gaming/flight simulator PC massively more useful. You can also set up a network so your WSL2 installation shows up on your network as another host. Which is nice. Docker and docker with CUDA are a breeze too. Overall, I’m pretty happy with WSL2, but realize there are some performance drawbacks - so if my machine wasn’t massively overspec’d, I’d be much more concerned.
- indigodaddy 4y ago
  Is there any official doc for the network/bridged setup? Wasn’t aware that was possible..
  - pridkett 4y ago
    It was a haul and I'm realizing now that I don't have it properly documented in my notes. But, that was a haul over the course of a couple years that had me experimenting with external USB network cards, virtual switches, and more. It's gotten way easier now. In your .wslconfig you'll want something like this: [wsl2] memory=48GB localhostforwarding=true networkingMode=bridged vmSwitch=WSL_external The key things are "networkingMode=bridged" and "vmSwitch=WSL_external". Then, open up Hyper-V manager on Windows, go to "Virtual Switch Manager" and create a new virtual switch using your primary ethernet adapter (I could never get this to work with my USB ethernet adapter). Check the boxed for "External Network" and "Allow management operating system to share this network adapter" and finally call it "WSL_external". That should do it for you. Incidentally, the port forwarding with WSL2 and Docker, has made this not entirely necessary, but I still like being able to SSH directly into my WSL2 host, not having to manage Windows' firewall rules, and being able to forward UDP for things like Mosh.
- deleted 4y ago
  [deleted]
colechristensen 4y ago
Likewise to the other commenters, I've been using pytorch + cuda on windows and it's been working fine using miniconda to install most of the compiled bits.

sxp 4y ago

I was able to get Stable Diffusion and other similar ML systems working in WSL2 on Win11. There might be performance differences between WSL and a native system. I haven't benchmarked AI in different configs, but my main use case is Rust+wgpu and there were noticeable performance differences because the GPU driver exposed by WSL didn't have as many features as the GPU driver used on Windows. I also had problems with other APIs on WSL such as Optix.

arprocter 4y ago
fwiw I have Stable Diffusion working in WSL2 on Windows 10
rfoo 4y ago
For my CUDA workload WSL2 (or really, Hyper-V GPU-PV) is about 75% native performance. Isn't too bad, I still prefer running non-virtualized Linux for anything longer than overnight. OTOH, run Linux on the metal and Windows in KVM works just better.

lostmsu 4y ago

I've been using PyTorch + CUDA on native Windows without much problems for a few years now.

deepsquirrelnet 4y ago
I’ve also been doing native windows without too much issue. I have run into difficulty setting up libraries like detectron though. For the record, I loathe working in the windows terminal. But for people like me, conda is a lifesaver. The m2-base packages bring a pretty good number of tools into the terminal and significantly resurrect its usability. I’m a little tempted to try wsl again with this improvement, but my last experience wasn’t great with it and I’ve gotten comfortable enough without it.
- lostmsu 4y ago
  CUDA worked in WSL since WSLg just started. I used it, as you mentioned, for a few PyTorch modules, that were Linux-only. Slight pain to setup, but one time only.

pridkett 4y ago

It works well enough for me to not worry about dual booting. Windows tends to claim some of the GPU memory - about 1.5GB on my 3090. If I do anything performance sensitive I’ll still do it in the cloud - but it has made my gaming/flight simulator PC massively more useful. You can also set up a network so your WSL2 installation shows up on your network as another host. Which is nice. Docker and docker with CUDA are a breeze too. Overall, I’m pretty happy with WSL2, but realize there are some performance drawbacks - so if my machine wasn’t massively overspec’d, I’d be much more concerned.

indigodaddy 4y ago
Is there any official doc for the network/bridged setup? Wasn’t aware that was possible..
- pridkett 4y ago
  It was a haul and I'm realizing now that I don't have it properly documented in my notes. But, that was a haul over the course of a couple years that had me experimenting with external USB network cards, virtual switches, and more. It's gotten way easier now. In your .wslconfig you'll want something like this: [wsl2] memory=48GB localhostforwarding=true networkingMode=bridged vmSwitch=WSL_external The key things are "networkingMode=bridged" and "vmSwitch=WSL_external". Then, open up Hyper-V manager on Windows, go to "Virtual Switch Manager" and create a new virtual switch using your primary ethernet adapter (I could never get this to work with my USB ethernet adapter). Check the boxed for "External Network" and "Allow management operating system to share this network adapter" and finally call it "WSL_external". That should do it for you. Incidentally, the port forwarding with WSL2 and Docker, has made this not entirely necessary, but I still like being able to SSH directly into my WSL2 host, not having to manage Windows' firewall rules, and being able to forward UDP for things like Mosh.
deleted 4y ago
[deleted]

colechristensen 4y ago

Likewise to the other commenters, I've been using pytorch + cuda on windows and it's been working fine using miniconda to install most of the compiled bits.