11 ms·
WSL with CUDA support
Windows Subsystem for Linux with CUDA support + TensorFlow
Let me save you many hours of pain and just give you the rundown.
* You MUST be on Windows 10 21H2 or above. 21H2 cannot currently be updated to (yeah, it's because they are pushing Windows 11) so you need to download it as an ISO.
* Install the latest "game ready driver" for your card (you don't need a special driver despite the nvidia documentation, I got mine here: https://www.nvidia.com/Download/index.aspx https://www.nvidia.com/Download/index.aspx).
* In general you should ignore the nvidia documentation as they make it almost impossible to figure out what combination of cuda & cudnn versions you need.
Steps to install:
1. Open CMD as administrator.
2. Run: wsl --install -d ubuntu
3. Open the Ubuntu WSL and enter the following commands:
# Install CUDA
sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub http://developer.download.nvidia.com/compute/cuda/repos/ubun...
sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 http://developer.download.nvidia.com/compute/cuda/repos/ubun... /" > /etc/apt/sources.list.d/cuda.list'
sudo apt-get update
sudo apt-get --yes install cuda-toolkit-11-2 cuda-toolkit-11-2
sudo sh -c 'echo "deb http://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 http://developer.download.nvidia.com/compute/machine-learnin... /" > /etc/apt/sources.list.d/nvidia-machine-learning.list'
sudo apt-get update
sudo apt install cuda-11-2 libcudnn8 libcudnn8-dev
sudo apt install libnvinfer8 libnvinfer-dev libnvinfer-plugin8
# Verify CUDA installation
cd /usr/local/cuda-11.2/samples/4_Finance/BlackScholes
sudo make
./BlackScholes
# Install TensorFlow
sudo apt-get update && sudo apt-get install python3-pip python3-dev
pip3 install tensorflow-gpu
If you are reading this in the future I have some advice:
* Look at this page to get at least an idea of what versions of cuda and cudnn will work together: https://www.tensorflow.org/install/source#gpu https://www.tensorflow.org/install/source#gpu
* Use apt-cache search <THING> to see the possible versions, allowing you to guess what you need to install. Good luck.
- DarkmSparks 5y agoWhy not just install any linux desktop + nvidia drivers? Thats what all the historic stuff was written on. Also, much better to go with opencl than cuda for new developments these days (unless you are being funded in one of the nvidia startup programs)
- tester756 5y agoDoes it mean that I can play games inside WSL so I don't have to install them on host?
- peatmonster 5y agoI haven't tested this but in theory yes. This setup gives you working GPU support.
- fartcannon 5y agoThe man hours wasted just to perpatuate the entrenched old money empire of computer software boggles the mind. We are prisoners to this nonsense. It's so hard to watch.
- supernovae 5y agosupposedly the right version of windows10 is hitting RTM builds here soon and no longer limited to dev channels.
- teruakohatu 5y agoWhy does this not work on Windows 11?
- peatmonster 5y agoCorrection: I assume it does but I didn't want to install Windows 11, nor do I meet the requirements.
- slenk 5y agoAre you saying you tried or just based off what he said about using Windows 10? I would assume it would work on 11
- peatmonster 5y ago* I have not tried Windows 11, I assume it works. * The minimum version that works is Windows 10 21H2. * Some sources say you need a newer build, they are wrong, my OS build: 19044.1469
- slenk 5y agoAny reason to use the older LTS and not 20.04?
- peatmonster 5y agoThis was all done using the default WSL Ubuntu: Ubuntu 20.04 LTS
- slenk 5y agoI see you are using the 18.04 repos when 20.04 repos are available?
- peatmonster 5y agoThis is fixed in the updated version: https://gist.github.com/peatmonster/d86306009b7adfae717c10e1724dac81 https://gist.github.com/peatmonster/d86306009b7adfae717c10e1... I cannot edit my HN post anymore.
- froh 5y agoThen why did you use 1804 repositories from Nvidia?
- tda 5y agowhich WSL version does this apply to, 1 or 2?
- peatmonster 5y ago2
- yjftsjthsd-h 5y agoIf I may make some minor cleanup/style suggestions? * You can do the key fetch, populate both apt sources.list.d files, then do a single `apt-get update && apt-get install --yes` rather than 3 updates and 4 installs. * You're using both `apt` and `apt-get` and only using `--yes` once (obviously fixed by taking the previous point, but while I'm mentioning things...) * If all but the last two steps use sudo, maybe consider just making the first step `sudo su` and then `exit` when you're done (very context dependent, but does let you skip needing `sh -c` to populate files) None of this is anything against you, just some possible improvements to have fewer steps and general stuff I'd mention in a normal code review. And full marks for, y'know, actually documenting it at all and sharing:)
- peatmonster 5y agoThese are all good suggestions in the cold hard light of day :) but I unfortunately can't edit my post.
- dang 5y agoI'll reopen it for editing. If you have any trouble email hn@ycombinator.com.
- capableweb 5y agoIt'd be great if the lines were prefixed with four spaces (" ") so the URLs don't get shorted as well.
- lmeyerov 5y agoDid this a few weeks ago as part of the ~annual check to see if we finally support non-linux local dev for graphistry folks. Super promising, some tweaks: - WSL2. Kernel virtualization feels as native as advertised, e.g., docker in wsl2/ubuntu is fast like docker in native ubuntu, vs. the slowness that is docker in os x - Ubuntu 20.04 works, if I remember right - We don't worry as much about library-level versions b/c we run via docker - OpenCL does not work. There's some bits about intel & amd GPUs, but nada on nvidia - Many nvidia-smi diagnostics do not work... and many ecosystem libraries assumed they did for stuff like initialization & memory management & monitoring. Prepare for whackamole of updating pydata etc. dependencies to ~December+ - TBD for GPU K8S; KinD/K3D seem likely via docker runtime passthrough, but minikube & friends seem riskier. Curious if anyone has that working in a non-painful setup, esp. w/ cross-container GPU sharing.. We still can't fully use+endorse WSL2 support for Nvidia hardware due to lack of even super minimal OpenCL versions, and risky core diagnostics surprises, but encouraging! (Good time to shout: if you're curious about end-to-end client/server GPU computing and already active on opengl/webgl, we're looking for someone into helping the next 100X of our viz engine!)
- my123 5y ago> OpenCL does not work. There's some bits about intel [& amd] GPUs, but nada on nvidia This is planned to be fixed. WSL2’s GPU support currently doesn’t provide a path to provide an alternate OpenCL loader ICD location. (And not for OpenGL either) Microsoft is developing CLon12 at https://github.com/microsoft/OpenCLOn12 https://github.com/microsoft/OpenCLOn12 as the option to provide OpenCL out of the box.
- happycube 5y agoHuh... bit of a way off, but that could lead to a rather convoluted way to implement OpenCL on Vulkan (CLon12->DXVK)
- my123 5y agoD3D12 has more compute features than Vulkan has. It works out for DXVK because games often don’t use those, but it’ll cause much more issues with CLon12. By the way, if you are ready to have a _limited_ implementation without a full feature set because of Vulkan API limitations, clvk is a thing. The list of limitations of that approach is at https://github.com/google/clspv/blob/master/docs/OpenCLCOnVulkan.md https://github.com/google/clspv/blob/master/docs/OpenCLCOnVu... tldr: Vulkan and OpenCL SPIR-V dialects are different, and the former has significant limitations affecting this use case
- omegalulw 5y agoI don't get why you won't just use Ubuntu or dual boot. Why WSL?
- peatmonster 5y agoOP here. Up until this point I've completely avoided Windows for work (for obvious reasons). I've exclusively used a Linux laptop for work. I use Windows on my desktop as my home machine. WSL means I can train machine learning models in my time off, without stopping me chilling out with my buddies.
- lostmsu 5y agoI have an opposite question: why not just use Windows?
- nikau 5y agoWhat if you need to run word or Visio or a million other windows only apps?
- ChuckNorris89 5y agoOr what if you prefer the Windows desktop vs the Linux alternative, or if your laptop has some obscure driver issues on Linux that are making you miserable, or what if your org only issues Windows machines with the infra built around the Microsoft ecosystem (SharePoint, Office,etc) or a million other reasons why people use Windows. Why do some people need to be so obnoxiously judgemental about other people's choice of OS?
- dTal 5y agoBecause it's adversarial. Your choice to use one OS makes it more difficult, in a tiny way, for me to use a different OS. You will write software for that OS. You will convince others to use that OS. You will tell your boss to get computers with that OS installed, and then other people at your job will have to deal with it. And every developer on that OS is one that isn't using, writing software, fixing problems etc on my OS. It's political. It's zero-sum. And Microsoft knows that all too well, and that is precisely the reason why WSL exists.
- low_tech_love 5y agoBeing using it for a while and I’m extremely satisfied. Having an Ubuntu terminal on Windows with native performance made me uninstall a bunch of windows stuff and now I’m back to my unix working routine. Couldn’t be happier to be honest.
- erikbye 5y agoWhat did you uninstall?
- low_tech_love 5y agoNode, python, git, etc Basically everything that I can use on the wsl command line.
- apfsx 5y agoMight want to add this, I updated to 21H2 this week by downloading a package from Microsoft. https://support.microsoft.com/en-us/topic/kb5003791-update-to-windows-10-version-21h2-by-using-an-enablement-package-8bc077be-18d7-4aac-81ce-6f6dad2cd384 https://support.microsoft.com/en-us/topic/kb5003791-update-t...
- aspaceman 5y agoYeah I dug into the update catalog and downloaded the update that way myself. So there's a few ways to get it. Folks above mentioned the Update Assistant giving it if the Windows Update screen doesn't.
- ngcc_hk 5y agoSeem great why undervote it?
- codingdave 5y ago> 21H2 cannot currently be updated to FWIW, I just checked on my system, and it totally had the option to download and update to 21H2. Which I'm currently running. So it is available to some systems.
- black3r 5y agoyep my system also has that option, although my laptop doesn't officially support Windows 11 so maybe there's that. I still don't understand Microsoft's arbitrary decision to not officially support my CPU (even though some Kaby Lake CPUs are supported and newer slower Pentiums are also supported smh)
- supernovae 5y agoIt’s because of security. If you don’t have the right chipset windows 11 is drastically slower than windows 10 on same hardware. i’m kind of glad they kept the security in place for once rather than backwards compatible at all cost you can force windows 11 if want to deal with the penalty. it will just warn you it doesn’t pass performance baseline - there are videos and websites that explain how to force it and ignore compatibility checks. just search for bypass system requirements windows 11
- virgulino 5y agoMy Windows 10 upgraded to 21H2 on November 19.
- zeusk 5y agoThey've been doing rolling updates since quite a bit now - every new major update is watched for things like badly behaving drivers and hardware, so instead of 100% population being affected it is more like 5% affected when they pause the rollout.
- easton 5y agoIf you run the Windows 10 Upgrade Assistant it will bring you to 21H2 even if Windows Update isn't offering it to you: https://www.microsoft.com/software-download/windows10 https://www.microsoft.com/software-download/windows10
- forgingahead 5y agoNice, thanks for sharing. Can you make a gist as well of this? I think Google and other search engines index public gists better than they do HN posts, will def help those searching around for a solution.
- peatmonster 5y agohttps://gist.github.com/peatmonster/d86306009b7adfae717c10e1724dac81 https://gist.github.com/peatmonster/d86306009b7adfae717c10e1...
- NGRhodes 5y agoWorth noting there are many different ways to get CUDA with WSL2 working. If you follow the MS docs, they do not mention anything about install CUDA on the Linux VM, you do not get a functional setup to start using CUDA and if you follow Nvidia's documentation, they push you down the docker container route. I personally prefer to setup Tensorflow using Anaconda rather than pip. A slight correct on the OP, the special driver NVIDIA mentions is for CUDA only (no display driver included), Nvidia does not "support" installing of their full display driver in WSL2 so YMMV in terms of reliability and performance. I personally gave up on WSL2, not because of WSL2, but because it is so hard to use remotely (I do not have frequent local access to my GPU workstation). I tried various instructions and WSL2 itself was fine when used locally (via remote desktop), but networking scripts I found were unreliable, or if I used Windows openssh server it kept crashing and bash.exe is too limited to be useful, just not worth the hassle and effort when I can install Linux native.
- vl 5y agoI used WSL2 with CUDA from linux for awhile, but forced update to Windows 11 (which broke everything) was a last straw for me. I went back to dual boot configuration with Ubuntu 20.04 LTS, and it saves so much time on fiddling with wsl, I highly recommend it.
- peatmonster 5y agoGetting a working Windows 10 install without it being liable to break itself is such an involved process..
- supernovae 5y agoi never have any problems.
- vijaybritto 5y agoAbsolutely never had a single blue screen or random crash. Maybe its because its a laptop and not a hand built rig. Whats your's?
- 5y ago
- Rucadi 5y agoThere is no need to do the apt-key etc... the toolkit is already in the packages in newer versions of ubuntu. Install the cuda driver: https://developer.nvidia.com/cuda/wsl https://developer.nvidia.com/cuda/wsl Install an ubuntu image using wsld, (wsld -i ubuntu -d cuda -o D:\.wsl2) apt update && apt install nvidia-cuda-toolkit Done. For installing cudnn maybe is better to go to the installation guide: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html https://docs.nvidia.com/deeplearning/cudnn/install-guide/ind...
- jcagalawan 5y agoThe toolkit in the packages by default is old though. I was setting up JAX on a Ubuntu 20.04 WSL 2 install and it was complaining about ptxas being old and giving bad results.
- ncmncm 5y agoWe are not better off with CUDA working in My WSL. That keeps us confined to NVIDIA. It suffices to have full Vulkan support. Then we can use Kompute, which is portable to any GPU, not just NVIDIA.
- lostmsu 5y agoFrom the tutorial it is unclear what Kompute adds over raw Vulkan apart from Tensor type.
- ncmncm 5y agoIts value is not so much in what it adds, but in what it subtracts: 2000 lines of boilerplate.
- NGRhodes 5y agoWhy Kompute and not OpenCL ?
- ncmncm 5y agoPowerful forces drive Vulkan implementation. OpenCL is always an afterthought, where it even happens at all.
- jonpalmisc 5y agoI don’t use Windows anymore so I don’t really have any investment in WSL, but I’m happy to keep seeing things in the WSL camp get even better. I’d argue WSL is the single best Windows innovation in years.
- alerighi 5y agoStill WSL2 lacks of two fundamental things: the lack of bridge networking (being able to connect to a server running in the WSL from another host in the LAN) and the lack of sharing serial ports with the host (better would be USB devices, but at least WSL1 did share serial ports and since most microcontroller programs and debug with serial port it was sufficient).
- _xerces_ 5y agoYou can work around the serial port issue using something like Comm Tunnel: https://www.serialporttool.com/GK/comm-tunnel/ https://www.serialporttool.com/GK/comm-tunnel/ Open Comm Tunnel and setup for your com port (from device manager) and port 2101 Then in WSL2 use socat to connect: socat -d -d tcp-connect:172.18.96.1:2101 pty,raw,echo=0 It will output something like: >socat[18786] N PTY is /dev/pts/2 Then from within WSL2 you know to use '/dev/pts/2' as the COM port for serial and it will connect to the COM port. It works most of the time for me, but Comm Port seems lag if the throughput gets too much for it to handle.
- dTal 5y agoI, on the other hand, wish it to be as painful and frustrating to use as possible, so that developers are pushed into using (and therefore by extension maintaining and improving) the real thing. I do not wish Microsoft or their "embrace and extend" strategy well. If Microsoft really "loved Linux" they'd release their own version of Wine. Let us use their programs on our OS, instead of vice versa! Never forget that this relationship is fundamentally adversarial.
- dgellow 5y ago- WSL2 - modern Windows Terminal - winget Things have become really cool for devs in the Windows world, I almost don’t have complaints these days :) (other than GUI related stuff and that Windows Insider is really unstable, but I guess that’s on me for using the dev channel)
- Kwpolska 5y ago> * You MUST be on Windows 10 21H2 or above. 21H2 cannot currently be updated to (yeah, it's because they are pushing Windows 11) so you need to download it as an ISO. You can also force an upgrade using Group Policy: https://www.tenforums.com/tutorials/159624-how-specify-target-feature-update-version-windows-10-a.html https://www.tenforums.com/tutorials/159624-how-specify-targe... (this doesn’t work on the Home edition though)
- sbarre 5y agoI feel like this isn't true. 21H2 didn't automatically update on my system but it was clearly available to me in Windows Update and I just clicked to update to it last week, and it did the update. I'm on Windows 10 Home.
- nopenopenopeno 5y agoWSL is so much extra work to maintain. I absolutely love running Ubuntu LTS as my daily OS. I use Wine to run Photoshop CC 2018 and Illustrator CC 2019, Nvidia drivers work excellently on Linux these days, and all my development tools just work. I dread the day I have to depend on anything else.
- peatmonster 5y agoGlad to hear that, the last time I tried running my GTX 1070 on Ubuntu the drivers weren't so great.
- kuboble 5y agoSo much extra work as compared with what? In my experience WSL is a godsend for those who have to work on windows and like to integrate Linux tools in daily work. Before there were things like cygwin, some ported tools, or Linux in vm. All limited and/or major pain to work with.
- pletnes 5y agoNo systemd/cron/…, one extra OS to patch/upgrade, slooow perf on NTFS, can’t ssh into it (trivially), the list goes on. It has its uses but linux has many other uses that it doesn’t reliably cover.
- supernovae 5y agoservice sshd start
- ninkendo 5y agoWSL(2) has no bridge networking, so you need to do port forwarding in windows (and enable the port through windows firewall) to make this work. It’s not super hard, but not as trivial as it could be if there were tighter integration.
- 5y ago