3 ms·
I've written a lot of software against GPUs, albiet some years back. The main challenge was that many of the best libraries had CUDA support (or CuDNN support o
by TuringNYC 5y ago
I've written a lot of software against GPUs, albiet some years back. The main challenge was that many of the best libraries had CUDA support (or CuDNN support on top) but not support for other GPU lines' frameworks.
Getting CUDA to work well is hard. Not hard on your laptop, not hard on a particular machine but hard to work everywhere when you dont know the target environment beforehand -- there are different OSs, different OS versions, different versions of CUDA, different cards with different capabilities. But we did get it to work fairly widely across client machines.
The same effort needs to be put into getting things to work for other manufacturers, except a layer deeper since now you're not even standardized on CUDA. Many companies just dont make the investment. Our startup didn't, because we wouldn't find people who could make it work cost effectively.
What I really wish is that the other manufacturers would themselves test popular frameworks against a matrix of cards under different operating systems and versions. We see some of that, for example, with the effort of getting TensorFlow to run on Apple's m1 and metal. I just dont see a random startup (e.g., mine w/ 12 employees) being able to achieve this.
For example, if I know from the manufacturer that I could get TensorFlow X to work on GPU Y on {Centos N, Ubuntu 18/20}, I would gladly expand support to those GPUs. But sometimes you dont know if it is even possible and you spin your wheels for days or weeks -- and if the marketshare for the card is limited, the business justification for the effort is hard to make. The manufacturers can address this issue.
- hhh 5y agoWhy not use the Nvidia docker runtime and put your application in a container?
- HelloNurse 5y agoCan you download a RTX 3080 from the Docker registry?
- analognoise 5y agoThe hardware is a hell of a lot cheaper than wasting a week of engineering time though isn't it?
- TuringNYC 5y agoNvidia docker runtime has been more recent. There were some issues with k8s usage and Nvidia docker runtime -- you couldnt use a non-integer number of allocations (e.g., cant split allocation of a GPU). That said, NVIDIA Docker Runtime is awesome now -- however, all this underscores further how much further behind the non-NVIDIA stack is!
- croes 5y agoWhat about OpenCL?
- shakna 5y agoOpenCL feels pretty stagnant. Showstopping bugs staying open for years. Card support is incredibly spotty. Feature support isn't even near parity with CUDA. This despite v3.0 being released just last year... And completely breaking the API.
- d110af5ccf 5y agoOpenCL certainly has the potential to be a universal API but support for it is surprisingly spotty given its age. For proprietary implementations, Intel appears to have the broadest and most consistent support. Nvidia skipped OpenCL 2.x for some technical reason (IIUC). AMD is a complete mess, for some reason not bothering (!!!) to roll out ROCm support for their two most recent generations of consumer GPUs. In open source "Linux only" land, Mesa mostly supports OpenCL 1.2 (https://mesamatrix.net/#OpenCL https://mesamatrix.net/#OpenCL) at this point. So if you're targeting Linux specifically then that's something at least. Good luck shipping an actual product using OpenCL that will "just work" across a wide variety of hardware and driver versions. POCL and CLVK are both experimental but might manage this "some day". In the mean time, resign yourself to writing Vulkan compute shaders. (Then realize that even those will only run on Apple devices via MoltenVK, and despair at the state of GPGPU standardization efforts.)
- MillenialMan 5y agoHow viable is it to replace CUDA with compute shaders?
- maeln 5y agoTechnically it is entirely viable. Vulkan/OpenGL compute shaders offer more or less a 1:1 equivalent to every CUDA features. It is more of an usability issue. CUDA has been design to be a GPGPU API from the get go and, therefor, tend to be "easier" to use. OpenCL could have been a better replacement, but the API was really not on par with CUDA when it comes to usability. SYCL looks like finally a good answer by the Khronos group but it is so late. You already have a lot of people who know how to use CUDA, a lot of learning resources, etc ...
- TuringNYC 5y agoMany organizations writing GPU-compliant software are not actually "writing CUDA" but they are either using key libraries which area using CUDA (e.g., TensorFlow) or it is a layer deeper (e.g., I use a deep learning library, the deep learning library using CuDNN, CuDNN uses CUDA.) Other orgs are using something written in another language that compiles into CUDA. Either way, to replace CUDA, that middle component needs to be replaced by someone and ideally it should be the card manufacturers themselves (IMHO.) I cant imagine any small/medium organization having sufficient engineering time to write the middle component and keep them up to date with the slew of new GPUs, OS updates, or new GPU features -- unless it is their core business.