4 ms·

> building a GPU-capable BLAS library is just a lot more work than one would think I made a few pieces when needed: dense and sparse matrices, conjugate gradie

> building a GPU-capable BLAS library is just a lot more work than one would think I made a few pieces when needed: dense and sparse matrices, conjugate gradient, householder QR decomposition, other things. Wasn’t terribly hard. I only needed to support relatively new GPUs, that’s one reason why. Another one, I based on DirectCompute not ROCm. The thing’s well tested (modern videogames are using compute shaders a lot), and works with all 3 vendors of GPUs. The main downside it’s Windows only. > NVIDIA has a GUI installer for CUDA. It’s still more friction compared to D3D runtime, which is an essential OS component installed and updated as a part of the OS.

fxtentacle 4y ago
DirectCompute is only the execution of fully trained neural networks, so they can skip all the gradient calculations, which are the compute-heavy and memory-heavy part. To give you and idea, I need 24GB of GPU RAM to build a background separation model, but once it's finished, it's only 500kb of parameters and it can run on CPU in JavaScript in realtime. As for the friction, CUDA is easy enough for motivated gaming kids to handle. AMD is not. But I'd predict that today's GPU enthusiast kids will become tomorrow's AI experts. As an example, look at the LAION chat which is a weird mixture of childish humor, gaming memes, and discussions about analytical approximations for probabilistic differential equations. (the Elucidated paper applied to Imagen's DDPM)
- Const-me 4y ago
  DirectCompute is pretty close to CUDA. When handled with care, it can do these compute-heavy pieces with similar performance. The main downside is the libraries: cuBLAS, cuFFT, and similar pieces of CUDA are manually optimized, and nVidia ships multiple versions of these compute kernels for different GPUs. DirectCompute doesn’t have an equivalent, need to implement manually in HLSL. This can be tricky because for optimal performance different chips need slightly different implementations.

fxtentacle 4y ago

DirectCompute is only the execution of fully trained neural networks, so they can skip all the gradient calculations, which are the compute-heavy and memory-heavy part. To give you and idea, I need 24GB of GPU RAM to build a background separation model, but once it's finished, it's only 500kb of parameters and it can run on CPU in JavaScript in realtime. As for the friction, CUDA is easy enough for motivated gaming kids to handle. AMD is not. But I'd predict that today's GPU enthusiast kids will become tomorrow's AI experts. As an example, look at the LAION chat which is a weird mixture of childish humor, gaming memes, and discussions about analytical approximations for probabilistic differential equations. (the Elucidated paper applied to Imagen's DDPM)

Const-me 4y ago
DirectCompute is pretty close to CUDA. When handled with care, it can do these compute-heavy pieces with similar performance. The main downside is the libraries: cuBLAS, cuFFT, and similar pieces of CUDA are manually optimized, and nVidia ships multiple versions of these compute kernels for different GPUs. DirectCompute doesn’t have an equivalent, need to implement manually in HLSL. This can be tricky because for optimal performance different chips need slightly different implementations.