3 ms·
DirectCompute is only the execution of fully trained neural networks, so they can skip all the gradient calculations, which are the compute-heavy and memory-hea
by fxtentacle 4y ago
DirectCompute is only the execution of fully trained neural networks, so they can skip all the gradient calculations, which are the compute-heavy and memory-heavy part.
To give you and idea, I need 24GB of GPU RAM to build a background separation model, but once it's finished, it's only 500kb of parameters and it can run on CPU in JavaScript in realtime.
As for the friction, CUDA is easy enough for motivated gaming kids to handle. AMD is not. But I'd predict that today's GPU enthusiast kids will become tomorrow's AI experts. As an example, look at the LAION chat which is a weird mixture of childish humor, gaming memes, and discussions about analytical approximations for probabilistic differential equations. (the Elucidated paper applied to Imagen's DDPM)
- Const-me 4y agoDirectCompute is pretty close to CUDA. When handled with care, it can do these compute-heavy pieces with similar performance. The main downside is the libraries: cuBLAS, cuFFT, and similar pieces of CUDA are manually optimized, and nVidia ships multiple versions of these compute kernels for different GPUs. DirectCompute doesn’t have an equivalent, need to implement manually in HLSL. This can be tricky because for optimal performance different chips need slightly different implementations.