When you compile CUDA code, it has an architecture flag, and you can specify compiling for multiple generations of the code. They’re to some degree backwards co

When you compile CUDA code, it has an architecture flag, and you can specify compiling for multiple generations of the code. They’re to some degree backwards compatible (you could run a sm35 package on an sm70 compatible GPU) but performance is much better if you compile appropriate for the GPU you have because it can take advantage of newer hardware features. There’s now 7 or 8 widely in use GPU architectures with only a few early ones so far unsupported by the compiler (though a few more are now deprecated). Just as an example - in my personal PC I have a 750Ti, a card released in 2014, and the CUDA compiler will only drop support for that in the next version. This means that you end up with multiple libraries for every package. On top of that, people want to install their package with PyPi for e.g. and go - but need the entire CUDA runtime libraries - which include cuFFT, cuBLAS, cuDNN, etc etc. So if every package includes 7 or 8 arch specific versions of the compiled code and the full CUDA RT distribution, it’s easy to see how sizes get big. The interesting thing is that exactly the same problem occurs on CPUs anyway. We have various optimisation levels in compilers for each family of processor, but aside from MKL I don’t know of any packages that are wheels that try to provide an optimised version of the code across CPU generations. This means anyone who wants good performance should always try to build from source rather than install the wheel.