Broadly speaking the reason it's the whole "making sure you are using the correct version of python that uses the correct NVIDIA drivers via the correct Pytorch

Broadly speaking the reason it's the whole "making sure you are using the correct version of python that uses the correct NVIDIA drivers via the correct Pytorch installation" that causes the issues. This is actually a solved problem. But it's been solved lots of different times in different incompatible ways which tend to clash on any individual computer.