In most cases, there is limited support for sparse operations. "Sparse Networks from Scratch: Faster Training without Losing Performance" https://arxiv.org/abs/

In most cases, there is limited support for sparse operations. "Sparse Networks from Scratch: Faster Training without Losing Performance" https://arxiv.org/abs/1907.04840 https://arxiv.org/abs/1907.04840 openly says "Currently, no GPU accelerated libraries that utilize sparse tensors exist, and as such we use masked weights to simulate sparse neural networks.". However, the situation seems to be very dynamic. See: - https://github.com/StanfordVL/MinkowskiEngine https://github.com/StanfordVL/MinkowskiEngine (Minkowski Engine is an auto-diff convolutional neural network library for high-dimensional sparse tensors) - https://github.com/rusty1s/pytorch_sparse https://github.com/rusty1s/pytorch_sparse (an efficient sparse tensor implementation for PyTorch; the official one is slower SciPy https://github.com/pytorch/pytorch/issues/16187; https://github.com/pytorch/pytorch/issues/16187; however, I failed to install it - it is not "pip install"-simple) EDIT: I checked it now and was able to install pytorch_sparse with one command. It is a dynamic field indeed.