> Sparse operations still don’t have good support in any of the generic SIMD instructions (avx2/512, Neon), or in GPU… This is incorrect. For GPUs: Nvidia A1

> Sparse operations still don’t have good support in any of the generic SIMD instructions (avx2/512, Neon), or in GPU… This is incorrect. For GPUs: Nvidia A100 (launched in 2020) has sparse matrix multiplication support for inference. For CPUs: Check out SLIDE from 2019: https://arxiv.org/pdf/1903.03129.pdf https://arxiv.org/pdf/1903.03129.pdf