For those curious, NVidia and Cerebras have been doing R&D in sparse neural nets for something like a decade. NVidia began adding hardware support for them seve

For those curious, NVidia and Cerebras have been doing R&D in sparse neural nets for something like a decade. NVidia began adding hardware support for them several generations ago (Ampere). It is significantly more complex than it appears at first sight.