2 ms·
They aren't. As far as I can tell, Nvidia does this to be able to double the number of TFlops they put on their website. (this might be a little unfair, the rea
by adgjlsfhk1 4y ago
They aren't. As far as I can tell, Nvidia does this to be able to double the number of TFlops they put on their website. (this might be a little unfair, the real reason is that in ML it might be possible to train a NN such that your matrices have this structure, but I haven't seen anyone other than Nvidia use it)
- bee_rider 4y agoI’m trying to think of cases where it might accidentally come up, and all I can think of is something like “oops I used complex but my values are actually real.”
- Firadeoclus 4y agoWhat you might do is train using dense matrices, then sparsify those (pick the 2 out of each set of 4 weights that are closest to zero, mask them out), then do a few more training iterations with the mask in place. It turns out that even without the extra training iterations you often lose surprisingly little in terms of quality of output. In reality you can sparsify a lot more, but 2 out of 4 is so simple and easy to implement in hardware, more complex schemes are much harder to justify. However, small matmuls (say, <2048 bytes in the K dimension) won't get anywhere near 2x performance.