4 ms·
Does a GPU run a pruned network slower than unpruned, or just not as faster as the modern shrinkages would suggest?
by lonelappde 6y ago
Does a GPU run a pruned network slower than unpruned, or just not as faster as the modern shrinkages would suggest?
- fxtentacle 6y agoA naive GPU implementation on a non-block sparse network will be just as slow as the full un-pruned network.
- visarga 6y agoAnother approach would be to lower precision. By going from 32bits to 16 or 8 you can get massive speedups in memory operations and computation.
- fxtentacle 6y agoYes, that'll work. But it's orthogonal to this paper.
- yalok 6y agoThe trick is to combine these 2 approaches. Quantizing to bytes helps a lot, but forces you to use much larger pruning group size to see any speed up from pruning.
- sk0g 6y agoNvidia's apex.amp does just this - attempts half-precision training where possible, and adjusts scaling factors during training. I achieved a very significant speedup using this on my RTX 2080 Super, though it doesn't support Windows that well.