2 ms·
Doesn't reducing the number of neurons drastically reduce memory requirements?
by joelthelion 3y ago
Doesn't reducing the number of neurons drastically reduce memory requirements?
- lopuhin 3y agoYes it might. "Reduction of number of neurons" is not static here, unlike traditional pruning approaches, here they still keep all weights, but the network dynamically selects which sub-portion of them to use. There is a related discussion of this in section 3.2 (page 4), but they don't think they mention actual memory bandwidth requirements/wins of their implementation, and probably there can be different tradeoffs for different devices.