4 ms·
I haven't heard of particularly compelling advantages for one pruning algorithm over another... So at this point, it seems best to just keep it dead simple. As
by sdenton4 7y ago
I haven't heard of particularly compelling advantages for one pruning algorithm over another... So at this point, it seems best to just keep it dead simple.
Assuming the lottery ticket hypothesis, you're just looking for the strong sub network, which should stand out under a number of approaches. The vestigial neurons shouldn't be doing much...
- AstralStorm 7y agoThey had to train the various pruned nets 15+ times each to know the cutoff to be a success... Also what is a strong subnetwork? What we get to know it's that their pruning algorithm produces a network that is vulnerable to weight randomization but better performing with original weights. If there is a better pruning algorithm, it could tell us more about structure of such subnetworks.