3 ms·
This makes me wonder what would happen if you took the sparse model, reset its weights, then trained it with the original training data.
by amelius 4y ago
This makes me wonder what would happen if you took the sparse model, reset its weights, then trained it with the original training data.
- sva_ 4y agoVery interesting idea. I'd hypothesize that it won't achieve the same(ish) accuracy, and that pruning might be required (similar to how humans go through a heavy pruning phase at an early age[0]). Would be worth setting up an experiment on a smaller scale. As some other commentator stated, there's currently a lot of low hanging fruit in optimizing NN. 0. https://en.m.wikipedia.org/wiki/Synaptic_pruning https://en.m.wikipedia.org/wiki/Synaptic_pruning