3 ms·
How is it different than pruning a neural network? It seems you could train the weights of a state of the art NN, then quantizite it, then prune it. It will re
by antpls 7y ago
How is it different than pruning a neural network?
It seems you could train the weights of a state of the art NN, then quantizite it, then prune it. It will remove some weights of the NN, then all the remaining weights are set to the same value. Isn't training then pruning more efficient than using an architecture search algorithm ?
- drewm1980 7y agoAt the risk of broad oversimplification, pruning trains and then does an architecture search. This does an architecture search and then trains.
- deleted 7y ago[deleted]
- p1esk 7y agoNo, here the architecture search is the training.
- p1esk 7y agothen all the remaining weights are set to the same value During quantization, weights are set to different values, in the extreme case just 2 different values (binarization). In this case, they are using multiple activation functions to provide various paths for signals to be modified (effectively playing the same role as weights).