4 ms·
I see they compare against various pruning techniques in related work, but do they compare accuracy and/or parameter reduction in their evaluation against lotte
by anonymousDan 5y ago
I see they compare against various pruning techniques in related work, but do they compare accuracy and/or parameter reduction in their evaluation against lottery ticket approaches (whatever the state of the art is there)?
- avidmoon 5y agoShort answer, as I'm on mobile: this is structured pruning, which is hardly comparable to unstructured methods like IMP (algorithm for lottery tickets)
- anonymousDan 5y agoThanks. Can you give an intuition as to the difference between 'structured' and 'unstructured' pruning? What are the pros and cons of each?
- ndronen 5y agoIn unstructured pruning, individual weights of a weight tensor are pruned with no constraints on their position in the tensor. In structured pruning, there are constraints on which weights are pruned. Think of pruning an entire output channels of a convolutional layer vs pruning arbitrary weights of the same tensor. Unstructured pruning allows for a greater percentage of the weights to be pruned, but unstructured pruning doesn't speed up a network on hardware accelerators, which favor dense matrix multiplications.