The goal of the post is to compress an already trained network. An more ambitious related goal is to perform the training of the network with at most 10% (or o

The goal of the post is to compress an already trained network. An more ambitious related goal is to perform the training of the network with at most 10% (or other small percentage) of the network weights being nonzero at any time during training. If possible without a great loss in accuracy, this could be particularly useful to save memory during training. The paper https://arxiv.org/abs/1711.05136 https://arxiv.org/abs/1711.05136 propose a method to achieve this.