If they are simply masking out low values weights after doing full training, they’ve basically recreated what I worked on for my graduate thesis. In contrast to

If they are simply masking out low values weights after doing full training, they’ve basically recreated what I worked on for my graduate thesis. In contrast to this work, what we worked on starts from iteration 0, and constantly adapts the lottery mask. You can reset weights back to their initial value, or set them back to their initial value tied to a decay term. This will zero initial values slowly to get computational sparsity if needed. https://arxiv.org/abs/1806.06949 https://arxiv.org/abs/1806.06949 https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/items/1.0371928 https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/...