What do you gain by doing that? It isn't any cheaper to train with connections removed. It could really damage the training if a parameter gets stuck at 0 that

What do you gain by doing that? It isn't any cheaper to train with connections removed. It could really damage the training if a parameter gets stuck at 0 that shouldn't be. And the sparsity penalty has traditionally been considered to be enough.