> Their key innovation was the idea that connections that were pruned after the network was trained might never have been necessary at all. To test this hypothe

> Their key innovation was the idea that connections that were pruned after the network was trained might never have been necessary at all. To test this hypothesis, they tried training the exact same network again, but without the pruned connections. Importantly, they "reset" each connection to the weight it was assigned at the beginning of training. > “It was surprising to see that re-setting a well-performing network would often result in something better,” says Carbin. This, intuitively, makes sense to me. It seems that the pruned model has to waste less training cycles on inferior weights, and it can therefore spend more cycles on further optimizing the good weights.