Fun tiny fact I found a while ago - The reverse of this also works! You can init deeper layers by copying the weights from previous layers. Not a big improveme

Fun tiny fact I found a while ago - The reverse of this also works! You can init deeper layers by copying the weights from previous layers. Not a big improvement in training time, but it does reduce the initial loss!