Thanks! Also that code seems pretty straightforward. Later there's this bit: conv_layer.weight.data = (eigenvectors/torch.sqrt(eigenvalues+eps))[-shape[0]:

Thanks! Also that code seems pretty straightforward. Later there's this bit: conv_layer.weight.data = (eigenvectors/torch.sqrt(eigenvalues+eps))[-shape[0]:, :, :, :] ## We don't want to train this, since this is implicitly whitening over the whole dataset Oh it looks like you then just "remove" the correlated data.. Actually, you're just pre-"padding" the basic (gaussian?) covariances. Ah so then you'd skip training basic statistical information that's already easy to calculate directly. Clever!