I would imagine that the weights of a finetuned model are highly correlated with the original weights. Having said that, simply permuting the neurons would make

I would imagine that the weights of a finetuned model are highly correlated with the original weights. Having said that, simply permuting the neurons would make it way harder to match them up, I can't think of a straightforward way to reverse it.