That seems pretty unlikely to me actually. As the models and training data get much bigger, I think the initial weights become less important (at least assuming

That seems pretty unlikely to me actually. As the models and training data get much bigger, I think the initial weights become less important (at least assuming your random weights have certain desirable statistical properties, which they do by construction usually).