Indeed, but if performance is affected so much by the initialisation, then I would avoid random initialisation in the first place. There are various publication

Indeed, but if performance is affected so much by the initialisation, then I would avoid random initialisation in the first place. There are various publications exploring different initialisation methods for various problems. I'm afraid that I cannot go into more specific details right now, but you can get more stable training and faster convergence with a better initialisation strategy.