The mistake you're making is thinking of weights as an avenue to store training data. This is wrong. I've hopefully explained it more in another comment but wei

The mistake you're making is thinking of weights as an avenue to store training data. This is wrong. I've hopefully explained it more in another comment but weights/neurons are essentially configuration settings.