Neural networks (At least CNNs) have so many parameters, you can hold nearly all of them at there initial values! Checkout this paper if you are interested: ht

Neural networks (At least CNNs) have so many parameters, you can hold nearly all of them at there initial values! Checkout this paper if you are interested: https://arxiv.org/abs/1806.06949 https://arxiv.org/abs/1806.06949 Disclaimer: I am an author and that version is somewhat out of date.