3 ms·
Sure it applies, but it's even less likely than typing in one of Google's 4096-bit RSA key by chance.
by pas 6y ago
Sure it applies, but it's even less likely than typing in one of Google's 4096-bit RSA key by chance.
- mensetmanusman 6y agoSo the lottery is so unlikely it is not worth thinking about? Are there bounds on the possibilities? Can’t you just check a lot of values and hope for the best?
- pas 6y agoUmm, on the contrary. It's worth thinking about, but not to directly solve it, but to help the training process converge faster. So if we can increase the chances by many orders of magnitude, then it'll help a lot with training. Basically it's still an open question of exactly how and why the training process can get stuck. It's almost as if the network gets sufficiently strong ideas (Bayesian priors) it starts to discard new evidence (the error signal doesn't help). That's probably why a maximum entropy (random) state is useful to "bootstrap" those efficient subnetworks that then learn fast.