3 ms·
It is counter intuitive. Noise is often an essential part to training, especially when it comes to generalization. Sometimes there is enough noise in the traini
by ttqidebg 5y ago
It is counter intuitive. Noise is often an essential part to training, especially when it comes to generalization. Sometimes there is enough noise in the training data to be sufficient.
Controlling how much noise and where it goes so that you end up with something usable is still a bit of an art. Not just in designing the model but also the training and distillation process. It’s a pain in the ass, but if the perf is there it can be worth it.
I’m inference speed bottlenecked so I’m keeping an eye on this sort of tech.
- dragontamer 5y agoI've done enough neural net stuff to know that noise can knock you out of local minima / local maxima. Well, not even neural nets... machine learning in general benefits. But there's a difference between noise that you can control (ie: on during training and you can increase/decrease at will) and noise that's fundamental to the system and uncontrollable. Still, since all neural nets show a resilience to a degree of noise, maybe that makes analog-circuits suddenly viable.