3 ms·
The problem I'd see is that the raw randomness of electronics is not the nice, uniform distribution that indeed may help NNs learns. It can be, often is, biased
by joe_the_user 5y ago
The problem I'd see is that the raw randomness of electronics is not the nice, uniform distribution that indeed may help NNs learns. It can be, often is, biased, skewed, time-varying and just about anything else. It could vary from one part of the chip to another, one chip to another and so-forth. The various circuits exists to turn a wide range of noise into digital ons and offs. Which means the raw circuits can have this wide range unless controlled.
It's worth considering that no one uses raw electronic noise for random/pseudo-random number generators. And that's an application that's in demand.
- ttqidebg 5y agoDropout, the example given, is not a normally distributed error - more like a salt noise. The impact on subsequent layers isn’t Normal either. My point is certain error modes other than normally distributed may not be so bad. E.g. skew as roughly half of the redundant paths for information will be inverted in the latent space. I’m not a hardware guy, but in my experience DNNs can be made rather resilient. To me it looks like it could work. If it does great.