ttqidebg

3 ms·

1. 1.▲
   by ttqidebg 5y ago

   Dropout, the example given, is not a normally distributed error - more like a salt noise. The impact on subsequent layers isn’t Normal either. My point is certain error modes other than normally distributed may not be so bad. E.g. skew as r
2. 2.▲
   by ttqidebg 5y ago

   It is counter intuitive. Noise is often an essential part to training, especially when it comes to generalization. Sometimes there is enough noise in the training data to be sufficient. Controlling how much noise and where it goes so that y
3. 3.▲
   by ttqidebg 5y ago

   Sure, but it’s effectively random which way the model learns the parameter impact and how values are used by subsequent layers. I’m assuming layers in the NN. Adding noise with things like dropout encourages the model to ‘grow’ redundancy.
4. 4.▲
   by ttqidebg 5y ago

   Using more parameters you can average out the errors and recover some accuracy.