6 ms·
The authors inject artificial noise in cycles to their Spiking Neural Networks. They call this noise injection as Slow Wave Cycles equivalent to what brain gene
by seesawtron 6y ago
The authors inject artificial noise in cycles to their Spiking Neural Networks. They call this noise injection as Slow Wave Cycles equivalent to what brain generates during sleep. I do not know what exactly goes during SW sleep in mammalian brain so it's hard to judge if this analogy by the authors makes sense.
However, noise injection on its own has been extensively shown to decorrelate signal from noise so that the networks don't fixate on noise in the input data while training. This is widely used when using CNNs on image data and is known to increase performance of network predictions. So I am not sure how much different this article's findings are from this.
No link to arxiv publication makes it harder to make sense of these findings.
Edit: "Sleep" is not relevant for most ANN architectures used today. Synaptic pruning however is, similar to dropout.
- sidpatil 6y agoThis sounds just like dithering. https://en.wikipedia.org/wiki/Dither https://en.wikipedia.org/wiki/Dither
- seesawtron 6y agoCool, yes we also add similar random (Gaussian) noise to image data before the networks see it. The decorrelation theory comes originally form signal processing field like your example shows. Old wisdom in new bottle.