3 ms·
From what little I've gathered about the subject, I think so. Spiking neural networks would be orders of magnitude more energy efficient than deep neural networ
by castle-bravo 9y ago
From what little I've gathered about the subject, I think so. Spiking neural networks would be orders of magnitude more energy efficient than deep neural networks for image and audio processing. The main hurdle lies in designing network structure and weights; the cost function of a classical neural network can be differentiated using basic calculus, but optimizing a spiking neural network is not as easy.
- AstralStorm 9y agoI think you can bridge the gap with a bit of discrete mathematics used to develop some forms of error correction. Worst case, quantum codes even.
- castle-bravo 9y agoCan you elaborate a bit? The connection between error correction codes and spiking neural networks isn't obvious to me.
- AstralStorm 9y agoA spiking neutral network can be considered a noisy real spiking channel transmitting contiguous real numbers. Similar on the surface to multibit sigma-delta modulation. There are many papers on this topic, e.g. https://papers.nips.cc/paper/4694-neuronal-spike-generation-mechanism-as-an-oversampling-noise-shaping-a-to-d-converter.pdf https://papers.nips.cc/paper/4694-neuronal-spike-generation-... Once you describe the specific modulation and noise properties you could recover the equivalent real valued continuous time neutral network from spiking neutral network. Then you can similarly map the training algorithms from gradient networks to "spike gradients". The next necessary step is to use (potentially quantum) interference in addition to integration or summation to train the interconnections. You will likely end up with quite complicated Hessians in both time and value.