4 ms·
One of the attempts to bring ANNs closer to biology is Spiking Neural Networks, where they introduce the timing component, but so far they're harder and less ef
by spyder 5y ago
One of the attempts to bring ANNs closer to biology is Spiking Neural Networks, where they introduce the timing component, but so far they're harder and less efficient to train than the non-spiking networks but can have the similar accuracy.
https://en.wikipedia.org/wiki/Spiking_neural_network https://en.wikipedia.org/wiki/Spiking_neural_network
Article with code and some nice visualizations:
https://guillaume-chevalier.com/spiking-neural-network-snn-with-pytorch-where-backpropagation-engenders-stdp-hebbian-learning/ https://guillaume-chevalier.com/spiking-neural-network-snn-w...
- m0zg 5y agoNote that compared to real neurons this is very, very crude, and it does not replicate their functionality in full. The key thing to understand is that a real neuron is not a constant thing, it is subject to its environment, meaning that if it receives a lot of spikes, for example, it may become temporarily depleted, and that very depletion will play a role in how the bigger system works. It also does not have "time steps" - it processes the signal continuously. In general, as in any biological system, everything depends on everything else, often in circuitous, ambiguous, non-obvious ways, which makes things extremely difficult to model and/or understand. What this means from the research perspective: it's like that joke with a drunk searching for car keys under a streetlamp because it's easier to search there. People make progress where they can until someone unusual comes along and installs a new "streetlamp" elsewhere. 100% of research and engineering (and life in general) works this way, not just neuroscience. It's just that at the moment the street lamps in this field are few, dim, and very far between.