5 ms·
Spiking Neural Networks
- nynx 5y agoSource: I'm an undergrad doing research in neuromorphic computing. A lot of information about SNNs misses a lot of recent findings on the effect of astrocytes on the dynamical state of the network. If you look in the recent literature on SNNs, you'll find that including astrocyte models in the networks improves memory and accuracy significantly in many cases. It's too bad this article doesn't mention that.
- orbifold 5y agoOne of my first memorable PhD experiences was getting invited to an impromptu meeting in Lithuania, where we met a Finish and Lithuanian scientist both working on astrocyte models and wondering how that could be applicable to neuromorphic hardware. Needless to say this didn't go anywhere, but at least we got to sample some really nice Lithuanian food.
- mathematicaster 5y agogreat handle!
- klowrey 5y agoHave any links you would recommend?
- periheli0n 5y agoNeuromorphic hardware has been usable for 10 years now. Since then, algorithms for neuromorphic hardware (i.e. spiking networks) always performed 'almost as good' as ANN solutions on GPUs (meaning: inferior). But each year a new generation of GPUs comes out, using modern processes, with excellent toolchains. In a direct comparison of power efficiency, GPUs win over NMHW most of the time. I would love to see Spiking Networks and NMHW take over machine learning but it has such a long way to go. And I seriously doubt the strategy, followed by most players, to try to beat good old ANNs at their own game. Unless we identify a problem set where event-based computing with spikes is the inherently natural solution, I find it hard to imagine that spiking networks will ever outcompete ANN solutions.
- a-dub 5y ago> Unless we identify a problem set where event-based computing with spikes is the inherently natural solution, I find it hard to imagine that spiking networks will ever outcompete ANN solutions. i'd guess that domain would be real time (unbuffered / unbatched) processing of raw sensory data. it seems reasonable that biological neural systems evolved for optimal processing of sensory information encoded temporally in spike trains, yet the few papers on neuromorphic computing i've seen tend to try and hammer spiking neural networks into a classic batch based machine learning paradigm and then score them against batch based anns.
- periheli0n 5y agoThis very much so. On the other hand, even biology often uses rate codes which are inefficient and limited in what they can represent, compared to all those timing-sensitive codes, like latency codes, rank order codes, phase codes, pattern codes, population codes etc. And when we look at the technical domain, event-based vision cameras spew out what could pass as spikes; but even in that area spiking networks have proven too limited compared to event-based algorithms that were only vaguely bioinspired. And this technology took about 20 years from conception to making a breakthrough on the market. So the question is whether spiking networks are indeed the future of computation. But without doubt, the concept is very interesting academically. A bit like Haskell :D
- bollu 5y agoWhat are these different types of codes you mention? Got a textbook/lecture series to learn more? :)
- periheli0n 5y agoUm, Dayan & Abbott's "Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems" comes to mind. It's rather neuroscience heavy though. Then there is "Spikes: Exploring the neural code" https://mitpress.mit.edu/books/spikes https://mitpress.mit.edu/books/spikes This one is focusing a lot on information theory.
- cblconfederate 5y agoI don't get the appeal of spiking networks. They struggle to solve problems that have already readily been solved by ANNs and they don't offer much in terms of biological realism - they don't account for neuronal geometry and dendritic nonlinear phenomena, nor do they explain the protein dependence of LTP.
- periheli0n 5y agoThese are two disparate applications of spiking networks. 1. Machine learning—yes, many in the field seem to be trying to reinvent ANNs with spikes. Not very useful in my opinion. 2. Modeling biological processes. A lot of progress has been made in neuroscience research thanks to spiking network models, coupled with dendritic computation and all other sorts of biological detail. But one would not normally use neuromorphic hardware if the end goal is biological realism. Only if biological realism is required in real time and on a constrained power budget, such as on a robot, is Neuromorphic hardware the weapon of choice.
- glial 5y agoI can't speak for the ML applications. But to neuroscientists, I imagine the appeal is: 1. the brain uses spiking neurons 2. the brain does sophisticated computations, predictive coding, etc ...how? Like with other basic research, I think it's a safe to bet that there will be applications for working SNNs that you or I have not imagined yet.
- JackFr 5y agoMakes me think that we can send a man to the moon but we can’t come up with a practical aircraft powered by flapping wings. Evolved biological process can be very subtle. They’re worth studying for their own sake but not necessarily best to solve general problems.
- periheli0n 5y agoSpiking networks are to Machine Learning like Haskell to C++ and Python: While not really used to solve many real-world problems, they are extremely interesting academically, and important concepts have ended up in the mainstream, like event-based sensing and control, or event-driven signal processing.