3 ms·
The main problem with SNNs is not that they don't work well. It's not that they are slow to run on CPUs/GPUs. Those would indeed be temporary problems. The fund
by p1esk 7y ago
The main problem with SNNs is not that they don't work well. It's not that they are slow to run on CPUs/GPUs. Those would indeed be temporary problems. The fundamental problem is we don't know the right abstraction level to imitate brain operations. It's far from clear that simulating spikes is necessary to implement brain's "learning algorithms". There are two main arguments for using spikes: "biological realism", and "energy efficiency". Neither is convincing: if you want to simulate a conventional CPU you don't want to simulate IV curves for transistors. You don't want to simulate CMOS gates. No, because the lowest abstraction level necessary to understand the computer operation is boolean logic. Anything below that is completely irrelevant. I strongly suspect that simulating spikes is below the lowest relevant abstraction level. By the way, Jeff Hawkins of Numenta agrees with me, and he's pretty strict about "biological plausibility" of his neocortex algorithms. As for energy efficiency - sure, spikes might be the most efficient way to compute and encode information given the constraints of biological brains. But why should we care about those constraints? We are building computing machinery in silicon, using vastly different technology, circuits, and tricks from what's available in "wetware". It does not make any sense to copy biological evolution tricks to improve efficiency when building things in silicon. Neuromorphic hardware papers always mention energy efficiency and then proceed to compare their analog spiking chips to digital (!) chips. That's ridiculous! I can't think of a single good argument why would spiking analog circuits be more energy efficient than non-spiking analog circuits, if we are talking about any existing computing hardware technology (or one that's likely to be developed in the foreseeable future).
Deep learning took off in 2012 not because faster hardware allowed us to develop good algorithms. The algorithms (gradient descent optimization, backpropagation, convolutional and recurrent layers) have been developed -- using slow hardware -- long before (Lecun demonstrated state of the art on MNIST back in 1998). The fast hardware allowed us to scale up the existing good algorithms. I don't see any good algorithms developed for SNNs. Perhaps these algorithms can be developed, perhaps faster hardware is indeed necessary, but as I argue above, the motivation to pursue this research is just not obvious to me.
Note that we shouldn't confuse this SNN research (such as the papers you cited), to efforts like Human Brain Project, where they're actually trying to derive higher level brain algorithms from accurate simulations of low level mechanics. Emphasis on accurate, because as any neuroscientist will tell you (e.g. [1]), these SNNs have very little to do with what's actually going on in a brain on any abstraction level.
[1] https://spectrum.ieee.org/tech-talk/semiconductors/devices/blue-brain-project-leader-angry-about-cat-brain https://spectrum.ieee.org/tech-talk/semiconductors/devices/b...