4 ms·
> Biology has evolved to use spikes (across many different species). Those different species didn't independently arrive at their own neuronal implementation,
by Udo 12y ago
> Biology has evolved to use spikes (across many different species).
Those different species didn't independently arrive at their own neuronal implementation, we're all using the same basic pattern (with heavy modifications).
More importantly though, in biology, most cells use temporal encoding - one spike isn't meant to be a binary event. Instead, the frequency of pulses over time is used to encode intensity. The receiving end can then continuously integrate the signal over time. I'm not sure the IBM chip uses spikes in the same way.
> Perhaps evolution didn't get the memo that non-spiking convolutional neural networks is the only architecture worth building.
Temporal encoding in nature is used for the obvious reason of hardware efficiency. If every axon had 8+ binary signal lines it would become unreasonably complex and large, and prone to failure. This somewhat mirrors the design decision in the TrueNorth chip, again for obvious reasons.
However, the two diverge algorithmically. In nature, signal processing is used to transport relative values over the network. The chip on the other hand really seems to use boolean encoding.
You're right about one point though: it's fairly certain that a large number of configurations and algorithms do produce "working" neural networks. Nobody said that imitating our own implementation details will lead to better results than anything else we might want to try.
And in fact, that's the reason why so many computational implementations of neuronal networks do not use temporal encoding, because encoding values in floating points or even integers comes more naturally to computers than it does to wetware. But it is an abstraction that takes up comparatively many resources.