3 ms·
To distill down the article: Most of what we're seeing now with neural networks was pretty well understood 20+ years ago. The hard part is forming hierarchies
by zackmorris 9y ago
To distill down the article:
Most of what we're seeing now with neural networks was pretty well understood 20+ years ago. The hard part is forming hierarchies or clusters of networks that are only active in certain scenarios, so that we can scale learning beyond simple pattern recognition (which is effectively a solved problem today).
I've read very few satisfactory articles on how to go about teaching large neural nets to learn multiple distinct patterns and know which networks to engage in different scenarios. This article is suggesting that dendrites form that orchestration layer by waiting for enough specific patterns to happen in close enough proximity that the neuron becomes engaged and goes to work when the rest of the pattern arrives. This gives the network some ability to predict what is coming, whereas the simple neural networks we model would only fire when the whole pattern matches.
There is a bit of hand waving here and I probably missed something so please add any insights of your own thanks!