4 ms·
You're dead on. We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse,
by famouswaffles 3y ago
You're dead on.
We've developed numerous different learning algorithms that are biologically plausible, but they all kinda work like backpropagation but worse, so we stuck with backpropagation. We've made more complicated neurons that better resemble biological neurons, but it is faster and works better if you just add extra simple neurons, so we do that instead. Spiking neural networks have connection patterns more similar to what you see in the brain, but they learn slower and are tougher to work with than regular layered neural networks, so we use layered neural networks instead.
Biology is Biology and Silicon is Silicon. Sometimes, Constraints can be just that - Constraints and not some secret sauce.
- tudorw 3y agoI'm interested in the 'slower/tougher' route you mentioned, do you recall the final performance, am assuming it was equivalent or worse. Given the vast disparity in power consumption between silicon and meat space it's surely a rich area to explore, especially given the feedback loop present where AI is being effectively used in more accurate observation and interpretation of brain activity.
- kaba0 3y agoAlso, biological “architecture” is so different compared to silicon design, that it might just the case that different algorithms perform best for the respective design. Biological systems are effectively huge distributed computing systems, where each unit is very weak. Backpropagation probably can’t easily be implemented for that.