4 ms·
If deep learning isn’t enough, then brains aren’t enough, right? Edit: No offense to Gary Marcus, first heard of him after reading this article.
by 9wzYQbTYsAIc 5y ago
If deep learning isn’t enough, then brains aren’t enough, right?
Edit: No offense to Gary Marcus, first heard of him after reading this article.
- ravi-delia 5y agoBrains are a) far far more recurrent than any practical deep learning system b) using something other than backprop, and probably that something is much more efficient
- 9wzYQbTYsAIc 5y agoDeep learning is like fish, there are a lot of types of fish. Some will eat you and some you eat. My understanding of deep learning is that it is a methodology, not a technique.
- ravi-delia 5y agoDeep learning is all the very many models that use layers of neural networks. You can tweak it a lot, but there comes a point where you're not working with neural networks anymore. The human brain has reached that point.
- 9wzYQbTYsAIc 5y ago> there comes a point where you're not working with neural networks anymore. The human brain has reached that point No offense, just startled, but that’s the most self-contradictory thing I’ve ever heard. The human brain is by definition a neural network…
- ravi-delia 5y ago"Neural Network", tragically, is a term that refers to a specific kind of model inspired by a sort of 70s understanding of human neurons. Deep learning refers to those models. The human brain is made of neurons, and by God do they network, but the term has been co-opted so it is not a neural network. A particular region or set of neurons that do one thing might be called a neural circuit, and the whole thing can be called the connectome (though that's sort of debatable).
- 9wzYQbTYsAIc 5y agoMaybe it’s time for “academic justice” to reclaim the term.
- ravi-delia 5y agoI'd love nothing more, believe me. But even if neural networks were taken to unambiguously refer to the neurons of animals, deep learning would still refer to the computational model, whatever its new name would be.
- 9wzYQbTYsAIc 5y agoTechnically, Artificial Neural Networks are what people mean when they say Neural Network, right? Biological Neural Network probably ought to be the default meaning of Neural Network, given the implicit homage to biological neurons. I always wondered why people stopped referring to ANN and just went with neural network. Perhaps some anti-science bias or something like that.
- mcswell 5y agoI wonder sometimes how tiny brains work. Take butterflies: they can flit among flowers on a bush or in a small area on the ground; how do they know how to move around so they fly to a nearby flower, then stop and insert their proboscis into the nectar-bearing region? I guess there is no real learning going on--the behavior is probably innate--but still, it seems miraculous.
- 9wzYQbTYsAIc 5y agoI believe they call that Hebbian Learning in neuroscience.
- phreeza 5y agoNot really, hebbian learning is also learning from experience. Butterfly behavior is probably mostly hardwired.
- neuah 5y agoI agree, it is miraculous. the fact that dna is ultimately storing all the info for specifying neural circuits that robustly support such complex innate behaviors (often with very little post development tuning / learning) , that to me is mind-blowing. And butterfly behavior is one thing, but what about innate detection of predators in some visual systems? How do you encode a snake detector in dna?
- tasty_freeze 5y agoNope. I don't think anyone thinks the brain is an N layer deep network. There are many networks, and they cross-couple inputs and outputs from various subnetworks into others. There are also hormonal regulatory systems with feedback mechanisms that interact with these networks both is inputs and outputs. The brain is hugely more complicated and messy than anything that, say, GPT-3 is doing.
- 9wzYQbTYsAIc 5y agoSo Parallel Distributed Processing, it is? Apparently deep learning has been beat-to-death as a word, but the way deep learning was explained to me, it is an approach not a specification.
- kortilla 5y agoYeah, if you dilute the meaning down to “feedback loop”, then deep learning and the brain work the same way.
- 9wzYQbTYsAIc 5y agoI’m not sure that it is correct to limit deep learning to just a feedback loop, but cybernetics (if the thermostat variety, not the terminator variety) might be the right word.
- hajile 5y agoI'd also add that the subconscious parts of the brain seem to function quite a bit differently from the conscious parts. It's hard to reason about something like "why does my eye see how it does" when those parts aren't consciously accessible.
- visarga 5y ago> If deep learning isn’t enough, then brains aren’t enough, right? No, not right. Humans need other humans to be smart, we need culture, technology and science. We're not that smart - just 700 years ago we were dying of the plague without even knowing the germ theory of disease, even with our lives on the line we couldn't solve it. We're only INCREMENTALLY smart over our current cultural level.
- 9wzYQbTYsAIc 5y agoHow do you explain non-human intelligence and tool usage, where there is no known culture?
- visarga 5y agoThey can still individually learn tool usage by imitation, random exploration and reinforcement. But without culture they can't build on it.
- krageon 5y agoHuman supremacist world-view. Sadly incredibly common, once you drill down.
- visarga 5y agoIt's culture-supremacist actually. Information needs to copy itself into the future, that's the essence of both life (genes) and culture (memes).
- 9wzYQbTYsAIc 5y agoCulture supremacy is still just bio-supremacy. Memes don’t exist except through genes. Arguing that memes exist outside of genes is like arguing that the Platonic solids are real.
- maze-le 5y agoActually, tool usage in primates is a form of culture (protoculture). Those groups that have mastered it can be better of than those that didn't -- giving them an evolutionary advantage. The "tools" can be suprisingly complex, see for example: https://web.archive.org/web/20110608070200/http://www.uic.edu/classes/osci/osci590/8_1%20Chimpanzee%20Medicine%20Chest.htm https://web.archive.org/web/20110608070200/http://www.uic.ed...