3 ms·

Before we get wildly over-optimistic, as every HN thread on AI becomes, realize that neural networks have basically nothing to do with actual biological neurons

Before we get wildly over-optimistic, as every HN thread on AI becomes, realize that neural networks have basically nothing to do with actual biological neurons. There are a bajillion things neurons do that neural networks do not do. And there are similarly many things that are a part of neural networks, that biological neurons do not do. In addition to the fact that neuroscientists still do not have a clear understanding of biological neurons: you cannot expect to reverse-engineer something you barely understand yourself.

andreyk 11y ago
This is not entirely true, as there are spiking neuron nets and neuromorphic chips (like IBM's TrueNorth) that emulate what we have so far understood about how neurons operate. But it's true that even these are simplified models and as far as I know cognitive science has not explained the higher level operation of the brain. In general I think breaking down how modern deep learning works makes it seem unlikely that it will lead to true AI without additional insights, so I am unsure why very intelligent people such as Elon Musk make it seem like an imminent threat - not to mention the idea is predicated on continued exponential increases in computing power. But the stuff going on in AI right now concerning image comprehension and question answering is fairly impressive, so I think it is feasible we will have impressive AI assistants and the like within a decade.
- pedrosorio 11y ago
  "This is not entirely true, as there are spiking neuron nets and neuromorphic chips (like IBM's TrueNorth) that emulate what we have so far understood about how neurons operate." How is this related to the neural networks we are discussing? (deep ANNs?) As far as I know, the ANN topic in machine learning, apart from its origins, is completely unrelated to the simulation of biological neural network models: https://en.wikipedia.org/wiki/Nervous_system_network_models https://en.wikipedia.org/wiki/Nervous_system_network_models They are often confused because of the name and because ANNs did begin as extremely simplified models of biological neural networks, but the machine learning concept that is setting all the records in vision / speech recognition serves no purpose as a model of a biological neural network.
  - nl 11y ago
    the machine learning concept that is setting all the records in vision / speech recognition serves no purpose as a model of a biological neural network. I think you are overstating this. There are definite links between the two, even if ANNs end up different because of the tools we have available to us. CNNs were explicitly designed to mimic the behavior of the visual cortex. Most of Geoff Hinton's career has been built around thinking very hard about biological computation. One of the major criticisms of back-propegation in neural networks is that it is biologicaly implausible. https://en.m.wikipedia.org/wiki/Convolutional_neural_network#History https://en.m.wikipedia.org/wiki/Convolutional_neural_network...
dahart 11y ago
> Before we get wildly over-optimistic, as every HN thread on AI becomes, realize that neural networks have basically nothing to do with actual biological neurons. Yes. I think I was already fully agreeing with you, and suggesting the same but in softer language. Maybe you meant that for the parent or more generally the thread, but I think that I should add something: I may have unintentionally conflated the discussion on neural nets and the biological research my friend was talking about. My limited understanding of what he said - and I'll go ask him for a reference - is that we (as in scientists somewhere, not me personally) have come to a more or less clear and complete understanding of the chemical and electrical processes in the neural functioning of these simple life-forms, from top to bottom. (I don't know that's true, but that's what I think I heard.) They then tried to put together a complete simulation of this simple lifeform's neurons. This simulation, as I understand it, is not a neural network per se, but something trying to be much closer to a biological simulation. And when they turn it on, apparently, it doesn't work. That story, if true, tends to confirm what you said; we're missing something in our understanding of biological neurons. Which, I think, lets me say more confidently that we can't suggest we have the building blocks for AI yet.

andreyk 11y ago

This is not entirely true, as there are spiking neuron nets and neuromorphic chips (like IBM's TrueNorth) that emulate what we have so far understood about how neurons operate. But it's true that even these are simplified models and as far as I know cognitive science has not explained the higher level operation of the brain. In general I think breaking down how modern deep learning works makes it seem unlikely that it will lead to true AI without additional insights, so I am unsure why very intelligent people such as Elon Musk make it seem like an imminent threat - not to mention the idea is predicated on continued exponential increases in computing power. But the stuff going on in AI right now concerning image comprehension and question answering is fairly impressive, so I think it is feasible we will have impressive AI assistants and the like within a decade.

pedrosorio 11y ago
"This is not entirely true, as there are spiking neuron nets and neuromorphic chips (like IBM's TrueNorth) that emulate what we have so far understood about how neurons operate." How is this related to the neural networks we are discussing? (deep ANNs?) As far as I know, the ANN topic in machine learning, apart from its origins, is completely unrelated to the simulation of biological neural network models: https://en.wikipedia.org/wiki/Nervous_system_network_models https://en.wikipedia.org/wiki/Nervous_system_network_models They are often confused because of the name and because ANNs did begin as extremely simplified models of biological neural networks, but the machine learning concept that is setting all the records in vision / speech recognition serves no purpose as a model of a biological neural network.
- nl 11y ago
  the machine learning concept that is setting all the records in vision / speech recognition serves no purpose as a model of a biological neural network. I think you are overstating this. There are definite links between the two, even if ANNs end up different because of the tools we have available to us. CNNs were explicitly designed to mimic the behavior of the visual cortex. Most of Geoff Hinton's career has been built around thinking very hard about biological computation. One of the major criticisms of back-propegation in neural networks is that it is biologicaly implausible. https://en.m.wikipedia.org/wiki/Convolutional_neural_network#History https://en.m.wikipedia.org/wiki/Convolutional_neural_network...

dahart 11y ago

> Before we get wildly over-optimistic, as every HN thread on AI becomes, realize that neural networks have basically nothing to do with actual biological neurons. Yes. I think I was already fully agreeing with you, and suggesting the same but in softer language. Maybe you meant that for the parent or more generally the thread, but I think that I should add something: I may have unintentionally conflated the discussion on neural nets and the biological research my friend was talking about. My limited understanding of what he said - and I'll go ask him for a reference - is that we (as in scientists somewhere, not me personally) have come to a more or less clear and complete understanding of the chemical and electrical processes in the neural functioning of these simple life-forms, from top to bottom. (I don't know that's true, but that's what I think I heard.) They then tried to put together a complete simulation of this simple lifeform's neurons. This simulation, as I understand it, is not a neural network per se, but something trying to be much closer to a biological simulation. And when they turn it on, apparently, it doesn't work. That story, if true, tends to confirm what you said; we're missing something in our understanding of biological neurons. Which, I think, lets me say more confidently that we can't suggest we have the building blocks for AI yet.