4 ms·
Excellent article, thanks for the contribution. Perhaps some independent validation, but I was coincidentally having this conversation the other day with a rel
by dahart 11y ago
Excellent article, thanks for the contribution.
Perhaps some independent validation, but I was coincidentally having this conversation the other day with a relatively well known computer vision researcher, about why it seems like the idea of neural nets has floundered for decades and suddenly it's the hot topic, and we're seeing massively improved results.
His answers, summarized, are that:
1- Big data is making possible the kind of training we could never do before.
2- Having big data & big compute has made some training breakthroughs that allowed the depth to increase dramatically. The number of layers was implicitly limited until recently because anything deep couldn't be practically trained.
3- The activation function has very commonly in the past been an S-curve, and some of the newer better results are using a linear function that is clamped on the low end at zero, but not clamped on top.
All really interesting to me. This is making me want to implement and play with neural nets!
Of course, now the big question: if we have a neural net big enough, and it works, can we simulate a human brain? (Apparently, according to my AI researcher friend, we're not there yet with the foundational building blocks. He mentioned researchers have tried simulating a life-form known to have a small number of neurons, like a thousand, and they can't get it to work yet.)
- dwiel 11y agoI agree with everything but the last. Simulating a human brain is perhaps necessary for consciousness uploading, but not for super human ai. We have been able to travel faster by machine than foot for a long time, in spite of not being able to build a robot that can simulate human running.
- dahart 11y ago> Simulating a human brain is perhaps necessary for consciousness uploading, but not for super human ai. I'm not suggesting otherwise, and what you said might be true, I don't know. But honest question: how can we know it's true before we can do either? How can we suggest neural nets are a good building block of super human AI, if we can't verify we can do regular human thinking, or sub-human thinking? Because the neural net is modeled after (simplified) brain neurons in the first place, it seems reasonable to suggest that if we can't simulate a simple brain, then we can't validate that the model is correct, right? It might be very productive, and it might 'work' in some sense, but we don't know whether it can truly act as the lego brick of brain building material until we can build a functional brain. Your observation is true, and worth considering. Machines can travel faster than humans, but that's different in part because we didn't start trying to simulate the human foot, right? The computational equivalent would be that machines can multiply a lot of numbers much faster than a human. That is super-human calculation, and it is mechanical thinking, of a sort, but most people wouldn't say it counts as "AI", perhaps the same way that most people wouldn't say a car counts as human running, even if it is faster. Neural nets are being used to classify images, find objects, identify people's faces in a crowd or in difficult to see situations. But, they currently can't tell you to stop classifying images because you're asking the wrong question, or interrupt you to say you're looking great today.
- argonaut 11y agoBefore we get wildly over-optimistic, as every HN thread on AI becomes, realize that neural networks have basically nothing to do with actual biological neurons. There are a bajillion things neurons do that neural networks do not do. And there are similarly many things that are a part of neural networks, that biological neurons do not do. In addition to the fact that neuroscientists still do not have a clear understanding of biological neurons: you cannot expect to reverse-engineer something you barely understand yourself.
- andreyk 11y agoThis is not entirely true, as there are spiking neuron nets and neuromorphic chips (like IBM's TrueNorth) that emulate what we have so far understood about how neurons operate. But it's true that even these are simplified models and as far as I know cognitive science has not explained the higher level operation of the brain. In general I think breaking down how modern deep learning works makes it seem unlikely that it will lead to true AI without additional insights, so I am unsure why very intelligent people such as Elon Musk make it seem like an imminent threat - not to mention the idea is predicated on continued exponential increases in computing power. But the stuff going on in AI right now concerning image comprehension and question answering is fairly impressive, so I think it is feasible we will have impressive AI assistants and the like within a decade.
- pedrosorio 11y ago"This is not entirely true, as there are spiking neuron nets and neuromorphic chips (like IBM's TrueNorth) that emulate what we have so far understood about how neurons operate." How is this related to the neural networks we are discussing? (deep ANNs?) As far as I know, the ANN topic in machine learning, apart from its origins, is completely unrelated to the simulation of biological neural network models: https://en.wikipedia.org/wiki/Nervous_system_network_models https://en.wikipedia.org/wiki/Nervous_system_network_models They are often confused because of the name and because ANNs did begin as extremely simplified models of biological neural networks, but the machine learning concept that is setting all the records in vision / speech recognition serves no purpose as a model of a biological neural network.
- eli_gottlieb 11y ago>Of course, now the big question: if we have a neural net big enough, and it works, can we simulate a human brain? There are many possible neural networks. Most of them are not human brains.
- pedrosorio 11y agoHe mentioned researchers have tried simulating a life-form known to have a small number of neurons, like a thousand, and they can't get it to work yet" 302 to be precise - https://en.wikipedia.org/wiki/OpenWorm https://en.wikipedia.org/wiki/OpenWorm