5 ms·
>> At the moment, it is very easy to suspect that it is only the economics of training a neural net (hardware, specialist attention, data gathering an processin
by YeGoblynQueenne 8y ago
>> At the moment, it is very easy to suspect that it is only the economics of
training a neural net (hardware, specialist attention, data gathering an
processing) that is holding us back from AGI.
Neural nets have fundamental limitations, particularly their inability to
generalise outside a narrow band around their training set [1].
For instance- you will not find anything in the neural network bibliography
about learning such a fundamental concept as counting, in the sense of finding
the number n+1 that follows from a given number n, for arbitrary n. A neural
net can certainly be trained on that task for numbers in a range [k,n] - but,
given numbers outside this range its performance would "fall off a cliff".
That's because neural networks can't learn general rules such as "x > y → y =
x + k" etc.
Add to this the fact that neural nets, while very good at specific tasks, must
be trained anew for each new task- and you see that there is a big problem in
getting anything approaching "general intelligence" (which might involve any
number of wildly varying tasks) just with neural nets. For one thing- even if
you could train a neural net model for something like a million tasks, there
would remain the question of somehow stitching them all together in a coherent
whole capable of performing the right task at the right time or combining
decisions from multiple models.
Finally, note that nobody is even trying to train a neural net to learn
"general intelligence" end-to-end from examples; first of all, because we have
no idea what constitutes general intelligence and therefore how to collect its
examples.
So it's not a matter of scale. More of a matter that there isn't anyone living
today that has any idea how to get to AGI with any existing technique-
including neural nets.
____________________
[1] https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html
- YeGoblynQueenne 8y agoAs an aside, in order to be able to speculate about how close we are to AGI, with neural networks or any other technique, it is important to first unerstand the state of the art in AI. As usual, I'll recommend the classic AI textbook, "AI - A modern approach" by Russel and Norvig: http://aima.cs.berkeley.edu/ http://aima.cs.berkeley.edu/
- bachbach 8y agoAre you really under attack for recommending the standard textbook? This website's community has got peculiar more and more over the last 3-5 years and not in a positive way. There were the assholes of yesteryear but even they seemed like more sincere assholes than present day assholes.
- YeGoblynQueenne 8y agoI'm guessing my comment was construed as being patronising. I can't imagine anyone would downvote a recommendation to read Russel & Norvig. It's a good book.
- fchollet 8y ago> there isn't anyone living today that has any idea how to get to AGI Not true.
- YeGoblynQueenne 8y agoWell then- who has?
- ghthor 8y agoI feel like Jeff Hawkins and his research group know how to get to AGI. There neural model has demonstrated generalization outside of training set that you say doesn't exist. https://youtu.be/dyD0kNZnaAk https://youtu.be/dyD0kNZnaAk
- YeGoblynQueenne 8y agoCould you please point at the time in that video where this superior generalisation ability is discussed? It's a long video and I'm not sure what I'm looking for. Better still, could you link to a paper where the relevant results are presented?
- ghthor 8y agoYes, I will find the demo this for you.
- ghthor 8y agohttps://youtu.be/izO2_mCvFaw?list=LL3W7v7dYI4PwfYukN4hJCSw&t=1829 https://youtu.be/izO2_mCvFaw?list=LL3W7v7dYI4PwfYukN4hJCSw&t... All the working examples he shares in this talk are exciting, but this one stands out.
- ghthor 8y agohttps://youtu.be/6ufPpZDmPKA?list=PLGtryE0TPZPGReCdQ_F-n0aWz4ub8igFk&t=1268 https://youtu.be/6ufPpZDmPKA?list=PLGtryE0TPZPGReCdQ_F-n0aWz... Sparse Distributed Representations, The data model of the brain and the key to how the brain generalizes.
- keghn 8y agoreally now?
- keghn 8y agoHere is a synopsis of my complete model with out my complete model of human psychology. https://groups.google.com/forum/#!topic/artificial-general-intelligence/0rHVcqNoFG8 https://groups.google.com/forum/#!topic/artificial-general-i...