3 ms·
Yes, I've seen what neural nets are now capable of- they are capable of exactly what they were always capable of, except "now" (in the last few years) we have m
by YeGoblynQueenne 7y ago
Yes, I've seen what neural nets are now capable of- they are capable of
exactly what they were always capable of, except "now" (in the last few years)
we have more data and more compute to train them to actually do it. Says Geoff
Hinton [1].
I have also seen what neural nets are incapable of. Specifically,
generalisation and reasoning. Says François Chollet of Keras [2].
AI, i.e. the sub-field of computer science research that is called "AI" and
that consists of conferences such as AAAI, IJCAI, NeurIPS, etc, and assorted
journals, cannot progress on the back of a couple of neural net architectures
incapable of generalisation and reasoning. We had reasoning down pat in the
'80s. Eventually, the hype cycle will end, the Next Big Thing™ will come
around and the hype cycle will start all over again. It's the nature of
revolutions, see?
So hold your horses. Deep learning is much more useful for AI researchers who
want to publish a paper in one of the big AI conferences, and to the FANG
companies who have huge data and compute, than it is to anyone else. Anyone
else who wants to do AI will need to wait their turn and hope something else
comes around that has reasonable requirements to use, and scales well. Just as
the original article suggests.
_________________
[1] http://techjaw.com/2015/06/07/geoffrey-hinton-deep-learning-in-baby-steps-and-the-future-of-google-and-ai/ http://techjaw.com/2015/06/07/geoffrey-hinton-deep-learning-...
Geoffrey Hinton: I think it’s mainly because of the amount of computation
and the amount of data now around but it’s also partly because there have
been some technical improvements in the algorithms. Particularly in the
algorithms for doing unsupervised learning where you’re not told what the
right answer is but the main thing is the computation and the amount of
data.
[2] https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html
Say, for instance, that you could assemble a dataset of hundreds of
thousands—even millions—of English language descriptions of the features of
a software product, as written by a product manager, as well as the
corresponding source code developed by a team of engineers to meet these
requirements. Even with this data, you could not train a deep learning model
to simply read a product description and generate the appropriate codebase.
That's just one example among many. In general, anything that requires
reasoning—like programming, or applying the scientific method—long-term
planning, and algorithmic-like data manipulation, is out of reach for deep
learning models, no matter how much data you throw at them. Even learning a
sorting algorithm with a deep neural network is tremendously difficult.
- ScottBurson 7y ago> We had reasoning down pat in the '80s. We had something, but if we really had reasoning "down pat", we would not now be reading an article about someone faking automated app development. Programming is all about reasoning.
- YeGoblynQueenne 7y agoThat programming needs reasoning doesn't mean that reasoning is the only thing you need if you want to do (automated) programming, or that reasoning is only useful in (automated) programming. That is relevant to your comment. The work on automated reasoning (or "inference", etc) really started in the '50s with Church and Turing, then reached a peak in the late 80's and 90's with work on automated theorem proving (there was a great big push at the time to solve very hard problems to do with the soundness and completeness of inference procedures, particularly resolution) and is still going on (for example with Constraint Programming and Answer Set Programming etc). The result of this work was logic programming. I'm leaving out all the work on functional programming that was just another branch of the same tree, if you like, because I don't know it that well but I'm sure others on this board can complete the picture. Then of course there was all the other classical AI stuff on planning, grammar learning, game playing etc etc that you can read about in Russel & Norvig. Now, all this work could potentially be turned to the task of automated programming- but automated programming was never the goal of all that research. There was a lot of work on program synthesis, but that was just another AI sub-field with its own specific goals, that were not the overarching goals of the field as a whole. That is why we don't have automated programming at the push of a button, today: because it was never the main subject of AI research. Edit: bit of a plug. Like I say in another comment, my PhD is on algorithms that learn logic programs from examples and background knowledge (both of which are also logic programs). That's Inductive Logic Porgramming. Our stuff works. We can learn recursive programs and even invent sub-programs that are necessary to complete a programming task and that are not provided by the user. We are making big leaps all the time and we're way, way ahead of neural program synthesis and the like. There's also a whole field of Inductive Functional Programming that does the same stuff but with functional programming languages. Automating app development with that sort of technique is mainly a matter of engineering- the research is out there. But, you haven't heard anything about it because the hullaballoo about deep learning is covering everything else up and most people don't even know there is AI outside of deep learning. Hence my comments in this thread (rather obviously).