3 ms·

Well of course you say that, you're an ML researcher who likely went into the field expecting further steady progress comparable to what we saw between 2012 and

Well of course you say that, you're an ML researcher who likely went into the field expecting further steady progress comparable to what we saw between 2012 and 2016. If actual progress in the field would be slower than what is currently still expected by the majority of people this would have dramatic consequences for future research investment, which is why you see most ML researchers reinforcing the hype or at least not talking the hype down. While the accomplishments in the last 8 years have been impressive and applications of those techniques have and will continue to have impact in the real world I think that there are extremely big obstacles in the way towards something that would be truly transformative and which would put a lot of people out of work, which the $20 trillion industry estimates assume. Unfortunately I have seen no evidence that there are sufficiently good ideas in the field to bypass the upcoming roadblocks.

randcraw 6y ago
I think a major problem in judging the advancement of AI is that its successes are not general purpose -- it works great for narrow tasks and not well for broad tasks that we humans do every day, especially those that require what we call "understanding" (as in modeling and employing semantics or relationships among objects like causal inference). Deep learning's big success is its ability to attach a label to a complex signal -- an image or a sound. That's pattern recognition: speech in, speech out, and image recognition. These tasks were largely beyond computers until 2012 but something that a child or a squirrel does very well, so it's been long known that skills like these are not useful signs of intelligence or rising levels of cognition. Other tasks DL does well are those that benefit from memorization of results form death-by-search and from vast amounts of RL simulation, like playing of board games and the transforming of patterns (images and sounds) in fun ways using GANs. But ever since Deep Blue's win over Kasparov using clever pattern matching of past chess games (by memorization), we've known that narrow skills like game play also are not useful signs of intelligence. Yet pattern matching skills are 99% of what deep learning hath wrought. Yes, that's useful, but it's not really intelligent. It shows no signs of thinking / cognition aside from probabilistic association / clustering. So there's no reason to imagine that techniques like deep nets will take us all the way to thinking like a human. Today, because of DL, we're much better at pattern matching. But in terms of what's essential to cognition, pattern matching achieves surprisingly little if your real goal is to THINK. As far as "thinking" tasks go, cognitive tasks like machine interpretation of intent in written text and machine translation between languages still suck, despite the impressive advances in semantic-surface associators like BERT and newer transformer-based NLP engines. To do more than answer basic questions about nouns and verbs, you need a model for deeper semantics and an understanding of logic and relations between actors. Until deep nets can model semantics that are not present in the test data, and employ logical inference, it can't be said to think, much less intelligently.

randcraw 6y ago

I think a major problem in judging the advancement of AI is that its successes are not general purpose -- it works great for narrow tasks and not well for broad tasks that we humans do every day, especially those that require what we call "understanding" (as in modeling and employing semantics or relationships among objects like causal inference). Deep learning's big success is its ability to attach a label to a complex signal -- an image or a sound. That's pattern recognition: speech in, speech out, and image recognition. These tasks were largely beyond computers until 2012 but something that a child or a squirrel does very well, so it's been long known that skills like these are not useful signs of intelligence or rising levels of cognition. Other tasks DL does well are those that benefit from memorization of results form death-by-search and from vast amounts of RL simulation, like playing of board games and the transforming of patterns (images and sounds) in fun ways using GANs. But ever since Deep Blue's win over Kasparov using clever pattern matching of past chess games (by memorization), we've known that narrow skills like game play also are not useful signs of intelligence. Yet pattern matching skills are 99% of what deep learning hath wrought. Yes, that's useful, but it's not really intelligent. It shows no signs of thinking / cognition aside from probabilistic association / clustering. So there's no reason to imagine that techniques like deep nets will take us all the way to thinking like a human. Today, because of DL, we're much better at pattern matching. But in terms of what's essential to cognition, pattern matching achieves surprisingly little if your real goal is to THINK. As far as "thinking" tasks go, cognitive tasks like machine interpretation of intent in written text and machine translation between languages still suck, despite the impressive advances in semantic-surface associators like BERT and newer transformer-based NLP engines. To do more than answer basic questions about nouns and verbs, you need a model for deeper semantics and an understanding of logic and relations between actors. Until deep nets can model semantics that are not present in the test data, and employ logical inference, it can't be said to think, much less intelligently.