4 ms·

We hear that opinion all the time. As someone working in neural net-based computer vision I'd basically agree that the current approaches are tending towards a

We hear that opinion all the time. As someone working in neural net-based computer vision I'd basically agree that the current approaches are tending towards a non-AGI local maximum, but I'd note that as compared to the 80s, this is an economically productive local maximum, which will likely help fuel new developments more efficiently than in previous waves. The next breakthrough may be made by someone academically untrained, but you can bet they'll have learned a whole lot of math, computer science, data science, and maybe neuroscience first.

spinningslate 7y ago
Agreed. I'm nowhere near expert enough to opine on how far state-of-the-art is from some global maxima. I'd contend that, for the most part, it doesn't matter. It's a bit like the whole ML vs AGI debate ("but ML is just curve fitting, it's not real intelligence"). The more pertinent question for human society is the impact it has - positive or negative. ML, with all its real or perceived weaknesses, is having a significant impact on the economy specifically and society generally. It'll be little consolation for white collar workers who lose their jobs that the bot replacing them isn't "properly intelligent". Equally, few people using Siri to control their room temperature or satnav will care that the underlying "intelligence" isn't as clever as we like to think we are. Maybe current approaches will prove to have a cliff-edge limitation like previous AI approaches did. That will be interesting from a scientific progress perspective. But even in its current state, contemporary ML has plently scope to bring about massive changes in society (and already is). We should be careful not to miss that in criticising current limitations.
- bjornsing 7y ago
  > as clever as we like to think we are. Word. I think we’re actually at a level now that we’ll soon start questioning how intelligent people really are, and how much of human intelligence is just an uncanny ability to hide incompetence/lack of deeper comprehension. (Of course we’re a hell of a long way from A.I. with deep comprehension, and may remain so for hundreds of years. It’s impossible to predict that kind of quantum leap IMHO.)
  - the8472 7y ago
    The questions are already being asked. > Humans Who Are Not Concentrating Are Not General Intelligences https://srconstantin.wordpress.com/2019/02/25/humans-who-are-not-concentrating-are-not-general-intelligences/ https://srconstantin.wordpress.com/2019/02/25/humans-who-are...
- rumticular 7y ago
  This perspective makes sense pragmatically, but in philosophical terms it’s a little absurd. Going back to Turing, the argument was for true, human creativity. The claim was that there is no theoretical reason a machine cannot write a compelling sonnet. After spending the better part of a century on that problem, we have made essentially zero progress. We still believe that there is no theoretical reason a machine cannot write a compelling sonnet. We still have zero models for how that could actually work. If you are a non-technical person who has been reading popular reporting about ML, you might well have been given the impression that something like GPT2 reflects progress on the sonnet problem. Some very technical people seem to believe this too? Which seems like an issue, because there’s just no evidence for it. Maybe a larger/deeper/more recurrent ML approach will magically solve the problem in the next twenty years. And maybe the first machine built in the 20th century that could work out symbolic logic faster than all of the human computers in the world would have magically solved it. There was no systematic model for the problem, so there was no reason to conclude one way or another, just as there isn’t any today. ML is a powerful metaprogramming technique, probably the most productive one developed yet. And that matters. It’s just still not at all what we’ve been proposing to the public for a hundred years. To the best of our understanding, it’s not even meaningfully closer. And that matters too, even if Siri still works fine.
  - visarga 7y ago
    > After spending the better part of a century on that problem, we have made essentially zero progress. I dunno, I heard music composed with neural nets that is above what the average human could achieve [1]. Not on par with the greatest composers, but over average human level. In the same line of thought, I have seen models do symbolic math better than automatic solvers, generate paintings better than average humans could paint, even translate better than average second language learners. I would rate current level in AI at 50% of human intelligence on average, and most of that was accomplished in the recent years. [1] Google Music Transformer - https://magenta.tensorflow.org/music-transformer https://magenta.tensorflow.org/music-transformer
  - p1esk 7y ago
    Re sonnet problem: we can use GPT-2 to generate 10k sonnets, then choose the best one (say by popular vote, or expert opinion, etc), it's quite likely to be "compelling" or at least on par with an average published sonnet. Do you agree? If yes, then with some further deep learning research, more training data, and bigger models, we will probably be able to eventually shrink the output space to 1k, 100, and eventually maybe just 10 sonnets to choose from, to get similar quality. Would this be considered "progress for that problem" in your opinion?
dna_polymerase 7y ago
> [..] this is an economically productive local maximum, which will likely help fuel new developments more efficiently than in previous waves. This is exactly it. Every previously seen AI Winter had in common that funding was cut back. However, Google or other companies in the realm could approach a point where further investment wouldn't make sense to them. Until then there won't be a Winter, maybe Autumn, because smaller players disappear.
mrec 7y ago
> The next breakthrough may be made by someone academically untrained, but you can bet they'll have learned a whole lot of math, computer science, data science, and maybe neuroscience first. I found this sentence particularly intriguing given that John Carmack recently announced that he was switching his main focus to AI.
- The_rationalist 7y ago
  Where can we follow his work?

spinningslate 7y ago

Agreed. I'm nowhere near expert enough to opine on how far state-of-the-art is from some global maxima. I'd contend that, for the most part, it doesn't matter. It's a bit like the whole ML vs AGI debate ("but ML is just curve fitting, it's not real intelligence"). The more pertinent question for human society is the impact it has - positive or negative. ML, with all its real or perceived weaknesses, is having a significant impact on the economy specifically and society generally. It'll be little consolation for white collar workers who lose their jobs that the bot replacing them isn't "properly intelligent". Equally, few people using Siri to control their room temperature or satnav will care that the underlying "intelligence" isn't as clever as we like to think we are. Maybe current approaches will prove to have a cliff-edge limitation like previous AI approaches did. That will be interesting from a scientific progress perspective. But even in its current state, contemporary ML has plently scope to bring about massive changes in society (and already is). We should be careful not to miss that in criticising current limitations.

bjornsing 7y ago
> as clever as we like to think we are. Word. I think we’re actually at a level now that we’ll soon start questioning how intelligent people really are, and how much of human intelligence is just an uncanny ability to hide incompetence/lack of deeper comprehension. (Of course we’re a hell of a long way from A.I. with deep comprehension, and may remain so for hundreds of years. It’s impossible to predict that kind of quantum leap IMHO.)
- the8472 7y ago
  The questions are already being asked. > Humans Who Are Not Concentrating Are Not General Intelligences https://srconstantin.wordpress.com/2019/02/25/humans-who-are-not-concentrating-are-not-general-intelligences/ https://srconstantin.wordpress.com/2019/02/25/humans-who-are...
rumticular 7y ago
This perspective makes sense pragmatically, but in philosophical terms it’s a little absurd. Going back to Turing, the argument was for true, human creativity. The claim was that there is no theoretical reason a machine cannot write a compelling sonnet. After spending the better part of a century on that problem, we have made essentially zero progress. We still believe that there is no theoretical reason a machine cannot write a compelling sonnet. We still have zero models for how that could actually work. If you are a non-technical person who has been reading popular reporting about ML, you might well have been given the impression that something like GPT2 reflects progress on the sonnet problem. Some very technical people seem to believe this too? Which seems like an issue, because there’s just no evidence for it. Maybe a larger/deeper/more recurrent ML approach will magically solve the problem in the next twenty years. And maybe the first machine built in the 20th century that could work out symbolic logic faster than all of the human computers in the world would have magically solved it. There was no systematic model for the problem, so there was no reason to conclude one way or another, just as there isn’t any today. ML is a powerful metaprogramming technique, probably the most productive one developed yet. And that matters. It’s just still not at all what we’ve been proposing to the public for a hundred years. To the best of our understanding, it’s not even meaningfully closer. And that matters too, even if Siri still works fine.
- visarga 7y ago
  > After spending the better part of a century on that problem, we have made essentially zero progress. I dunno, I heard music composed with neural nets that is above what the average human could achieve [1]. Not on par with the greatest composers, but over average human level. In the same line of thought, I have seen models do symbolic math better than automatic solvers, generate paintings better than average humans could paint, even translate better than average second language learners. I would rate current level in AI at 50% of human intelligence on average, and most of that was accomplished in the recent years. [1] Google Music Transformer - https://magenta.tensorflow.org/music-transformer https://magenta.tensorflow.org/music-transformer
- p1esk 7y ago
  Re sonnet problem: we can use GPT-2 to generate 10k sonnets, then choose the best one (say by popular vote, or expert opinion, etc), it's quite likely to be "compelling" or at least on par with an average published sonnet. Do you agree? If yes, then with some further deep learning research, more training data, and bigger models, we will probably be able to eventually shrink the output space to 1k, 100, and eventually maybe just 10 sonnets to choose from, to get similar quality. Would this be considered "progress for that problem" in your opinion?

dna_polymerase 7y ago

> [..] this is an economically productive local maximum, which will likely help fuel new developments more efficiently than in previous waves. This is exactly it. Every previously seen AI Winter had in common that funding was cut back. However, Google or other companies in the realm could approach a point where further investment wouldn't make sense to them. Until then there won't be a Winter, maybe Autumn, because smaller players disappear.

mrec 7y ago

> The next breakthrough may be made by someone academically untrained, but you can bet they'll have learned a whole lot of math, computer science, data science, and maybe neuroscience first. I found this sentence particularly intriguing given that John Carmack recently announced that he was switching his main focus to AI.

The_rationalist 7y ago
Where can we follow his work?