4 ms·

AI is going to invent that revolutionary software, but unfortunately you and I won't understand it or be able to manipulate it. Be careful what you wish for.

miguelrochefort 8y ago
Is it coming soon? I'm not sure I can wait 5 more years.
- sverige 8y ago
  Oh, well, probably not 5 years. More like 20 years away for human-level AI, according to everyone I've read since the 80s. Fusion power will be coming online right around then, too, so that's good. And the true value of the blockchain will finally be evident in about 20 years, too. Which is really good, because that's around the time we are likely to discover the secrets of immortality.
calebh 8y ago
Imo gradient descent is not going to get us to general AI. I'm a programming languages theory person, and I also know a fair bit about machine learning. The current problem with neural networks is that they suck at processing variable length data, and they have problems with remembering the past. Programs can be represented by trees, graphs, or text. Deep learning isn't great at dealing with graphs or trees, and it's not that good at text either. The other issue is that deep learning works great for recognizing common patterns, but it sucks when faced with novel situations. So I don't think that we're going to have programs that program anytime soon. The first applications of AI to programming will probably be with programming assistants or with AI guided proof solvers. Programmer productivity will improve, but we're not going to see everyone losing their jobs.
- nmca 8y ago
  The latter of those two points is much more valid than the former. You're comment about generalisation difficulty is pretty accurate, but as for "sucks at variable length data", I think neural machine translation [0] and the fact that schemes including RNNs just won M4 [1] indicate that this is incorrect. Your point about remembering the past is true (it's hard), but people are actively working on it. Unitary neural nets, and the fast/slow weight paradigm are very different angles that seem promising. As for handling trees + graphs, this actually works very well. Thomas Kipf is pushing this area forward, GATs [2] are a nice random example of how dominant differentiable programming (eg NNs) can be in this area. Unfortunately these graph approaches don't parallelise as nicely on GPUs as CNNs. Your predictions (assistants) seem likely to me. [0] https://arxiv.org/abs/1804.09849 https://arxiv.org/abs/1804.09849 [1] https://www.m4.unic.ac.cy https://www.m4.unic.ac.cy [2] https://arxiv.org/abs/1710.10903 https://arxiv.org/abs/1710.10903

miguelrochefort 8y ago

Is it coming soon? I'm not sure I can wait 5 more years.

sverige 8y ago
Oh, well, probably not 5 years. More like 20 years away for human-level AI, according to everyone I've read since the 80s. Fusion power will be coming online right around then, too, so that's good. And the true value of the blockchain will finally be evident in about 20 years, too. Which is really good, because that's around the time we are likely to discover the secrets of immortality.

calebh 8y ago

Imo gradient descent is not going to get us to general AI. I'm a programming languages theory person, and I also know a fair bit about machine learning. The current problem with neural networks is that they suck at processing variable length data, and they have problems with remembering the past. Programs can be represented by trees, graphs, or text. Deep learning isn't great at dealing with graphs or trees, and it's not that good at text either. The other issue is that deep learning works great for recognizing common patterns, but it sucks when faced with novel situations. So I don't think that we're going to have programs that program anytime soon. The first applications of AI to programming will probably be with programming assistants or with AI guided proof solvers. Programmer productivity will improve, but we're not going to see everyone losing their jobs.

nmca 8y ago
The latter of those two points is much more valid than the former. You're comment about generalisation difficulty is pretty accurate, but as for "sucks at variable length data", I think neural machine translation [0] and the fact that schemes including RNNs just won M4 [1] indicate that this is incorrect. Your point about remembering the past is true (it's hard), but people are actively working on it. Unitary neural nets, and the fast/slow weight paradigm are very different angles that seem promising. As for handling trees + graphs, this actually works very well. Thomas Kipf is pushing this area forward, GATs [2] are a nice random example of how dominant differentiable programming (eg NNs) can be in this area. Unfortunately these graph approaches don't parallelise as nicely on GPUs as CNNs. Your predictions (assistants) seem likely to me. [0] https://arxiv.org/abs/1804.09849 https://arxiv.org/abs/1804.09849 [1] https://www.m4.unic.ac.cy https://www.m4.unic.ac.cy [2] https://arxiv.org/abs/1710.10903 https://arxiv.org/abs/1710.10903