6 ms·

In case you're not aware, AlphaGo's key component is based on the same type of Deepmind system that learned to play dozens of Atari games, to superhuman levels,

by Eliezer 11y ago

In case you're not aware, AlphaGo's key component is based on the same type of Deepmind system that learned to play dozens of Atari games, to superhuman levels, by watching the pixels, without any programmatic adaptation to the particular Atari game. At least the version of AlphaGo that played in October was far less specialized for Go than Deep Blue was for chess. Demis Hassabis says that next up after this is getting Deepmind to play Go without any programmatic specialization for Go. Your reply would be appropriate if we were talking about Deep Blue, chess, and 1997.

Fede_V 11y ago
That's incorrect. The features that AlphaGo uses are not pixel level features, but board states - and the architecture between AlphaGo and the Atari network is completely different. It's still an incredibly achievement - but it's important to be accurate.
- DavidSJ 11y ago
  For AlphaGo, a "pixel" is a point on the board. It uses essentially the same convolutional neural networks (CNNs) that are in state-of-the-art machine vision systems. But yes, the overall architecture is rather different from the Atari system, due to the integration of that CNN with Monte Carlo Tree Search.
  - drawnwren 11y ago
    Sorry, you're off base a bit. The Atari system did use a Deep Neural Network / Reinforcement algorithm, but as the original poster was trying to point out, the rules of Go were very much hard coded into AlphaGo. From what this [1] says, multiple DNNs are learning how to traverse Monte Carlo trees of Go games. The reinforcement piece comes in choosing which of the Go players is playing the best games. While the higher portions do share some similarities with the Atari system, at a basic level this is a machine that was designed and trained to play Go. AlphaGO is 'essentially the same' as the Atari system in the same way that all Neural Networks are 'essentially the same.' Is this an extremely impressive accomplishment? Yes. However, doesn't seem to qualify as anything close to generalizable. [1] http://googleresearch.blogspot.com/2016/01/alphago-mastering-ancient-game-of-go.html http://googleresearch.blogspot.com/2016/01/alphago-mastering...
    - DavidSJ 11y ago
      I didn't say AlphaGo is essentially the same as Deep Q Networks. I said the convolutional neural network part of it is essentially the same. We agree that the integration of that CNN into the rest of the system is very different.
    - Houshalter 11y ago
      It's best to say that alphago uses neural networks, which are extremely general. The same way planes and cars both use internal combustion engines. ICEs are extremely general. They produce mechanical energy from gas, and are totally uncaring whether you put them into a plane or a car. The body of the plane is necessary, but isn't really the interesting part. Likewise NNs are uncaring what application you put them into. Give them a different input and a different goal, and they will learn to do that instead. Alphago gave it's NN's control over a monte carlo search tree, and that turned out to be enough to beat Go. They could plug the same AI into a car and it would learn to control that instead. Note that even without the monte carlo search system, it was able to beat most amateurs, and predict the moves experts would make most of the time.
      - argonaut 11y ago
        Even without the neural net system, AI is able to beat most amateurs, and predict moves experts would make.
        Houshalter 11y ago
        I'm not sure that's correct. MCTS has well known weaknesses, and isn't even a predictive algorithm. MCTS on it's own couldn't get anywhere near beating the top Go champion, that requires deepminds neural networks.
        argonaut 11y ago
        http://www.milesbrundage.com/blog-posts/alphago-and-ai-progress http://www.milesbrundage.com/blog-posts/alphago-and-ai-progr... The best Go program before AlphaGo was CrazyStone, ranked at 5-dan ("high amateur" range).
  - eru 11y ago
    Actually, putting a piece of software in front that infers board states from a video feed would be an easy problem.
    - Fede_V 11y ago
      That's actually true - going from the pixel level to the board state is trivial and not particularly interesting.
      - eru 11y ago
        It's trivial today. It would have been interesting perhaps twenty years ago?
        argonaut 11y ago
        No, because even handcrafted computer vision systems from 20 years ago would be able to parse a Go board (edge detection + check the color contrast).
        eru 11y ago
        True. I guess you'd have to go even further back.
YeGoblynQueenne 11y ago
>> In case you're not aware, AlphaGo's key component is based on the same type of Deepmind system that learned to play dozens of Atari games, to superhuman levels, by watching the pixels, without any programmatic adaptation to the particular Atari game. The Atari-playing AI watched the pixels indeed, but it was also given a set of actions to choose from and more importantly, a reward representing the change in the game score. That means it wasn't able to learn the significance of the score on its own, or to generalise from the significance of the changing score in one game, to another. It also played Atari games, that _have_ scores, so it would have been completely useless in situations where there is no score, or a clear win/loss situation. AlphaGo is also similarly specialised to play Go. As is machine learning in general: someone has to tell the algorithm what it needs to learn, either through data engineering, or reward functions etc. A general AI would learn what is important on its own, like humans do, so machine learning has not yet shown that it can develop into AGI.
- Houshalter 11y ago
  I think you are confusing utility functions with intelligence. All AIs need utility functions. An AI without a utility function would just do nothing. It would have no reason to beat Atari games, because it wouldn't get any reward for doing so. Even humans have utility functions. For example, we get rewards for having sex, or eating food, or just making social relationships with other humans. Or we have negative reinforcement from pain, and getting hurt, or getting rejected socially. You can come up with more complicated utility functions. Like instead of beating the game, it's goal could be to explore as much of the game as possible. To discover novel things in the game. Kind of like a sense of boredom or novelty that humans have. But in the end it's still just a utility function, it doesn't change how the algorithm itself works to achieve it. AGI is entirely agnostic to the utility function.
  - YeGoblynQueenne 11y ago
    >> I think you are confusing utility functions with intelligence. No, what I'm really saying is that you can't have an autonomous agent that needs to be told what to do all the time. In machine learning, we train algorithms by giving them examples of what we want them to learn, so basically we tell them what to learn. And if we want them to learn something new, we have to train them again, on new data. Well, that's not conducive to autonomous or "general" intelligence. There may be any number of tasks that your "general" AI will need to perform competently at. What's it gonna do? Come back to you and cry every time it fails at something? So then you have a perpetual child AI that will never stand on its own two feet as an adult, because there's always something new for it to learn. Happy little AI, for sure, but not very useful and not very "general". Except for a general nuisance, maybe. Edit: I'm saying that machine learning can't possibly lead to general AI, because it's crap at learning useful things on its own.

by Eliezer 11y ago

Fede_V 11y ago

That's incorrect. The features that AlphaGo uses are not pixel level features, but board states - and the architecture between AlphaGo and the Atari network is completely different. It's still an incredibly achievement - but it's important to be accurate.

DavidSJ 11y ago
For AlphaGo, a "pixel" is a point on the board. It uses essentially the same convolutional neural networks (CNNs) that are in state-of-the-art machine vision systems. But yes, the overall architecture is rather different from the Atari system, due to the integration of that CNN with Monte Carlo Tree Search.
- drawnwren 11y ago
  Sorry, you're off base a bit. The Atari system did use a Deep Neural Network / Reinforcement algorithm, but as the original poster was trying to point out, the rules of Go were very much hard coded into AlphaGo. From what this [1] says, multiple DNNs are learning how to traverse Monte Carlo trees of Go games. The reinforcement piece comes in choosing which of the Go players is playing the best games. While the higher portions do share some similarities with the Atari system, at a basic level this is a machine that was designed and trained to play Go. AlphaGO is 'essentially the same' as the Atari system in the same way that all Neural Networks are 'essentially the same.' Is this an extremely impressive accomplishment? Yes. However, doesn't seem to qualify as anything close to generalizable. [1] http://googleresearch.blogspot.com/2016/01/alphago-mastering-ancient-game-of-go.html http://googleresearch.blogspot.com/2016/01/alphago-mastering...
  - DavidSJ 11y ago
    I didn't say AlphaGo is essentially the same as Deep Q Networks. I said the convolutional neural network part of it is essentially the same. We agree that the integration of that CNN into the rest of the system is very different.
  - Houshalter 11y ago
    It's best to say that alphago uses neural networks, which are extremely general. The same way planes and cars both use internal combustion engines. ICEs are extremely general. They produce mechanical energy from gas, and are totally uncaring whether you put them into a plane or a car. The body of the plane is necessary, but isn't really the interesting part. Likewise NNs are uncaring what application you put them into. Give them a different input and a different goal, and they will learn to do that instead. Alphago gave it's NN's control over a monte carlo search tree, and that turned out to be enough to beat Go. They could plug the same AI into a car and it would learn to control that instead. Note that even without the monte carlo search system, it was able to beat most amateurs, and predict the moves experts would make most of the time.
    - argonaut 11y ago
      Even without the neural net system, AI is able to beat most amateurs, and predict moves experts would make.
      - Houshalter 11y ago
        I'm not sure that's correct. MCTS has well known weaknesses, and isn't even a predictive algorithm. MCTS on it's own couldn't get anywhere near beating the top Go champion, that requires deepminds neural networks.
        argonaut 11y ago
        http://www.milesbrundage.com/blog-posts/alphago-and-ai-progress http://www.milesbrundage.com/blog-posts/alphago-and-ai-progr... The best Go program before AlphaGo was CrazyStone, ranked at 5-dan ("high amateur" range).
- eru 11y ago
  Actually, putting a piece of software in front that infers board states from a video feed would be an easy problem.
  - Fede_V 11y ago
    That's actually true - going from the pixel level to the board state is trivial and not particularly interesting.
    - eru 11y ago
      It's trivial today. It would have been interesting perhaps twenty years ago?
      - argonaut 11y ago
        No, because even handcrafted computer vision systems from 20 years ago would be able to parse a Go board (edge detection + check the color contrast).
        eru 11y ago
        True. I guess you'd have to go even further back.

YeGoblynQueenne 11y ago

>> In case you're not aware, AlphaGo's key component is based on the same type of Deepmind system that learned to play dozens of Atari games, to superhuman levels, by watching the pixels, without any programmatic adaptation to the particular Atari game. The Atari-playing AI watched the pixels indeed, but it was also given a set of actions to choose from and more importantly, a reward representing the change in the game score. That means it wasn't able to learn the significance of the score on its own, or to generalise from the significance of the changing score in one game, to another. It also played Atari games, that _have_ scores, so it would have been completely useless in situations where there is no score, or a clear win/loss situation. AlphaGo is also similarly specialised to play Go. As is machine learning in general: someone has to tell the algorithm what it needs to learn, either through data engineering, or reward functions etc. A general AI would learn what is important on its own, like humans do, so machine learning has not yet shown that it can develop into AGI.

Houshalter 11y ago
I think you are confusing utility functions with intelligence. All AIs need utility functions. An AI without a utility function would just do nothing. It would have no reason to beat Atari games, because it wouldn't get any reward for doing so. Even humans have utility functions. For example, we get rewards for having sex, or eating food, or just making social relationships with other humans. Or we have negative reinforcement from pain, and getting hurt, or getting rejected socially. You can come up with more complicated utility functions. Like instead of beating the game, it's goal could be to explore as much of the game as possible. To discover novel things in the game. Kind of like a sense of boredom or novelty that humans have. But in the end it's still just a utility function, it doesn't change how the algorithm itself works to achieve it. AGI is entirely agnostic to the utility function.
- YeGoblynQueenne 11y ago
  >> I think you are confusing utility functions with intelligence. No, what I'm really saying is that you can't have an autonomous agent that needs to be told what to do all the time. In machine learning, we train algorithms by giving them examples of what we want them to learn, so basically we tell them what to learn. And if we want them to learn something new, we have to train them again, on new data. Well, that's not conducive to autonomous or "general" intelligence. There may be any number of tasks that your "general" AI will need to perform competently at. What's it gonna do? Come back to you and cry every time it fails at something? So then you have a perpetual child AI that will never stand on its own two feet as an adult, because there's always something new for it to learn. Happy little AI, for sure, but not very useful and not very "general". Except for a general nuisance, maybe. Edit: I'm saying that machine learning can't possibly lead to general AI, because it's crap at learning useful things on its own.