5 ms·

Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understa

Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understand that it's easy to implement for OpenAI but I think having the game developers provide a real bot-capable API is much better. I hope the latter is what Blizzard will provide for their DeepMind collaboration.

mtgx 10y ago
Seems unlikely. The focus seems to be on improving AI through "vision". The idea is to make the AI learn skills the same way a human would (at least in the first years of life). Google's AlphaGo also learned from screen pixels. So these would be human-like bots, rather than bot-like bots, like you normally have in games. The bot would simply learn by doing, until it masters the game, not by getting access to game algorithms.
- SamBam 10y ago
  > Google's AlphaGo also learned from screen pixels. Source? That literally seems to make zero sense to me. Go can be represented in a super-simple state. Why make it spend millions of cycles learning to categorize pixels into that state you already have?
  - tiler 10y ago
    I would guess that they trained AlphaGo from many thousands of hours of match footage. Writing a computer vision script to segment / extract the data may cost cycles as you say, but would save many human hours by eliminating the need to re-watch the footage and literally type out state information for each move.
    - ironrabbit 10y ago
      Alphago was actually trained directly on game state (plus some extra computed state like "how many liberties will I have if I play this move" or "will I win this ladder"). A huge number of pro games (and countless amateur games) are available on servers like KGS in a nice computer-digestible format.
    - komaromy 10y ago
      I'm not too familiar with Go, but in chess we have millions of text-format games and very little real-time footage.
    - SamBam 10y ago
      Again, that doesn't make sense because the moves have almost certainly already been typed out by somebody. It's the same in chess. There are databases containing millions of games.
- gwern 10y ago
  > Google's AlphaGo also learned from screen pixels. It did not. It received the state of the board as one array, another board state array for capture/komi (since Go does have global state which is not visible purely from the board representation), and a few additional features to help it out with stuff like ladders. It was architected with convolutional layers, but over the Go grid, not pixels. See the AlphaGo paper pg11 for the exact structure of the input: http://www.postype.com/files/2016/04/08/16/05/03384c91046e8e72f7dc6f20d204231e.pdf http://www.postype.com/files/2016/04/08/16/05/03384c91046e8e... Could it have learned from pixels (augmented by the additional necessary global state)? Sure. But that would've been a waste of computation since the visual layout of a Go board is fixed and static, unlike Atari games.
alkalait 10y ago
Getting the game state data means deciding a-priori what features the AI should learn on. The whole point of the deep learning paradigm is to allow a machine to learn such features that enable good prediction, visualization, generation (aka. hallucination), etc. Instead, researchers have provided the raw feed input data to these agents with the hope that the learned features could be interpreted as game state data by humans.
- BaronSamedi 10y ago
  I would say that it is a point of deep learning rather than "the whole point". For an AI to interact with the real-world then building models from vision (as we do) makes a lot of sense. In the virtual world, however, it makes no sense. Model data is already available and the AI has no need for something as inefficient as vision. We humans have to use vision (and sound, etc) in games because we have do not have access to direct data feeds, computers have no such limitation. Why cripple the AI by imposing human limitations on it?
  - lawless123 10y ago
    If they want their AGI to applicable to the real world or software with incomplete or insufficient APIS they have to do it the way they are doing it here. There isn't an API for me to check if i'm still on the foot path and not the road as i walk down the street. I can't use an API to tell me water is boiling and that i shouldn't stick my hand in it.
chongli 10y ago
If your goal is to build General AI, getting access to the game state information is cheating and ultimately self defeating because it does not generalize across games. Expert humans are able to pick up and play a brand new game with a high degree of initial success despite zero knowledge of the memory states within the game. We do this by reliance on our vast knowledge of video game and literary tropes as well as experience playing past games. I have yet to see any video game bot make use of this stuff to figure out how to play Zelda, for example.
- BaronSamedi 10y ago
  An AI which can learn any game from "looking" at a screen is a very ambitious goal. I doubt it is achievable in the near term. Humans can do it because of years of learning about the world. Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games such as Starcraft, LoL, and Dota--especially the latter 2 since they are team games.
  - lawless123 10y ago
    That's not their goal though.They want AGI.
  - chongli 10y ago
    Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games Those sorts of goals are better suited to individual hobbyists. OpenAI is a blue sky research project set up by billionaires with the goal of improving all of mankind. I hope I'm not being too uncharitable when I say that your comment reminds me of those who scorned the Apollo program for its ambition.
    - BaronSamedi 10y ago
      I support research in AI of all kinds, however, its history is one of vastly over-promising and under-delivering. Modest but demonstrable progress is better, in my view, than yet more hype.
pakl 10y ago
If the goal is to have AI that is aware of the real world (even a chatbot), then using game state is a crutch that doesn't help us solve the real problem.

mtgx 10y ago

Seems unlikely. The focus seems to be on improving AI through "vision". The idea is to make the AI learn skills the same way a human would (at least in the first years of life). Google's AlphaGo also learned from screen pixels. So these would be human-like bots, rather than bot-like bots, like you normally have in games. The bot would simply learn by doing, until it masters the game, not by getting access to game algorithms.

SamBam 10y ago
> Google's AlphaGo also learned from screen pixels. Source? That literally seems to make zero sense to me. Go can be represented in a super-simple state. Why make it spend millions of cycles learning to categorize pixels into that state you already have?
- tiler 10y ago
  I would guess that they trained AlphaGo from many thousands of hours of match footage. Writing a computer vision script to segment / extract the data may cost cycles as you say, but would save many human hours by eliminating the need to re-watch the footage and literally type out state information for each move.
  - ironrabbit 10y ago
    Alphago was actually trained directly on game state (plus some extra computed state like "how many liberties will I have if I play this move" or "will I win this ladder"). A huge number of pro games (and countless amateur games) are available on servers like KGS in a nice computer-digestible format.
  - komaromy 10y ago
    I'm not too familiar with Go, but in chess we have millions of text-format games and very little real-time footage.
  - SamBam 10y ago
    Again, that doesn't make sense because the moves have almost certainly already been typed out by somebody. It's the same in chess. There are databases containing millions of games.
gwern 10y ago
> Google's AlphaGo also learned from screen pixels. It did not. It received the state of the board as one array, another board state array for capture/komi (since Go does have global state which is not visible purely from the board representation), and a few additional features to help it out with stuff like ladders. It was architected with convolutional layers, but over the Go grid, not pixels. See the AlphaGo paper pg11 for the exact structure of the input: http://www.postype.com/files/2016/04/08/16/05/03384c91046e8e72f7dc6f20d204231e.pdf http://www.postype.com/files/2016/04/08/16/05/03384c91046e8e... Could it have learned from pixels (augmented by the additional necessary global state)? Sure. But that would've been a waste of computation since the visual layout of a Go board is fixed and static, unlike Atari games.

alkalait 10y ago

Getting the game state data means deciding a-priori what features the AI should learn on. The whole point of the deep learning paradigm is to allow a machine to learn such features that enable good prediction, visualization, generation (aka. hallucination), etc. Instead, researchers have provided the raw feed input data to these agents with the hope that the learned features could be interpreted as game state data by humans.

BaronSamedi 10y ago
I would say that it is a point of deep learning rather than "the whole point". For an AI to interact with the real-world then building models from vision (as we do) makes a lot of sense. In the virtual world, however, it makes no sense. Model data is already available and the AI has no need for something as inefficient as vision. We humans have to use vision (and sound, etc) in games because we have do not have access to direct data feeds, computers have no such limitation. Why cripple the AI by imposing human limitations on it?
- lawless123 10y ago
  If they want their AGI to applicable to the real world or software with incomplete or insufficient APIS they have to do it the way they are doing it here. There isn't an API for me to check if i'm still on the foot path and not the road as i walk down the street. I can't use an API to tell me water is boiling and that i shouldn't stick my hand in it.

chongli 10y ago

If your goal is to build General AI, getting access to the game state information is cheating and ultimately self defeating because it does not generalize across games. Expert humans are able to pick up and play a brand new game with a high degree of initial success despite zero knowledge of the memory states within the game. We do this by reliance on our vast knowledge of video game and literary tropes as well as experience playing past games. I have yet to see any video game bot make use of this stuff to figure out how to play Zelda, for example.

BaronSamedi 10y ago
An AI which can learn any game from "looking" at a screen is a very ambitious goal. I doubt it is achievable in the near term. Humans can do it because of years of learning about the world. Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games such as Starcraft, LoL, and Dota--especially the latter 2 since they are team games.
- lawless123 10y ago
  That's not their goal though.They want AGI.
- chongli 10y ago
  Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games Those sorts of goals are better suited to individual hobbyists. OpenAI is a blue sky research project set up by billionaires with the goal of improving all of mankind. I hope I'm not being too uncharitable when I say that your comment reminds me of those who scorned the Apollo program for its ambition.
  - BaronSamedi 10y ago
    I support research in AI of all kinds, however, its history is one of vastly over-promising and under-delivering. Modest but demonstrable progress is better, in my view, than yet more hype.

pakl 10y ago

If the goal is to have AI that is aware of the real world (even a chatbot), then using game state is a crutch that doesn't help us solve the real problem.