5 ms·

Does this really learn a game engine that a human user could play the game with? I can't tell from the article or the paper. It sounds like it learns rules from

Does this really learn a game engine that a human user could play the game with? I can't tell from the article or the paper. It sounds like it learns rules from how the objects are seen to move and tries to replay that from the learned rules. I'm not sure what the point of that would be, when the video it learned from already replays without error. If someone has more insight let me know what I'm missing.

yorwba 9y ago
The rules it learns take player input into account, so yes, it should be possible to play the replica just like the original game (modulo glitches).
- YeGoblynQueenne 9y ago
  The video frames it learns from have no information about player input ("press X to jump"). So I don't see how it can learn a function from player inputs to game engine states- without which there's no way to actually play the game, let alone in a different way than the original.
  - yorwba 9y ago
    Maybe I'm misunderstanding the paragraph in the paper that begins with The final type of neighbor that the system handles, changes a rule from being handled normally to being considered a “control” rule. This handles the case of rules that the player makes decisions upon. After rereading it I'm no longer sure of my previous interpretation. Maybe they are generating these rules conditional on the player input without telling the training process what the input was. But they evaluate the realism of the learned engine against the original by training a reinforcement-learning agent to play the game, which means that they somehow connected those conditional rules to their corresponding input values.
    - YeGoblynQueenne 9y ago
      The way it's reported is confusing but after a second, more careful read-through, I still can't see that the "engine" they learn is anything like an ordinary game engine that can be compared by a human player's inputs. Specifically, they don't mention anywhere that their engine was played by a human- to prove its playability. Maybe that's an omission or they didn't consider it important, but it makes it very hard to figure out if this is at all possible. Finally, there's this bit on the 4th page (next to the algorithm box) where they say: "At this point we can guarantee that we have an engine that can predict the entire sequence of frames from the initial frame. Notably this means that the engine can only reflect changes that actually occur in the input sequence of parsed frames". I take this to mean that, given a specific starting frame, their engine wil always predict the same sequence- which must mean its prediction does not take into account player input. But it's true that this point is very hard to figure out. >> Maybe they are generating these rules conditional on the player input without telling the training process what the input was. Basically, yeah- that's my understanding.
    - YeGoblynQueenne 9y ago
      Ah, as to the reinforcement-learning agent- this is also very confusing (because of the use of "engine" to mean both their engine and the Infinite Mario engine used in the last experiment) but I think this RL agent was created to play the Infinite Mario engine, not their engine: For a baseline we drew on the “Infinite Mario” engine used for the Mario level playing AI competition [Togelius et al., 2010] and constructed a simple reinforcement learning agent for the engine utilizing the value iteration algorithm. We further constructed two variations on this agent. The first made use of the native forward simulation available in the “Infinite Mario” engine to construct an initial policy rather than relying on the typical approach of a uniformly random initial policy. (...) In comparison, we constructed an agent that instead made use of forward simulation according to the learned game engine (with numerical values and names converted to the appropriate equivalent in the “Infinite Mario” engine). I think this means that they used a) a ready-made "forward simulation" agent included in the Infinite Mario engine, b) a RL agent they trained and c) a "forward simulation" agent _using their trained engine values as input_. But all three agents were playing the game on the Infinite Mario engine- not on their learned engine. Their learned engine only served to generate inputs for the "forward simulation". Which must mean that their engine cannot be played, in the traditional sense, for example by connecting its inputs to the outputs of an AI player, or by giving a controller to a human. If it could, they would most probably be demonstrating this ability by having an AI agent play a game of Mario on their learned engine. This doesn't seem to be possible. Again- that's my reading and I too am not 100% sure about it, because it's really not made clear in the paper.

yorwba 9y ago

The rules it learns take player input into account, so yes, it should be possible to play the replica just like the original game (modulo glitches).

YeGoblynQueenne 9y ago
The video frames it learns from have no information about player input ("press X to jump"). So I don't see how it can learn a function from player inputs to game engine states- without which there's no way to actually play the game, let alone in a different way than the original.
- yorwba 9y ago
  Maybe I'm misunderstanding the paragraph in the paper that begins with The final type of neighbor that the system handles, changes a rule from being handled normally to being considered a “control” rule. This handles the case of rules that the player makes decisions upon. After rereading it I'm no longer sure of my previous interpretation. Maybe they are generating these rules conditional on the player input without telling the training process what the input was. But they evaluate the realism of the learned engine against the original by training a reinforcement-learning agent to play the game, which means that they somehow connected those conditional rules to their corresponding input values.
  - YeGoblynQueenne 9y ago
    The way it's reported is confusing but after a second, more careful read-through, I still can't see that the "engine" they learn is anything like an ordinary game engine that can be compared by a human player's inputs. Specifically, they don't mention anywhere that their engine was played by a human- to prove its playability. Maybe that's an omission or they didn't consider it important, but it makes it very hard to figure out if this is at all possible. Finally, there's this bit on the 4th page (next to the algorithm box) where they say: "At this point we can guarantee that we have an engine that can predict the entire sequence of frames from the initial frame. Notably this means that the engine can only reflect changes that actually occur in the input sequence of parsed frames". I take this to mean that, given a specific starting frame, their engine wil always predict the same sequence- which must mean its prediction does not take into account player input. But it's true that this point is very hard to figure out. >> Maybe they are generating these rules conditional on the player input without telling the training process what the input was. Basically, yeah- that's my understanding.
  - YeGoblynQueenne 9y ago
    Ah, as to the reinforcement-learning agent- this is also very confusing (because of the use of "engine" to mean both their engine and the Infinite Mario engine used in the last experiment) but I think this RL agent was created to play the Infinite Mario engine, not their engine: For a baseline we drew on the “Infinite Mario” engine used for the Mario level playing AI competition [Togelius et al., 2010] and constructed a simple reinforcement learning agent for the engine utilizing the value iteration algorithm. We further constructed two variations on this agent. The first made use of the native forward simulation available in the “Infinite Mario” engine to construct an initial policy rather than relying on the typical approach of a uniformly random initial policy. (...) In comparison, we constructed an agent that instead made use of forward simulation according to the learned game engine (with numerical values and names converted to the appropriate equivalent in the “Infinite Mario” engine). I think this means that they used a) a ready-made "forward simulation" agent included in the Infinite Mario engine, b) a RL agent they trained and c) a "forward simulation" agent _using their trained engine values as input_. But all three agents were playing the game on the Infinite Mario engine- not on their learned engine. Their learned engine only served to generate inputs for the "forward simulation". Which must mean that their engine cannot be played, in the traditional sense, for example by connecting its inputs to the outputs of an AI player, or by giving a controller to a human. If it could, they would most probably be demonstrating this ability by having an AI agent play a game of Mario on their learned engine. This doesn't seem to be possible. Again- that's my reading and I too am not 100% sure about it, because it's really not made clear in the paper.