3 ms·
As is usual in such cases, it pays to look at the paper for all the details that the hype article leaves out. They use prior knowledge about the sprites that c
by yorwba 9y ago
As is usual in such cases, it pays to look at the paper for all the details that the hype article leaves out.
They use prior knowledge about the sprites that can appear in the game to extract them from the video frames. The extraction result is then represented as a list of facts (sprites present, their absolute and relative locations, velocities and the camera scroll position). Learned rules are applied to these facts to derive a representation of the next frame, which is then displayed.
That means that platforms and killer objects are discovered concepts, but the features that allow their discovery (relative positions) are already hardcoded. Since most games have collision-based interactions, that is likely not much of a limitation.
The greater problem will be with hidden information. If e.g. a bomb stays black for 5 seconds after being placed before it starts blinking red and then exploding, their method would be unable to learn when the bomb should start to blink, because their representation of game state has no memory. Same with objects dropped off-screen, inventory, quest status or any other information that is relevant to the game, but not always displayed on the screen.