That's pretty cool (especially the ending of Tetris!), but the crucial difference is that it's trying to learn by observing what happens during a known-good run

That's pretty cool (especially the ending of Tetris!), but the crucial difference is that it's trying to learn by observing what happens during a known-good run, then it sounds like some game-specific learning is performed. In contrast, the Arcade Learning Environment (the benchmark used in TFA) is completely autonomous; there is no human guidance or game-specific learning: anything it learns must work well across all of the games (ie. it's either generally useful, or it gets surrounded by a bunch of conditionals). In particular, the author of your link says he's surprised that the algorithm can sometimes get further than twice the length of his pre-recorded training sequence.