4 ms·

Aren't these quite different tasks, though? There's a big difference in 'learning to play a specific game well' vs. 'learning to play arbitrary games'; such a b

by cryptoz 12y ago

Aren't these quite different tasks, though? There's a big difference in 'learning to play a specific game well' vs. 'learning to play arbitrary games'; such a big difference that I think they're entirely different disciplines. Correct me if I'm wrong, but the software in the research you reference was given the ruleset to the game, right? And DeepMind's software is not given that information, I think. I doubt they intentionally omitted that work, I think it's more likely they didn't consider it relevant enough.

fragsworth 12y ago
They are different tasks, but I'm not seeing any really clear descriptions of what DeepMind can and cannot do. It's possible that their software is only good at a very specific kind of thing that happened to be in line with what Google wants. And for all we know, they could be at a complete loss as to how to progress. I mean, to what extent did they restrict what it means to be an "arbitrary game"? I highly doubt their software can play Pictionary, for instance, but I haven't found anything that really explains their limitations. Because of this, I am leaning towards the cynical and assume it's just hype, and not actually that incredible.
Houshalter 12y ago
It's basically the same algorithm, or at least very similar. The main difference is they use huge neural networks running on GPUs, and they feed it raw video data, rather than the game board state directly. It's not any less impressive though, to my knowledge no one had done anything like that before. That is, beating video games with raw video data and reinforcement learning.
- jessriedel 12y ago
  Did they hard-code the rules of backgammon into the software, or only the board state? I think there's a sort of conceptual ladder visual input --> game state --> games rules --> game strategy and it's very important to specify which rungs the software started on.
  - JabavuAdams 12y ago
    From what I've read, DeepMind's approach is to just feed in the raw pixel data and the score. No rules, or anything like that.
  - Houshalter 12y ago
    Just the position of the pieces on the board. They did give it some other features to help it. I forget what they were though, but just simple stuff that was calculated directly from the board state.
mturmon 12y ago
Thanks for the correction. The "arbitrary" qualifier is not in TFA, but (as, indeed, you said) that's the point of the demo, e.g.: https://www.youtube.com/watch?v=EfGD2qveGdQ https://www.youtube.com/watch?v=EfGD2qveGdQ Note that they're using just the video signal from the game as input. It's really a sad comment on the state of reporting at MIT Tech Review that you learn more about the tech from a youtube video than from an article. (My complaint is not with the DeepMind people, it's with the article, which should put the work in context.)
- jessriedel 12y ago
  I like your comments and I am all about holding Tech Review to a high standard, but I think I am going to side with them on this. The key part is "from scratch". I'd venture that there are lots of AI projects that similarly relevant precursors to DeepMind, all of which (including your Backgammon examples) do not actually accomplish the same from-scratch abilities described.
  - mturmon 12y ago
    The thing about Tesauro's backgammon work that excited the community is that the system trained by playing itself (http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node108.html http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node108.htm... -- "To apply the learning rule we need a source of backgammon games. Tesauro obtained an unending sequence of games by playing his learning backgammon player against itself."). Also, it didn't use an elaborate set of features and heuristics adapted for backgammon, just a simple representation of the state of the board (a list of 0/1 variables encoding how many pieces of each color are on each position). This is pretty close to "from scratch", and I think the article would have done well to point out what is actually new here.
- gumby 12y ago
  > It's really a sad comment on the state of reporting at MIT Tech Review that.. I feel compelled to point out that the only connection between the "MIT" tech review and MIT is that the magazine licenses the name from the alumni association. It's how the alumni associations funds itself and every MIT grad gets a lifetime subscription to a version of the magazine with the alumni notes bound into the back. I doubt many of us read it. I don't know how many people other than MIT grads read it, but I would imagine vanishingly few. A friend of mine calls it "the magazine of things that will never happen" which I think is dead on. It's a shame because the editor, Jason Pontin, as actually a good guy so it's surprising that the magazine continued to suck after he took it over. There are many reasons to criticize MIT (don't I know it!) but you can't judge the institute by this magazine.

by cryptoz 12y ago

fragsworth 12y ago

They are different tasks, but I'm not seeing any really clear descriptions of what DeepMind can and cannot do. It's possible that their software is only good at a very specific kind of thing that happened to be in line with what Google wants. And for all we know, they could be at a complete loss as to how to progress. I mean, to what extent did they restrict what it means to be an "arbitrary game"? I highly doubt their software can play Pictionary, for instance, but I haven't found anything that really explains their limitations. Because of this, I am leaning towards the cynical and assume it's just hype, and not actually that incredible.

Houshalter 12y ago

It's basically the same algorithm, or at least very similar. The main difference is they use huge neural networks running on GPUs, and they feed it raw video data, rather than the game board state directly. It's not any less impressive though, to my knowledge no one had done anything like that before. That is, beating video games with raw video data and reinforcement learning.

jessriedel 12y ago
Did they hard-code the rules of backgammon into the software, or only the board state? I think there's a sort of conceptual ladder visual input --> game state --> games rules --> game strategy and it's very important to specify which rungs the software started on.
- JabavuAdams 12y ago
  From what I've read, DeepMind's approach is to just feed in the raw pixel data and the score. No rules, or anything like that.
- Houshalter 12y ago
  Just the position of the pieces on the board. They did give it some other features to help it. I forget what they were though, but just simple stuff that was calculated directly from the board state.

mturmon 12y ago

Thanks for the correction. The "arbitrary" qualifier is not in TFA, but (as, indeed, you said) that's the point of the demo, e.g.: https://www.youtube.com/watch?v=EfGD2qveGdQ https://www.youtube.com/watch?v=EfGD2qveGdQ Note that they're using just the video signal from the game as input. It's really a sad comment on the state of reporting at MIT Tech Review that you learn more about the tech from a youtube video than from an article. (My complaint is not with the DeepMind people, it's with the article, which should put the work in context.)

jessriedel 12y ago
I like your comments and I am all about holding Tech Review to a high standard, but I think I am going to side with them on this. The key part is "from scratch". I'd venture that there are lots of AI projects that similarly relevant precursors to DeepMind, all of which (including your Backgammon examples) do not actually accomplish the same from-scratch abilities described.
- mturmon 12y ago
  The thing about Tesauro's backgammon work that excited the community is that the system trained by playing itself (http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node108.html http://webdocs.cs.ualberta.ca/~sutton/book/ebook/node108.htm... -- "To apply the learning rule we need a source of backgammon games. Tesauro obtained an unending sequence of games by playing his learning backgammon player against itself."). Also, it didn't use an elaborate set of features and heuristics adapted for backgammon, just a simple representation of the state of the board (a list of 0/1 variables encoding how many pieces of each color are on each position). This is pretty close to "from scratch", and I think the article would have done well to point out what is actually new here.
gumby 12y ago
> It's really a sad comment on the state of reporting at MIT Tech Review that.. I feel compelled to point out that the only connection between the "MIT" tech review and MIT is that the magazine licenses the name from the alumni association. It's how the alumni associations funds itself and every MIT grad gets a lifetime subscription to a version of the magazine with the alumni notes bound into the back. I doubt many of us read it. I don't know how many people other than MIT grads read it, but I would imagine vanishingly few. A friend of mine calls it "the magazine of things that will never happen" which I think is dead on. It's a shame because the editor, Jason Pontin, as actually a good guy so it's surprising that the magazine continued to suck after he took it over. There are many reasons to criticize MIT (don't I know it!) but you can't judge the institute by this magazine.