5 ms·

Misleading title. FB seems to be taking a self-learning approach, whereas hobbyists are defining bot rules themselves. Hobbyists are winning at the moment with

by vm 9y ago

Misleading title. FB seems to be taking a self-learning approach, whereas hobbyists are defining bot rules themselves. Hobbyists are winning at the moment with an approach that doesn't scale. Self-learning has a better chance of dominating the field. Edit: HN mods updated title to be less click-baity (thanks). Earlier it said "FACEBOOK QUIETLY ENTERS STARCRAFT WAR FOR AI BOTS, AND LOSES". The "and loses" at the end was misleading.

toxik 9y ago
As someone who has implemented quite a few reinforcement learning techniques and seen their limitations, I would be surprised if RL could overcome handcrafting for SC any day soon.
- AstralStorm 9y ago
  The main way the AI bots have problems is with timing. The neutral networks used have no way of encoding time dependent actions in a reasonable way. (As opposed to say a fuzzy decision tree with explicit time input.) And if you try to explicitly include it, curse of dimensionality strikes back hard. Both absolute and relative timing have to be handled. And relative since specific salient action... Plus the real reward is very sparse. Say, crippling mineral production early may or may not snowball. Likewise being a unit or two up...
  - comicjk 9y ago
    What that tells me is that they haven't yet come up with the right featurization - that is, the function that maps input data into the actual neural network node values. The appropriate featurization would include the time information but reduce its dimensionality by hard-coding some basic assumptions, of the kind that humans presumably make when processing the same data.
    - yingxie3 9y ago
      I think these guys (and most people using deep models) try to avoid hand-crafted features as much as possible.
      - eutectic 9y ago
        Alphago used several hand-crafted features as of the Nature paper, so DeepMind at least is not above a little feature engineering.
      - kastnerkyle 9y ago
        Gabriel (as well as the others on the team) have definitely looked at these areas - if things were left out/not "featurized" it was likely done via an ablation test, or showed improvement over benchmarks, or maybe just to set a baseline, as he is quoted in the main article. I don't know what techniques they used here, but I am excited to find out! On the specific issue of encoding time-dependent behaviors in models, I think it is related to a broader issue that shows up in many application areas. To me the critical factor is that these models are ruthlessly good at exploiting local dependencies and totally forgetting long-term global dependencies or respecting required structure in control/generation. This basically means it is very difficult to train long-term, time dependent behavior without tricks (early/mid/late game models, extensive handcrafting of the inputs, or using high level "macro actions"). Indeed, FAIR's recent mini-RTS engine ELF directly gives macro actions, in part to look closer at how well global strategies are really handled and remove one factor of complexity [0]. Gabriel's PhD thesis was entirely on Bayesian models for RTS AI, applied to SC:BW [1], so I am sure he is well aware of the "classic/rules based" approaches for this. [0] https://code.facebook.com/posts/132985767285406/introducing-elf-an-extensive-lightweight-and-flexible-platform-for-game-research/ https://code.facebook.com/posts/132985767285406/introducing-... [1] http://emotion.inrialpes.fr/people/synnaeve/phdthesis/phdthesis.html http://emotion.inrialpes.fr/people/synnaeve/phdthesis/phdthe...
  - eutectic 9y ago
    I suspect you might be able to do surprisingly well with just a few simple features, e.g. what did I last see at each position and how long ago was that, how many of each enemy unit have I seen simultaneously and at what time, etc. As to the sparsity of reward, I'm not sure this is such a big problem. Once the AI learns that e.g. 'resources are good', it can then learn how to optimize resource production. You could even give the process a head start by learning a function of time+various resources+assorted features to win rate from human games to use as the reward function.
    - NikolaeVarius 9y ago
      "Resources are good" doesn't really mean anything. Yes resources are good, but how do you know when to expand? Judging from opponents movements, you can tell if they're turtling, going for some cheese strat, or doing some build where they may not be able to respond to a aggressive expansion. Of course if you choose wrong, you lost the game.
- unpwn 9y ago
  Why do you say that? The dota2 bot open ai did earlier this year seemed pretty convincing and similar...
  - dxhdr 9y ago
    Starcraft is a much larger, more complex, more freeform game than Dota 2. It's like Go compared to chess.
    - cli 9y ago
      I disagree with this (I used to play Warcraft 3, and currently play Dota 2), but that's beside the point. The Dota 2 OpenAI is only set up for one mirror matchup (impossible in real games) involving one hero on each side, in one lane, and only for the first 10-ish minutes. This is maybe 1% of a real Dota game.
      - kazagistar 9y ago
        I think you are both correct. Starcraft has a far larger space of verbs at any given moment, and many of them can impact each other, giving it one form of complexity, while Dota2 clearly has a much more complex set of units and abilities, leading to more possibilities total, even if the number of possible actions moment to moment are more limited. But yeah, the bot was a teeny little bit of the game, impressive as it was.
    - sametmax 9y ago
      A lot of complexity in dota comes from the interactions between 10 players. Make it 10 ai having to communicate using chat. Make them pick and ban. Make all objects available. And then you'll have real complexity. With hundreds of unit x objects, jungling, roshan, cd and pick + ban, you can actually get at the sc level of complexity.
      - Retric 9y ago
        It's 10 players vs ~200 bots. So, SC is still a much more complex space. DoTA has non player bots, but they are similar to SC buildings and follow very simple rules.
smrtinsert 9y ago
Factually correct title.
tree_of_item 9y ago
Misleading? How? It's the title of the actual article, and it's what happened. Just because they'll probably win in the future because their approach is more sound, doesn't mean they didn't lose now.
- vm 9y ago
  Makes it seem like they lost in an AI bot battle. But they are in fact creating AI bots that compete against rule-based bots. Big difference.
  - tree_of_item 9y ago
    They did lose in an AI bot battle. A rule based bot is still an AI.
    - eradicatethots 9y ago
      Things can be accurate but misleading. This is a good example of misleading and accurate.
      - AstralStorm 9y ago
        Automated decision making does not an intelligence make on it's own. Call again when it can actually learn the game from limited inputs available to human players.
  - richardknop 9y ago
    AI is not just neural networks / deep learning. Those rule-based bots are AI too, just different approach (something called expert systems is subcategory of AI too).
deleted 9y ago
[deleted]

by vm 9y ago

toxik 9y ago

As someone who has implemented quite a few reinforcement learning techniques and seen their limitations, I would be surprised if RL could overcome handcrafting for SC any day soon.

AstralStorm 9y ago
The main way the AI bots have problems is with timing. The neutral networks used have no way of encoding time dependent actions in a reasonable way. (As opposed to say a fuzzy decision tree with explicit time input.) And if you try to explicitly include it, curse of dimensionality strikes back hard. Both absolute and relative timing have to be handled. And relative since specific salient action... Plus the real reward is very sparse. Say, crippling mineral production early may or may not snowball. Likewise being a unit or two up...
- comicjk 9y ago
  What that tells me is that they haven't yet come up with the right featurization - that is, the function that maps input data into the actual neural network node values. The appropriate featurization would include the time information but reduce its dimensionality by hard-coding some basic assumptions, of the kind that humans presumably make when processing the same data.
  - yingxie3 9y ago
    I think these guys (and most people using deep models) try to avoid hand-crafted features as much as possible.
    - eutectic 9y ago
      Alphago used several hand-crafted features as of the Nature paper, so DeepMind at least is not above a little feature engineering.
    - kastnerkyle 9y ago
      Gabriel (as well as the others on the team) have definitely looked at these areas - if things were left out/not "featurized" it was likely done via an ablation test, or showed improvement over benchmarks, or maybe just to set a baseline, as he is quoted in the main article. I don't know what techniques they used here, but I am excited to find out! On the specific issue of encoding time-dependent behaviors in models, I think it is related to a broader issue that shows up in many application areas. To me the critical factor is that these models are ruthlessly good at exploiting local dependencies and totally forgetting long-term global dependencies or respecting required structure in control/generation. This basically means it is very difficult to train long-term, time dependent behavior without tricks (early/mid/late game models, extensive handcrafting of the inputs, or using high level "macro actions"). Indeed, FAIR's recent mini-RTS engine ELF directly gives macro actions, in part to look closer at how well global strategies are really handled and remove one factor of complexity [0]. Gabriel's PhD thesis was entirely on Bayesian models for RTS AI, applied to SC:BW [1], so I am sure he is well aware of the "classic/rules based" approaches for this. [0] https://code.facebook.com/posts/132985767285406/introducing-elf-an-extensive-lightweight-and-flexible-platform-for-game-research/ https://code.facebook.com/posts/132985767285406/introducing-... [1] http://emotion.inrialpes.fr/people/synnaeve/phdthesis/phdthesis.html http://emotion.inrialpes.fr/people/synnaeve/phdthesis/phdthe...
- eutectic 9y ago
  I suspect you might be able to do surprisingly well with just a few simple features, e.g. what did I last see at each position and how long ago was that, how many of each enemy unit have I seen simultaneously and at what time, etc. As to the sparsity of reward, I'm not sure this is such a big problem. Once the AI learns that e.g. 'resources are good', it can then learn how to optimize resource production. You could even give the process a head start by learning a function of time+various resources+assorted features to win rate from human games to use as the reward function.
  - NikolaeVarius 9y ago
    "Resources are good" doesn't really mean anything. Yes resources are good, but how do you know when to expand? Judging from opponents movements, you can tell if they're turtling, going for some cheese strat, or doing some build where they may not be able to respond to a aggressive expansion. Of course if you choose wrong, you lost the game.
unpwn 9y ago
Why do you say that? The dota2 bot open ai did earlier this year seemed pretty convincing and similar...
- dxhdr 9y ago
  Starcraft is a much larger, more complex, more freeform game than Dota 2. It's like Go compared to chess.
  - cli 9y ago
    I disagree with this (I used to play Warcraft 3, and currently play Dota 2), but that's beside the point. The Dota 2 OpenAI is only set up for one mirror matchup (impossible in real games) involving one hero on each side, in one lane, and only for the first 10-ish minutes. This is maybe 1% of a real Dota game.
    - kazagistar 9y ago
      I think you are both correct. Starcraft has a far larger space of verbs at any given moment, and many of them can impact each other, giving it one form of complexity, while Dota2 clearly has a much more complex set of units and abilities, leading to more possibilities total, even if the number of possible actions moment to moment are more limited. But yeah, the bot was a teeny little bit of the game, impressive as it was.
  - sametmax 9y ago
    A lot of complexity in dota comes from the interactions between 10 players. Make it 10 ai having to communicate using chat. Make them pick and ban. Make all objects available. And then you'll have real complexity. With hundreds of unit x objects, jungling, roshan, cd and pick + ban, you can actually get at the sc level of complexity.
    - Retric 9y ago
      It's 10 players vs ~200 bots. So, SC is still a much more complex space. DoTA has non player bots, but they are similar to SC buildings and follow very simple rules.

smrtinsert 9y ago

Factually correct title.

tree_of_item 9y ago

Misleading? How? It's the title of the actual article, and it's what happened. Just because they'll probably win in the future because their approach is more sound, doesn't mean they didn't lose now.

vm 9y ago
Makes it seem like they lost in an AI bot battle. But they are in fact creating AI bots that compete against rule-based bots. Big difference.
- tree_of_item 9y ago
  They did lose in an AI bot battle. A rule based bot is still an AI.
  - eradicatethots 9y ago
    Things can be accurate but misleading. This is a good example of misleading and accurate.
    - AstralStorm 9y ago
      Automated decision making does not an intelligence make on it's own. Call again when it can actually learn the game from limited inputs available to human players.
- richardknop 9y ago
  AI is not just neural networks / deep learning. Those rule-based bots are AI too, just different approach (something called expert systems is subcategory of AI too).

deleted 9y ago

[deleted]