3 ms·

I don't know that they specifically said there was no domain-specific knowledge. iirc they said they didn't "teach it the rules of dota" but they also said the

I don't know that they specifically said there was no domain-specific knowledge. iirc they said they didn't "teach it the rules of dota" but they also said the training involved "coaching". I interpret that to include showing the bot useful techniques (like creep blocking) which the AI then learned leads to higher win rates, etc.

arnioxux 9y ago
I think "coaching" just means "curriculum learning" (which is basically as you've described). But in the future coaching might indeed just be a human expert talking in natural language to teach it how to be better! https://www.youtube.com/watch?v=L_jGDV_5gPA&t=6 https://www.youtube.com/watch?v=L_jGDV_5gPA&t=6
sondr3 9y ago
They also mentioned that in the beginning the bot figured out that the best way to win was to not play the game (aka hiding in it's own base). Then it started running around wildly, dying to enemy towers in the wrong lanes at the map. So it definitely took some nudging getting it to do something more than being AFK in base.
- StavrosK 9y ago
  How did it figure that? There's pretty much no way to win if you're just staying in base.
  - Shaanie 9y ago
    Barring faction imbalance, there's a 50% chance to win by staying in base. However, I'm not sure how running around and dying could negatively impact your win percentage unless the other bot is also outside of the base.
    - StavrosK 9y ago
      I don't understand what you mean. If you stay in base, the opponent won't, and will quickly win the game.
      - tinco 9y ago
        This is during the training phase, the opponent will be a similarily trained AI.
        StavrosK 9y ago
        Ah, I see, thanks.
    - tinco 9y ago
      You might (if timed correctly) steal aggro from your creep wave which will lead you to win the game if the other hero is AFK. You might also pull aggro from their creep wave, which I think should also win you the game in most circumstances.
  - darkmighty 9y ago
    The idea is that by staying in base you don't die, while your opponent (which is still pretty bad) roams around and dies. This highlights the importance of gradual adversarial training: if the opponent was perfect, there would simply be no way to win, and hence no signal on how to gradually improve to eventually find the optimal strategy. Also shows a weakness in most training algorithms today that don't have, for example, the capability to watch their opponent closely and try to mimic it's actions in case he's better than you. Humans do this so quickly you can see Dendi already tried replicating the AI's strategy after 1 loss.
- eighthCoffee 9y ago
  It doesn't does not need any nudging. The idea is that first generations run around dying so the optimal response is to just stay put and let the opponent die. This now has basically 100% win rate so both of them do it, lowering it to 50%. The other part is that there are random variations to the strategy it uses. This it experiments - they won't stay in base forever since that is giving you only 50% winrate (based on whose creeps would push tower first). This random variation then means that the bot might go out, autoattack a creep wave and then come back to to base. Or anything similar to that. This new strategy now has almost 100% win rate and the stay in base is no longer viable. If it went out and died to a tower, it doesn't matter, it can just not use it and there will be period of base AFKing again, but it only takes one improvement (pushing a wave or hitting tower couple times) to make the base AFK obsolete and force new progress.
- e12e 9y ago
  > They also mentioned that in the beginning the bot figured out that the best way to win was to not play the game (aka hiding in it's own base). "A strange game. The only winning move is not to play." - AI "Joshua" in "Wargames", 1983
gnufied 9y ago
Not whole lot but some of this stuff can be seen in this video - https://www.youtube.com/watch?v=wpa5wyutpGc https://www.youtube.com/watch?v=wpa5wyutpGc I still can't quite tell if this behaviour was coached to the bot or it figured it out itself.

arnioxux 9y ago

I think "coaching" just means "curriculum learning" (which is basically as you've described). But in the future coaching might indeed just be a human expert talking in natural language to teach it how to be better! https://www.youtube.com/watch?v=L_jGDV_5gPA&t=6 https://www.youtube.com/watch?v=L_jGDV_5gPA&t=6

sondr3 9y ago

They also mentioned that in the beginning the bot figured out that the best way to win was to not play the game (aka hiding in it's own base). Then it started running around wildly, dying to enemy towers in the wrong lanes at the map. So it definitely took some nudging getting it to do something more than being AFK in base.

StavrosK 9y ago
How did it figure that? There's pretty much no way to win if you're just staying in base.
- Shaanie 9y ago
  Barring faction imbalance, there's a 50% chance to win by staying in base. However, I'm not sure how running around and dying could negatively impact your win percentage unless the other bot is also outside of the base.
  - StavrosK 9y ago
    I don't understand what you mean. If you stay in base, the opponent won't, and will quickly win the game.
    - tinco 9y ago
      This is during the training phase, the opponent will be a similarily trained AI.
      - StavrosK 9y ago
        Ah, I see, thanks.
  - tinco 9y ago
    You might (if timed correctly) steal aggro from your creep wave which will lead you to win the game if the other hero is AFK. You might also pull aggro from their creep wave, which I think should also win you the game in most circumstances.
- darkmighty 9y ago
  The idea is that by staying in base you don't die, while your opponent (which is still pretty bad) roams around and dies. This highlights the importance of gradual adversarial training: if the opponent was perfect, there would simply be no way to win, and hence no signal on how to gradually improve to eventually find the optimal strategy. Also shows a weakness in most training algorithms today that don't have, for example, the capability to watch their opponent closely and try to mimic it's actions in case he's better than you. Humans do this so quickly you can see Dendi already tried replicating the AI's strategy after 1 loss.
eighthCoffee 9y ago
It doesn't does not need any nudging. The idea is that first generations run around dying so the optimal response is to just stay put and let the opponent die. This now has basically 100% win rate so both of them do it, lowering it to 50%. The other part is that there are random variations to the strategy it uses. This it experiments - they won't stay in base forever since that is giving you only 50% winrate (based on whose creeps would push tower first). This random variation then means that the bot might go out, autoattack a creep wave and then come back to to base. Or anything similar to that. This new strategy now has almost 100% win rate and the stay in base is no longer viable. If it went out and died to a tower, it doesn't matter, it can just not use it and there will be period of base AFKing again, but it only takes one improvement (pushing a wave or hitting tower couple times) to make the base AFK obsolete and force new progress.
e12e 9y ago
> They also mentioned that in the beginning the bot figured out that the best way to win was to not play the game (aka hiding in it's own base). "A strange game. The only winning move is not to play." - AI "Joshua" in "Wargames", 1983

gnufied 9y ago

Not whole lot but some of this stuff can be seen in this video - https://www.youtube.com/watch?v=wpa5wyutpGc https://www.youtube.com/watch?v=wpa5wyutpGc I still can't quite tell if this behaviour was coached to the bot or it figured it out itself.