5 ms·

> "By defeating theDota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcementlearning can achieve superhuman performance on a diffi

> "By defeating theDota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcementlearning can achieve superhuman performance on a difficult task." Isn't this a bit of a leap though, it came with massive caveats to the game: 1. Drafted from a 17 hero pool. 17 out of 115?? 2. No summons or illusions. Again drastically reduce possibilities in game. 3. This AI is trained against real players for years, so it has enormous experience against this type of opponent. The opposite applies to humans who never compete against bots, so have no experience against this type of opponent. If I recall correctly the more people played against the bots the better humans performed in successive games. Winning two games with all these restrictions and caveats is still impressive, but it feels like they overstate things. Not to mention the flawless mechanics and communication between the bots...

justicezyx 7y ago
Plus, the match happened right after OG had roster changes, probably the weakest point of the team's history, well below the later achieved TI champion status, which is long after this match was held. Well, anyway, I am OK with some degree of PR additions...
dx87 7y ago
I don't think it's overstating it too much. Even with the restrictions, the bots are still performing well above the capabilities of the vast majority of human players. The restriction on illusions and summons was also for the benefit of the players; the OpenAI team didn't want the bots to win through flawless micro skills. For point 3, even though the players didn't get to train against bots, they have the advantage of being able to learn and react accordingly. Since the bots can only learn when their models are being trained, they're trivial to beat if you use a novel technique that they haven't seen before.
- reroute1 7y ago
  The paper mentions not using illusions and summons because of the added technical complexity required to make them work: "We removed these to avoid the added technical complexity of enabling the agent to control multiple units." So maybe it could be to their advantage, but it's also not something that could be easily technically accomplished at this time?
  - willhk 7y ago
    The work I've seen recently from them on multi-agent play seems to indicate this is a problem they're very successfully working on. https://youtu.be/kopoLzvh5jY https://youtu.be/kopoLzvh5jY
- gdxhyrd 7y ago
  If you want the bot to avoid winning by micro, you add delays, cooldowns on interactions, imperfection on clicking, misclicks, etc. on the level of a human pro. In any case, simplifying the game is usually done to make training far cheaper.
  - john-radio 7y ago
    But the human competitors also require large amounts of training in order to competently play Dota 2, and their training is not simplified that training in a similar way. I realize that "fairness" is not really the point of having humans play against bots, but doesn't it damage the usefulness of the comparison from having them perform a human activity?
- swfsql 7y ago
  just that by not leaving players walking randomly is already an astonishing accomplishment to my simple eyes.
joefkelley 7y ago
People tend to focus on what has been left out, but think about what they actually did learn about: Drafting from that pool, item and skill builds, last-hitting, creep aggro, laning in general, jungling, item and spell usage, ganking, team-fight positioning, pushing objectives, warding, map control, farm priority, when to retreat vs engage. All of these require an understanding of micro vs macro goals and how they relate. Surely this qualifies as "a difficult task."
- hychoi99 7y ago
  Dota's built-in bot can already do most of that
  - me_me_me 7y ago
    Dota bots are useless.Can be easily exploited and defeated. And that's being said by someone who is on the worse send of bell curve :)
the8472 7y ago
> This AI is trained against real players for years, AIUI the bulk of the training was self-play. > Not to mention the flawless mechanics and communication between the bots... The bots had no communication channel. > The opposite applies to humans who never compete against bots That was covered by the openai open matches. humans could play against the AI for several days and find exploitable flaws. Most didn't, a handful did. That is pretty impressive considering that humans can learn while the AI is frozen during those matches.
- reroute1 7y ago
  > AIUI the bulk of the training was self-play. Maybe but it also has played against humans in many instances across many years time span. > The bots had no communication channel. The execution between them was flawless though, acting as one mind whereas a team of human has to communicate ideas. It's a clear advantage, but it just feels hacky in a way since it's not really comparing the same thing. 5 humans are a diverse group of people. Maybe it's not fair to knock the bots for this behavior though. > That was covered by the openai open matches. humans could play against the AI for several days and find exploitable flaws. Most didn't, a handful did. That is pretty impressive considering that humans can learn while the AI is frozen during those matches. I'm talking about playing against OG, who didn't spend days playing against bots. Beating regular players is great, but not the accomplishment they are portraying. The bots had a pretty specific playstyle, and if pros had dedicated time to beating them I think there would be different results.
  - the8472 7y ago
    > Maybe but it also has played against humans in many instances across many years time span. Those are evaluation matches, they don't feed into the training data. > The bots had a pretty specific playstyle, and if pros had dedicated time to beating them I think there would be different results. Well, the pros could have joined the open games too, I don't know if any did.
    - reroute1 7y ago
      Yea I think you are right about the training data. So that would be more impressive. I still would like to see a team dedicate some time to it, but that's really not in the interest of a pro team. Also the things that affect real tournaments like drafting would be really interesting to see some day also as that is a huge part and considered a more cerebral aspect of the game.

justicezyx 7y ago

Plus, the match happened right after OG had roster changes, probably the weakest point of the team's history, well below the later achieved TI champion status, which is long after this match was held. Well, anyway, I am OK with some degree of PR additions...

dx87 7y ago

I don't think it's overstating it too much. Even with the restrictions, the bots are still performing well above the capabilities of the vast majority of human players. The restriction on illusions and summons was also for the benefit of the players; the OpenAI team didn't want the bots to win through flawless micro skills. For point 3, even though the players didn't get to train against bots, they have the advantage of being able to learn and react accordingly. Since the bots can only learn when their models are being trained, they're trivial to beat if you use a novel technique that they haven't seen before.

reroute1 7y ago
The paper mentions not using illusions and summons because of the added technical complexity required to make them work: "We removed these to avoid the added technical complexity of enabling the agent to control multiple units." So maybe it could be to their advantage, but it's also not something that could be easily technically accomplished at this time?
- willhk 7y ago
  The work I've seen recently from them on multi-agent play seems to indicate this is a problem they're very successfully working on. https://youtu.be/kopoLzvh5jY https://youtu.be/kopoLzvh5jY
gdxhyrd 7y ago
If you want the bot to avoid winning by micro, you add delays, cooldowns on interactions, imperfection on clicking, misclicks, etc. on the level of a human pro. In any case, simplifying the game is usually done to make training far cheaper.
- john-radio 7y ago
  But the human competitors also require large amounts of training in order to competently play Dota 2, and their training is not simplified that training in a similar way. I realize that "fairness" is not really the point of having humans play against bots, but doesn't it damage the usefulness of the comparison from having them perform a human activity?
swfsql 7y ago
just that by not leaving players walking randomly is already an astonishing accomplishment to my simple eyes.

joefkelley 7y ago

People tend to focus on what has been left out, but think about what they actually did learn about: Drafting from that pool, item and skill builds, last-hitting, creep aggro, laning in general, jungling, item and spell usage, ganking, team-fight positioning, pushing objectives, warding, map control, farm priority, when to retreat vs engage. All of these require an understanding of micro vs macro goals and how they relate. Surely this qualifies as "a difficult task."

hychoi99 7y ago
Dota's built-in bot can already do most of that
- me_me_me 7y ago
  Dota bots are useless.Can be easily exploited and defeated. And that's being said by someone who is on the worse send of bell curve :)

the8472 7y ago

> This AI is trained against real players for years, AIUI the bulk of the training was self-play. > Not to mention the flawless mechanics and communication between the bots... The bots had no communication channel. > The opposite applies to humans who never compete against bots That was covered by the openai open matches. humans could play against the AI for several days and find exploitable flaws. Most didn't, a handful did. That is pretty impressive considering that humans can learn while the AI is frozen during those matches.

reroute1 7y ago
> AIUI the bulk of the training was self-play. Maybe but it also has played against humans in many instances across many years time span. > The bots had no communication channel. The execution between them was flawless though, acting as one mind whereas a team of human has to communicate ideas. It's a clear advantage, but it just feels hacky in a way since it's not really comparing the same thing. 5 humans are a diverse group of people. Maybe it's not fair to knock the bots for this behavior though. > That was covered by the openai open matches. humans could play against the AI for several days and find exploitable flaws. Most didn't, a handful did. That is pretty impressive considering that humans can learn while the AI is frozen during those matches. I'm talking about playing against OG, who didn't spend days playing against bots. Beating regular players is great, but not the accomplishment they are portraying. The bots had a pretty specific playstyle, and if pros had dedicated time to beating them I think there would be different results.
- the8472 7y ago
  > Maybe but it also has played against humans in many instances across many years time span. Those are evaluation matches, they don't feed into the training data. > The bots had a pretty specific playstyle, and if pros had dedicated time to beating them I think there would be different results. Well, the pros could have joined the open games too, I don't know if any did.
  - reroute1 7y ago
    Yea I think you are right about the training data. So that would be more impressive. I still would like to see a team dedicate some time to it, but that's really not in the interest of a pro team. Also the things that affect real tournaments like drafting would be really interesting to see some day also as that is a huge part and considered a more cerebral aspect of the game.