15 ms·

OpenAI Five Benchmark: Results

281 points by runesoerensen 8y ago

deleted 8y ago
[deleted]
runesoerensen 8y ago
Related discussion from yesterday's match: https://news.ycombinator.com/item?id=17693169 https://news.ycombinator.com/item?id=17693169
samfriedman 8y ago
The included video of "growing pains" - aberrant behaviors found during training - is interesting. This kind of thing is fairly common when training RL agents: some discussion of how the OpenAI team solved these issues would be nice.
- jaggederest 8y ago
  Video link on youtube: https://www.youtube.com/watch?v=xiyUSI-TEgo https://www.youtube.com/watch?v=xiyUSI-TEgo
minimaxir 8y ago
During the thread from the competition there were a few comments suggesting that the neural network could easily be run on a personal computer. Given the massive network architecture linked in the post (https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchmark_results/network_diagram_08_06_2018.pdf https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchm...), I am rather curious what hardware was used to make predictions for the Benchmark match. Especially due to the 2048 unit LSTM. The model outputs are also interesting; they're all discrete actions (even movements), no continuous outputs (aside from win probability).
- bllguo 8y ago
  not sure I follow; what's so interesting there? Wouldn't a continuous output ultimately have to be translated into a discrete action anyway?
  - minimaxir 8y ago
    The big ones are the X and Y offsets/moves; you'd expect it the output to be an explicit coordinate to move to/act upon (e.g. a specific value in the -400 to 400 space), but per the original announcement (https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/), there's a 9v9 grid space. (-400 to 400 by increments of 100 each axis) Although making discrete choices 2-3 times a second is indistinguishable from continuous movement anyways.
- sgillen 8y ago
  I think it just comes down to the fact that discrete action spaces are easier to work with for most RL frameworks (not sure what exactly open AI is using here though).
doctorpangloss 8y ago
Would you be interested in developing an agent for Hearthstone?
- ralfd 8y ago
  Already done, sort of. Blizzard was not amused, as botting makes the game less fun, so it wasnt released. https://elie.net/blog/hearthstone/i-am-a-legend-hacking-hearthstone-with-machine-learning-defcon-talk-wrap-up https://elie.net/blog/hearthstone/i-am-a-legend-hacking-hear...
- dgant 8y ago
  There's a substantial community around Hearthstone AI: https://hearthsim.info/ https://hearthsim.info/
atx7 8y ago
I like how their priority is not removing the significant restrictions of couriers, items and heroes, rather playing a team of professionals on a "custom game" based on dota heroes
- apeace 8y ago
  If you watch the match or read their blog, you’ll see they frequently say removing those restrictions is their top priority. They simply wanted to see what level the AI is already at by playing top-level humans for the first time. Hence the name “benchmark”. In fact, they have been removing restrictions at a fairly rapid pace already.
  - deleted 8y ago
    [deleted]
- deleted 8y ago
  [deleted]
- iotb 8y ago
  As a seasoned player of Dota watching the replays, I can see exactly what's going on. There's lots of snowballing/cheese which are the least common denominators of gameplay and least intellectual. Such dynamics would no doubt be dominated by a bot like program which is why they choose them. I could imagine that people who aren't seasoned players are impressed by this but this reads like an average wood league "south" game. People who play Dota will know what I'm referring to in reference to "south". In such games, the strategy is simple : You beat out the brute-force cheese snowballing w/ intellect. However, this is where they no doubt decided to put up the artificial red tape. You're 100% right and being down voted for it as I no doubt will. I am not impressed because I've played a ton of games and observed a ton of dynamics. You can tell in the first 10min what level and type of game play is occurring and it isn't of the intellectual variant. It's of the bot variant. Broader Dota was designed and constantly updated to combat such gameplay. When people settle on cheese strategies or exploits, they intentionally change the game balance to prevent it. I'm more interested in how this is achieved than what it has been tuned to try to trick people into believing. Looking under the hood, I see the same thing when I look across all Weak AI solutions : Lots of hand coded steering functions with dynamic weights populated by incredible amounts of brute-force random searching (past a human lifetime of average gameplay). While this may secure headlines and funding, this is miles away from the real direction you have to go towards Strong AI. It doesn't seem many well funded/popularized groups understand this thus will ultimately end up fooling themselves. Point out this truth and you get down votes. Essentially the broader community plugging their ears which is why a large number of these efforts are due for a significant failure.
crsv 8y ago
In case anyone would like to watch a VOD of the event, it can be found here: https://www.twitch.tv/openai/video/293517383 https://www.twitch.tv/openai/video/293517383 It was a really wonderfully done event overall and there's several interviews throughout with different folks from the OpenAI team.
- abhiminator 8y ago
  YouTube mirrors: Game 1&2 [0] Game 3 [1] [0] https://www.youtube.com/watch?v=eaBYhLttETw https://www.youtube.com/watch?v=eaBYhLttETw [1] https://www.youtube.com/watch?v=_QQYaVUODkE https://www.youtube.com/watch?v=_QQYaVUODkE
  - ufo 8y ago
    (These are unofficial highlights btw, not mirrors)
apeace 8y ago
If anyone from OpenAI sees this: The one thing that really surprised me yesterday was when OpenAI Five seemed to pause the game. The commentators speculated that “it was learning” because the humans had paused in game 1 due to lag. I assume that’s not right, as OpenAI five is not training itself as it’s playing (wouldn’t make much sense to add one more game sample to the billions it has already trained on). I thought it was interesting that the commentators had this misconception, and was wondering what lead to OpenAI Five hitting pause.
- deleted 8y ago
  [deleted]
- nicklovescode 8y ago
  I think the bot was setup to pause if it ran into any lag or other minor issues (was at the event and heard this from someone there)
- gdb 8y ago
  A network blip caused all the players to drop from the game. Incidentally, we'd just changed the code a few days earlier from "automatically surrender when a human disconnects" to "do nothing if a human disconnects; automatically pause if all humans disconnect". Had done a lot of advance planning for what might break!
  - abhiminator 8y ago
    Off the tangent question here -- Do you have any plans to pit OpenAI Five against the winners of this year's 'The International 8' tournament scheduled for later this month? Folks in the DotA 2 community are going crazy over this possibility, would be incredible if this happens.
    - Dibes 8y ago
      Looks like they are at least playing pros from TI8[0]. [0]: "These results give us confidence in moving to the next phase of this project: playing a team of professionals at The International later this month." from https://blog.openai.com/openai-five-benchmark-results/ https://blog.openai.com/openai-five-benchmark-results/ at the bottom
iMuzz 8y ago
Can someone ELI5 what the significance of this is? Why was Dota chosen as the game for an AI to get good at? Is Dota more difficult than Go? Why or why not?
- krotton 8y ago
  They wrote about it here (The Problem section): https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/
- crsv 8y ago
  In short, it's significant progress in the OpenAI team's quest to build really effective self trained artificial intelligence. Conquest in a problem space like this has real world applications that could translate to things like extremely effective artificial limb dexterity (according to the talking points from the event). Dota 2 was chosen because it's a nice combination of pre-determined rules (to an extent) and an extremely complex problem set of possible moves and actions. Dota's development team also was really supportive to this effort and they have an API that suits the Open AI team's purposes of interacting with the game really well. Great fit all around. Dota 2, especially when played 5v5, is orders of magnitude more complex than in terms of possible decisions and moves than Go.
- doodhwala 8y ago
  DotA is substantially different compared to Go in the following main senses: 1) In Go, you are allowed to see the entire board and pieces at all times -- it is a complete information game. On the other hand, games like DotA have partial information because you are not able to see where your opponents are and what they are doing at all times. 2) Go, Chess and many of the Atari games are single player games. OpenAI wanted to see if machine learning can be applied in a multiplayer setting where the problem needs to be solved at a global / team level. 3) DotA has so many mechanics and strategies where you have a lot of choices to be made. One of the challenges is whether one can look at the overall outcome of the game and reason about what particular choices went into the winning or losing of the game. This (long event horizon) makes it extremely difficult to learn such models. OpenAI interfaced with a complicated game like DotA through an API provided by Valve which made it lot easier. Instead of seeing the game screen, they got snapshots of data of around 35KB per observation (co-ordinates of heroes, creeps etc). In the absence of this API, they would have had to use substantially more computational resources to render the in game graphics and this would also make the training process extremely slow. This benchmark was an experiment that demonstrated that tackling such a class of problems is indeed possible (given a lot of computational resources and an environment to train in). During the interviews in between and after the games, they mentioned that the algorithms that they have used can have many applications in all fields a̶l̶t̶h̶o̶u̶g̶h̶ ̶s̶p̶e̶c̶i̶f̶i̶c̶ ̶e̶x̶a̶m̶p̶l̶e̶s̶ ̶w̶e̶r̶e̶ ̶n̶o̶t̶ ̶p̶r̶o̶v̶i̶d̶e̶d̶ @crsv's comment describes the example they talked about. Perhaps an ELI15, but hope this helps.
johncoogan 8y ago
I wonder how long it will be for them to build a version that actually uses the screen images as opposed to all the data from the Bot API. Seems like it would be a lot harder when you add an image processing step and restrict processing of information from across the map without actually scrolling over there.
- minimaxir 8y ago
  Per the architecture, the model does use a CNN to process minimap data.
  - sgillen 8y ago
    Yes but the comment you're replying to is saying that it would be interesting to only rely on information from visual input on the screen, rather than on getting (for example) the absolute XY position of every player as a direct input to the network.
    - wetpaws 8y ago
      It's like writing a bot that would play chess using video feed from camera. Yes, it's doable and an interesting problem on it's own but completely unrelated to what openai is doing.
      - sgillen 8y ago
        yes I see what you are saying with the chess example, but I think in this case adding the visual layer actually adds interesting problems related to what openAI is trying to do. See my reply to lawrenceyan above.
      - adamrezich 8y ago
        Chess is a poor example because it's a turn-based game whereas Dota is real-time and incredibly fast-paced. Visually parsing a chessboard to know which piece is which is trivial, and also you have all the time in the world to do it (between turns). In Dota things happen so quickly, particle effects pop off all over the place, and through all of it, you have to constantly manually re-place your camera in the optimal position. In chess, both players always have perfect information about the game-state, and this is far from the case in Dota. OpenAI does account for fog of war, so it's not COMPLETELY omniscient, but it is still more omniscient than human players ever have the ability to be, without having to fiddle with the camera etc.
_0ffh 8y ago
I think the third game would probably have been a lot more tense, if they'd let the OAI5 make the last one or two picks by itself.
- gpm 8y ago
  I bet they weren't quite sure what they had to do to make the game even. The previous rounds were hugely in favor of the ai and it sounds like they don't have much experience playing against humans at this level.
- make3 8y ago
  I think they wanted to show that the humans could win one
KaoruAoiShiho 8y ago
Did you think the AI would win despite the adversarial draft?
- backpropaganda 8y ago
  I'm curious to know if the AI can win with a 50-50 draft. It can be argued that the main reason the humans lost was because they didn't have a feel for the meta of this 18-hero version of Dota, while the bots did. What if you give the humans an equal chance at the game wrt the draft? That would have made for a much more interesting game than having the humans play a game they've never played before in their lives.
  - KaoruAoiShiho 8y ago
    It seems likely that actual pros instead of a mismash of commentators and community stars would be able to win game 2 even with the draft favoring the AI.
    - ufo 8y ago
      For what is worth, those commentators are actually very good players. Every year they team up in the qualifiers for the world championship and they usually make it very far.
      - iotb 8y ago
        I'm an active player with enough hours logged to interpret the game. That being said, the observations should also be obvious to even the most novice player. I don't need a commercialized and obviously favorable and biased stream of commentary clouding a much more sound analytical capability. I'm pretty sure they're not going invite commentators that point out the obvious reality as to what is going on. Per the commentators, bot like behavior gets extra positive humanized characterization and is marked as intelligent. Whereas snowball-cheese is known as the lowest tier and least intelligent strategy in the game. There's even a set of memes in the community for people who attempt such strategies. People need to start doing more critical analysis of their own and stop relying on commercialized and biased information and commentary when settling on an viewpoint. Watching others play only gets you so far in understanding. When you play and you see what I'm saying for yourself, you can skim through the provided clips and understand exactly what's going on. The fact that this continues to get hyped up vs someone stating what's going on is plain sad.
a_humean 8y ago
I'm really excited to see the limits get lifted, in particular around items and couriers. The team of semi-pros/ex-pros/pros that played against the AI commented that the AI was using the highly unusual 5 invincible couriers to enable a style of play that isn't possible in normal dota. The AI's solution to dota was unrelenting aggression once a small early game lead was established after ~10 minutes. This early aggression was possible because the AI was able to ferry items (healing consumables) with such frequency that once the human team made one mistake in a team fight it wasn't possible to recover. Normally after winning an objective you are forced to reset as you have expended a lot of resources to order achieve your objective, and you are actually most vulnerable to a counter play right after you win something big. With a constant stream of healing this risk was significantly reduced. Also, the drafting is a clear limitation that needs to get lifted. The AI was essentially pursuing a "death-ball" strategy of grouping as 4 or 5 and pushing right down a lane, which can be countered by picking mobile and fast heroes that can put pressure on different parts of the map and slow down the "death-ball" by forcing reactions. However, none of those heroes were draft-able, and so the humans were forced to play the AI's strategy. The AI's strategy favors team fight coordination (laying stuns correctly, and correctly calculating whether damage through nukes is sufficient to kill any particular target) and reaction times, at which the AI was clearly superior to the human team.
- shelune 8y ago
  I think it's because the organizers want to make sure the bots will have a good performance here. Ofc OpenAI is awesome but it's impossible to cover such a complicated game as DotA within just 1 year. What they have achieved though is still awesome. I'm just slightly bugged by the fact that the developers didn't take action execution into delay consideration. The response time is 200ms but humans also need some more time to drag the mouse and click to perform the action. Their insane reactions actually make me less impressed.
  - ufo 8y ago
    I got the impression that they weren't particularly looking for a hero pool with a "deathball" meta, and the hero pool they selected had more to do with those being some of the simplest heroes to program at first. There is a lot of overlap between with the heroes they implemented and the set of introductory heroes that are recommended for new players who are playing their first games. Regarding the insane reaction, I wonder if there is a natural way to handicap the AI to more human-like reaction times. Reaction time is not a good measure on its own because human reaction time can vary a lot depending on the level of surprise. That said, my impression from watching the game was that the power of the AI had less to do with perfect reaction time and more to do with their "hive mind" coordination. If an enemy ever gets in the wrong position, they are immediately punished by a concerted attack. Humans have a harder time doing this because they need to communicate their intentions first. Sometimes each player will be focusing their attention on a different target.
sigi45 8y ago
Why did i miss it?! :( I think first it was planned for 28th now i read it one day later :/
- ufo 8y ago
  It was originally scheduled for July 28th, but was rescheduled to yesterday. Well, at least you can still watch the twitch vods :/ That said, you still have a chance to catch the match against a professional team that is going to happen sometime between the 20th and 25th.
fermienrico 8y ago
Side comment: Open AI's design is excellent and it doesn't get in the way of reading their articles. They've taken a Stripe-like design approach without getting too overboard. Well done!
- joliv 8y ago
  There’s a reason their site is Stripe-like: they hired Stripe’s creative director:) https://dribbble.com/luddep https://dribbble.com/luddep
  - dclausen 8y ago
    and Stripe's former CTO: https://www.linkedin.com/in/thegdb/ https://www.linkedin.com/in/thegdb/
YeGoblynQueenne 8y ago
>> These results show that Five is a step towards advanced AI systems which can handle the complexity and uncertainty of the real world. Great results that deserve congratulations- but they show nothing of the sort. The distance between a game and the real world is about twice as big as the distance between a game and a simulation of the real world (which a game isn't even). Winning at dota is nothing like negotiating the real world.
- red75prime 8y ago
  I can see your comments going up to "Weak AIs designing, producing and controlling agile robots is a great step, but it is a long way before they learn enough human psychology to perform as well as a door-to-door salesperson." I'm not sure what will follow.
deleted 8y ago
[deleted]
exabrial 8y ago
5 thousand lifetimes of play versus .45 lifetimes, I think the outcome was predictable!
- make3 8y ago
  yeah, but you gotta add hundred + millions of evolutionary learning to model the physical world and the actions of our peers, built in to our (mammal) brains
exabrial 8y ago
Is the ai forced to use voice communication and chat to coordinate bots?
- make3 8y ago
  there is no communication at AI between bots afaik
simonebrunozzi 8y ago
Prediction and hope: within 1-2 years, Open.AI will be able to offer a GREAT AI-powered challenge to human players across many games. I occasionally play strategy games and I am always frustrated by a very stupid AI that gets challenging only by beefing up its stats. I would love to play against a smart AI. I don't do multiplayer because: I like to play offline and/or whenever I want, and I play so rarely that most other human players wouldn't enjoy playing with me. Open.AI team: please listen!
- sgillen 8y ago
  Yes an actual good AI in Civ would vastly improve that game for me.
tehsauce 8y ago
For a research organization openai sure seems to be putting a lot of effort into PR. Beating players using a subset of the game is impressive but there is no mention whatsoever of the fact that only a subset of the game is allowed. The research is cool by itself, why do they feel the need to promote so heavily that the AI is beating human players "in the first 17 minutes of" rather than giving a realistic picture of their progress. This is how you create an AI winter.
alkonaut 8y ago
How much of an AI’s advantage is ”I/O” related and how much is actual strength in strategy/gameplay? Are the AIs actions rate-limited, delayed, and perhaps time randomized a bit so that an AI doesn’t get an advantage because it can to actions faster or better synchronized to exact times, or more responsive (say to avoid any action under a certain reaction time, all actions are scheduled 50+rand(5) after the AI queues them)?

281 points by runesoerensen 8y ago

deleted 8y ago

[deleted]

Related discussion from yesterday's match: https://news.ycombinator.com/item?id=17693169 https://news.ycombinator.com/item?id=17693169

samfriedman 8y ago

The included video of "growing pains" - aberrant behaviors found during training - is interesting. This kind of thing is fairly common when training RL agents: some discussion of how the OpenAI team solved these issues would be nice.

jaggederest 8y ago
Video link on youtube: https://www.youtube.com/watch?v=xiyUSI-TEgo https://www.youtube.com/watch?v=xiyUSI-TEgo

minimaxir 8y ago

During the thread from the competition there were a few comments suggesting that the neural network could easily be run on a personal computer. Given the massive network architecture linked in the post (https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchmark_results/network_diagram_08_06_2018.pdf https://s3-us-west-2.amazonaws.com/openai-assets/dota_benchm...), I am rather curious what hardware was used to make predictions for the Benchmark match. Especially due to the 2048 unit LSTM. The model outputs are also interesting; they're all discrete actions (even movements), no continuous outputs (aside from win probability).

bllguo 8y ago
not sure I follow; what's so interesting there? Wouldn't a continuous output ultimately have to be translated into a discrete action anyway?
- minimaxir 8y ago
  The big ones are the X and Y offsets/moves; you'd expect it the output to be an explicit coordinate to move to/act upon (e.g. a specific value in the -400 to 400 space), but per the original announcement (https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/), there's a 9v9 grid space. (-400 to 400 by increments of 100 each axis) Although making discrete choices 2-3 times a second is indistinguishable from continuous movement anyways.
sgillen 8y ago
I think it just comes down to the fact that discrete action spaces are easier to work with for most RL frameworks (not sure what exactly open AI is using here though).

doctorpangloss 8y ago

Would you be interested in developing an agent for Hearthstone?

ralfd 8y ago
Already done, sort of. Blizzard was not amused, as botting makes the game less fun, so it wasnt released. https://elie.net/blog/hearthstone/i-am-a-legend-hacking-hearthstone-with-machine-learning-defcon-talk-wrap-up https://elie.net/blog/hearthstone/i-am-a-legend-hacking-hear...
dgant 8y ago
There's a substantial community around Hearthstone AI: https://hearthsim.info/ https://hearthsim.info/

atx7 8y ago

I like how their priority is not removing the significant restrictions of couriers, items and heroes, rather playing a team of professionals on a "custom game" based on dota heroes

apeace 8y ago
If you watch the match or read their blog, you’ll see they frequently say removing those restrictions is their top priority. They simply wanted to see what level the AI is already at by playing top-level humans for the first time. Hence the name “benchmark”. In fact, they have been removing restrictions at a fairly rapid pace already.
- deleted 8y ago
  [deleted]
deleted 8y ago
[deleted]
iotb 8y ago
As a seasoned player of Dota watching the replays, I can see exactly what's going on. There's lots of snowballing/cheese which are the least common denominators of gameplay and least intellectual. Such dynamics would no doubt be dominated by a bot like program which is why they choose them. I could imagine that people who aren't seasoned players are impressed by this but this reads like an average wood league "south" game. People who play Dota will know what I'm referring to in reference to "south". In such games, the strategy is simple : You beat out the brute-force cheese snowballing w/ intellect. However, this is where they no doubt decided to put up the artificial red tape. You're 100% right and being down voted for it as I no doubt will. I am not impressed because I've played a ton of games and observed a ton of dynamics. You can tell in the first 10min what level and type of game play is occurring and it isn't of the intellectual variant. It's of the bot variant. Broader Dota was designed and constantly updated to combat such gameplay. When people settle on cheese strategies or exploits, they intentionally change the game balance to prevent it. I'm more interested in how this is achieved than what it has been tuned to try to trick people into believing. Looking under the hood, I see the same thing when I look across all Weak AI solutions : Lots of hand coded steering functions with dynamic weights populated by incredible amounts of brute-force random searching (past a human lifetime of average gameplay). While this may secure headlines and funding, this is miles away from the real direction you have to go towards Strong AI. It doesn't seem many well funded/popularized groups understand this thus will ultimately end up fooling themselves. Point out this truth and you get down votes. Essentially the broader community plugging their ears which is why a large number of these efforts are due for a significant failure.

crsv 8y ago

In case anyone would like to watch a VOD of the event, it can be found here: https://www.twitch.tv/openai/video/293517383 https://www.twitch.tv/openai/video/293517383 It was a really wonderfully done event overall and there's several interviews throughout with different folks from the OpenAI team.

abhiminator 8y ago
YouTube mirrors: Game 1&2 [0] Game 3 [1] [0] https://www.youtube.com/watch?v=eaBYhLttETw https://www.youtube.com/watch?v=eaBYhLttETw [1] https://www.youtube.com/watch?v=_QQYaVUODkE https://www.youtube.com/watch?v=_QQYaVUODkE
- ufo 8y ago
  (These are unofficial highlights btw, not mirrors)

apeace 8y ago

If anyone from OpenAI sees this: The one thing that really surprised me yesterday was when OpenAI Five seemed to pause the game. The commentators speculated that “it was learning” because the humans had paused in game 1 due to lag. I assume that’s not right, as OpenAI five is not training itself as it’s playing (wouldn’t make much sense to add one more game sample to the billions it has already trained on). I thought it was interesting that the commentators had this misconception, and was wondering what lead to OpenAI Five hitting pause.

deleted 8y ago
[deleted]
nicklovescode 8y ago
I think the bot was setup to pause if it ran into any lag or other minor issues (was at the event and heard this from someone there)
gdb 8y ago
A network blip caused all the players to drop from the game. Incidentally, we'd just changed the code a few days earlier from "automatically surrender when a human disconnects" to "do nothing if a human disconnects; automatically pause if all humans disconnect". Had done a lot of advance planning for what might break!
- abhiminator 8y ago
  Off the tangent question here -- Do you have any plans to pit OpenAI Five against the winners of this year's 'The International 8' tournament scheduled for later this month? Folks in the DotA 2 community are going crazy over this possibility, would be incredible if this happens.
  - Dibes 8y ago
    Looks like they are at least playing pros from TI8[0]. [0]: "These results give us confidence in moving to the next phase of this project: playing a team of professionals at The International later this month." from https://blog.openai.com/openai-five-benchmark-results/ https://blog.openai.com/openai-five-benchmark-results/ at the bottom

iMuzz 8y ago

Can someone ELI5 what the significance of this is? Why was Dota chosen as the game for an AI to get good at? Is Dota more difficult than Go? Why or why not?

krotton 8y ago
They wrote about it here (The Problem section): https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/
crsv 8y ago
In short, it's significant progress in the OpenAI team's quest to build really effective self trained artificial intelligence. Conquest in a problem space like this has real world applications that could translate to things like extremely effective artificial limb dexterity (according to the talking points from the event). Dota 2 was chosen because it's a nice combination of pre-determined rules (to an extent) and an extremely complex problem set of possible moves and actions. Dota's development team also was really supportive to this effort and they have an API that suits the Open AI team's purposes of interacting with the game really well. Great fit all around. Dota 2, especially when played 5v5, is orders of magnitude more complex than in terms of possible decisions and moves than Go.
doodhwala 8y ago
DotA is substantially different compared to Go in the following main senses: 1) In Go, you are allowed to see the entire board and pieces at all times -- it is a complete information game. On the other hand, games like DotA have partial information because you are not able to see where your opponents are and what they are doing at all times. 2) Go, Chess and many of the Atari games are single player games. OpenAI wanted to see if machine learning can be applied in a multiplayer setting where the problem needs to be solved at a global / team level. 3) DotA has so many mechanics and strategies where you have a lot of choices to be made. One of the challenges is whether one can look at the overall outcome of the game and reason about what particular choices went into the winning or losing of the game. This (long event horizon) makes it extremely difficult to learn such models. OpenAI interfaced with a complicated game like DotA through an API provided by Valve which made it lot easier. Instead of seeing the game screen, they got snapshots of data of around 35KB per observation (co-ordinates of heroes, creeps etc). In the absence of this API, they would have had to use substantially more computational resources to render the in game graphics and this would also make the training process extremely slow. This benchmark was an experiment that demonstrated that tackling such a class of problems is indeed possible (given a lot of computational resources and an environment to train in). During the interviews in between and after the games, they mentioned that the algorithms that they have used can have many applications in all fields a̶l̶t̶h̶o̶u̶g̶h̶ ̶s̶p̶e̶c̶i̶f̶i̶c̶ ̶e̶x̶a̶m̶p̶l̶e̶s̶ ̶w̶e̶r̶e̶ ̶n̶o̶t̶ ̶p̶r̶o̶v̶i̶d̶e̶d̶ @crsv's comment describes the example they talked about. Perhaps an ELI15, but hope this helps.

johncoogan 8y ago

I wonder how long it will be for them to build a version that actually uses the screen images as opposed to all the data from the Bot API. Seems like it would be a lot harder when you add an image processing step and restrict processing of information from across the map without actually scrolling over there.

minimaxir 8y ago
Per the architecture, the model does use a CNN to process minimap data.
- sgillen 8y ago
  Yes but the comment you're replying to is saying that it would be interesting to only rely on information from visual input on the screen, rather than on getting (for example) the absolute XY position of every player as a direct input to the network.
  - wetpaws 8y ago
    It's like writing a bot that would play chess using video feed from camera. Yes, it's doable and an interesting problem on it's own but completely unrelated to what openai is doing.
    - sgillen 8y ago
      yes I see what you are saying with the chess example, but I think in this case adding the visual layer actually adds interesting problems related to what openAI is trying to do. See my reply to lawrenceyan above.
    - adamrezich 8y ago
      Chess is a poor example because it's a turn-based game whereas Dota is real-time and incredibly fast-paced. Visually parsing a chessboard to know which piece is which is trivial, and also you have all the time in the world to do it (between turns). In Dota things happen so quickly, particle effects pop off all over the place, and through all of it, you have to constantly manually re-place your camera in the optimal position. In chess, both players always have perfect information about the game-state, and this is far from the case in Dota. OpenAI does account for fog of war, so it's not COMPLETELY omniscient, but it is still more omniscient than human players ever have the ability to be, without having to fiddle with the camera etc.

_0ffh 8y ago

I think the third game would probably have been a lot more tense, if they'd let the OAI5 make the last one or two picks by itself.

gpm 8y ago
I bet they weren't quite sure what they had to do to make the game even. The previous rounds were hugely in favor of the ai and it sounds like they don't have much experience playing against humans at this level.
make3 8y ago
I think they wanted to show that the humans could win one

KaoruAoiShiho 8y ago

Did you think the AI would win despite the adversarial draft?

backpropaganda 8y ago
I'm curious to know if the AI can win with a 50-50 draft. It can be argued that the main reason the humans lost was because they didn't have a feel for the meta of this 18-hero version of Dota, while the bots did. What if you give the humans an equal chance at the game wrt the draft? That would have made for a much more interesting game than having the humans play a game they've never played before in their lives.
- KaoruAoiShiho 8y ago
  It seems likely that actual pros instead of a mismash of commentators and community stars would be able to win game 2 even with the draft favoring the AI.
  - ufo 8y ago
    For what is worth, those commentators are actually very good players. Every year they team up in the qualifiers for the world championship and they usually make it very far.
    - iotb 8y ago
      I'm an active player with enough hours logged to interpret the game. That being said, the observations should also be obvious to even the most novice player. I don't need a commercialized and obviously favorable and biased stream of commentary clouding a much more sound analytical capability. I'm pretty sure they're not going invite commentators that point out the obvious reality as to what is going on. Per the commentators, bot like behavior gets extra positive humanized characterization and is marked as intelligent. Whereas snowball-cheese is known as the lowest tier and least intelligent strategy in the game. There's even a set of memes in the community for people who attempt such strategies. People need to start doing more critical analysis of their own and stop relying on commercialized and biased information and commentary when settling on an viewpoint. Watching others play only gets you so far in understanding. When you play and you see what I'm saying for yourself, you can skim through the provided clips and understand exactly what's going on. The fact that this continues to get hyped up vs someone stating what's going on is plain sad.

a_humean 8y ago

I'm really excited to see the limits get lifted, in particular around items and couriers. The team of semi-pros/ex-pros/pros that played against the AI commented that the AI was using the highly unusual 5 invincible couriers to enable a style of play that isn't possible in normal dota. The AI's solution to dota was unrelenting aggression once a small early game lead was established after ~10 minutes. This early aggression was possible because the AI was able to ferry items (healing consumables) with such frequency that once the human team made one mistake in a team fight it wasn't possible to recover. Normally after winning an objective you are forced to reset as you have expended a lot of resources to order achieve your objective, and you are actually most vulnerable to a counter play right after you win something big. With a constant stream of healing this risk was significantly reduced. Also, the drafting is a clear limitation that needs to get lifted. The AI was essentially pursuing a "death-ball" strategy of grouping as 4 or 5 and pushing right down a lane, which can be countered by picking mobile and fast heroes that can put pressure on different parts of the map and slow down the "death-ball" by forcing reactions. However, none of those heroes were draft-able, and so the humans were forced to play the AI's strategy. The AI's strategy favors team fight coordination (laying stuns correctly, and correctly calculating whether damage through nukes is sufficient to kill any particular target) and reaction times, at which the AI was clearly superior to the human team.

shelune 8y ago
I think it's because the organizers want to make sure the bots will have a good performance here. Ofc OpenAI is awesome but it's impossible to cover such a complicated game as DotA within just 1 year. What they have achieved though is still awesome. I'm just slightly bugged by the fact that the developers didn't take action execution into delay consideration. The response time is 200ms but humans also need some more time to drag the mouse and click to perform the action. Their insane reactions actually make me less impressed.
- ufo 8y ago
  I got the impression that they weren't particularly looking for a hero pool with a "deathball" meta, and the hero pool they selected had more to do with those being some of the simplest heroes to program at first. There is a lot of overlap between with the heroes they implemented and the set of introductory heroes that are recommended for new players who are playing their first games. Regarding the insane reaction, I wonder if there is a natural way to handicap the AI to more human-like reaction times. Reaction time is not a good measure on its own because human reaction time can vary a lot depending on the level of surprise. That said, my impression from watching the game was that the power of the AI had less to do with perfect reaction time and more to do with their "hive mind" coordination. If an enemy ever gets in the wrong position, they are immediately punished by a concerted attack. Humans have a harder time doing this because they need to communicate their intentions first. Sometimes each player will be focusing their attention on a different target.

sigi45 8y ago

Why did i miss it?! :( I think first it was planned for 28th now i read it one day later :/

ufo 8y ago
It was originally scheduled for July 28th, but was rescheduled to yesterday. Well, at least you can still watch the twitch vods :/ That said, you still have a chance to catch the match against a professional team that is going to happen sometime between the 20th and 25th.

fermienrico 8y ago

Side comment: Open AI's design is excellent and it doesn't get in the way of reading their articles. They've taken a Stripe-like design approach without getting too overboard. Well done!

joliv 8y ago
There’s a reason their site is Stripe-like: they hired Stripe’s creative director:) https://dribbble.com/luddep https://dribbble.com/luddep
- dclausen 8y ago
  and Stripe's former CTO: https://www.linkedin.com/in/thegdb/ https://www.linkedin.com/in/thegdb/

YeGoblynQueenne 8y ago

>> These results show that Five is a step towards advanced AI systems which can handle the complexity and uncertainty of the real world. Great results that deserve congratulations- but they show nothing of the sort. The distance between a game and the real world is about twice as big as the distance between a game and a simulation of the real world (which a game isn't even). Winning at dota is nothing like negotiating the real world.

red75prime 8y ago
I can see your comments going up to "Weak AIs designing, producing and controlling agile robots is a great step, but it is a long way before they learn enough human psychology to perform as well as a door-to-door salesperson." I'm not sure what will follow.

deleted 8y ago

[deleted]

exabrial 8y ago

5 thousand lifetimes of play versus .45 lifetimes, I think the outcome was predictable!

make3 8y ago
yeah, but you gotta add hundred + millions of evolutionary learning to model the physical world and the actions of our peers, built in to our (mammal) brains

exabrial 8y ago

Is the ai forced to use voice communication and chat to coordinate bots?

make3 8y ago
there is no communication at AI between bots afaik

simonebrunozzi 8y ago

Prediction and hope: within 1-2 years, Open.AI will be able to offer a GREAT AI-powered challenge to human players across many games. I occasionally play strategy games and I am always frustrated by a very stupid AI that gets challenging only by beefing up its stats. I would love to play against a smart AI. I don't do multiplayer because: I like to play offline and/or whenever I want, and I play so rarely that most other human players wouldn't enjoy playing with me. Open.AI team: please listen!

sgillen 8y ago
Yes an actual good AI in Civ would vastly improve that game for me.

tehsauce 8y ago

For a research organization openai sure seems to be putting a lot of effort into PR. Beating players using a subset of the game is impressive but there is no mention whatsoever of the fact that only a subset of the game is allowed. The research is cool by itself, why do they feel the need to promote so heavily that the AI is beating human players "in the first 17 minutes of" rather than giving a realistic picture of their progress. This is how you create an AI winter.

alkonaut 8y ago

How much of an AI’s advantage is ”I/O” related and how much is actual strength in strategy/gameplay? Are the AIs actions rate-limited, delayed, and perhaps time randomized a bit so that an AI doesn’t get an advantage because it can to actions faster or better synchronized to exact times, or more responsive (say to avoid any action under a certain reaction time, all actions are scheduled 50+rand(5) after the AI queues them)?