5 ms·

The use of embeddings in OpenAI Five

62 points by caliber 8y ago

backpropaganda 8y ago
> I think it is amazing that one relatively simple mathematical construct can produce such a complicated behavior. No complicated behavior was produced. > Or, I don’t know, maybe it says something about the complexity of Dota 2 game? Dota 2 game was not played. A tiny subset of the game was attempted. Since people seem to be just buying whatever OpenAI propaganda sells them, let me be specific: only 18 heroes are in the game. The combinatorics explode when you go from 18 to 110. Go on a 5x5 board is a joke compared to 19x19 Go. This is not hard to understand. > Do short term tactics combined with fast reaction time beat long-term strategy? In a game designed for humans, perhaps yes, because the game wouldn't be tested against extremely fast reaction times. The game was meant to be a strategic game for humans. Just because OpenAI Five appears to play Dota 2 (still doesn't beat any serious players though) doesn't imply anything fundamental about tactics beating strategy.
- tlb 8y ago
  It was a subset in early demos, but not any more: https://blog.openai.com/the-international-2018-results/ https://blog.openai.com/the-international-2018-results/
  - deleted 8y ago
    [deleted]
  - backpropaganda 8y ago
    This is the kind of misinformation OpenAI is so great at. Otherwise smart people like tlb are led to believe that this was the full game even though it's not even close to the full game. Thanks for helping me make my point.
    - chibg10 8y ago
      It's an Elon Musk outfit; misinformation is in its DNA.
- prestonh 8y ago
  >> No complicated behavior was produced. The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. >> Dota 2 game was not played. A tiny subset of the game was attempted. Since people seem to be just buying whatever OpenAI propaganda sells them, let me be specific: only 18 heroes are in the game. The combinatorics explode when you go from 18 to 110. Go on a 5x5 board is a joke compared to 19x19 Go. This is not hard to understand. Already mentioned that the hero pool has been opened up. Along with the removal of the other restrictions (invincible courier, items) this is basically pure DotA. >> Just because OpenAI Five appears to play Dota 2 (still doesn't beat any serious players though) doesn't imply anything fundamental about tactics beating strategy. The bots beat a team of 5 casters (granted, with some of the older restrictions in place) who are individually in the top 1% of DotA players by MMR. You're flat out wrong or at the very least inaccurate in all three statements that you made.
  - gsich 8y ago
    >The bots beat a team of 5 casters (granted, with some of the older restrictions in place) who are individually in the top 1% of DotA players by MMR. And without the 5 couriers the caster team probably would have won.
  - backpropaganda 8y ago
    > The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (Merlini hadn't played for months). The bots were garbage at the TI, and got beaten without any problem by the pro teams. > Already mentioned that the hero pool has been opened up. Along with the removal of the other restrictions (invincible courier, items) this is basically pure DotA. The hero pool is still 18 heroes. Dota 2 has over 110 heroes. Can you please try to think what makes you say something so wrong with so much confidence? > The bots beat a team of 5 casters (granted, with some of the older restrictions in place) who are individually in the top 1% of DotA players by MMR. Casters don't count as serious players. Would you regard any sports commentators as good players? Would you put 5 of them randomly in a team and say that's representative of the best players of that sport?
kenhwang 8y ago
Going to echo what I said last time before everyone falls for the propaganda. The bots did not play at a top tier level, the bots didn't even play at a level that could be considered median level. They did not discover new mechanics. They couldn't even understand basic mechanics. Their coordination was no more sophisticated than "wander around together and press all the buttons on the first enemy". They were showing some signs of competitiveness because of their consistently better than human mechanical skills. You can go a long way in Dota2 off solid mechanical skills.
- debacle 8y ago
  I think that smart teams are going to extract what they can out of these bots. Even top players had difficulties dealing with the inhumanness of the bots - people are generally trained to compete against something that acts to some degree on instinct, not probability. The bots lack if fear in e.g. diving a tower before 5 minutes definitely paid off to a certain degree. I also personally feel that the caster stack of players intentionally lost their matches to hype OpenAI. The limitations of the bots were very apparent, and the pro players at TI easily outsmarted the bots by either: - Split pushing with 1 hero and fighting with 4. - Baiting a lower priority hero (e.g. Lion). The AI would very often vastly overcommit for kills in the lategame.
  - kenhwang 8y ago
    Not even lategame, they overcommitted at all points in the game. Those tower dives for trades early game rarely lead to an objective or advantage. They were able to trade early because superior mechanical advantage matters more early game. By the time midgame rolled around, it was pretty clear how naive their strategy was. It has an element of surprise to it since it's not a very human strategy, but just because it's not human doesn't make it remotely good. It's like watching a a car drive on a sidewalk in reverse uphill and honking to avoid pedestrians. It's very impressive that the car figured out that driving on sidewalks reduces collisions with other vehicles, and honking reduces the chance of hitting pedestrians, and it's doing that all while driving in reverse which is very hard for a human to do. But no one in their right mind would call that good driving.
nindalf 8y ago
Some of the comments in the thread are the usual HN contrarianism - "oh, this isn't impressive because X, Y, Z". This is not the case. What OpenAI have accomplished is extraordinary, ~~especially the part where there is no explicit communication between the bots~~ (not true, see below). They managed to coordinate simply by looking at what the others were doing. Did they make elementary errors? Yes they did. Did they fail to grasp the strategy of avoiding fights that the humans used? Yes they did. The fact is that even a rank amateur like me (90th percentile when I was playing) wouldn't have made those mistakes. But I still strongly believe that what has been achieved so far is nothing short of extraordinary. Judging by their rate of progress[1] I would not be surprised to see them come back in a few months with these problems figured out, capable of beating the best teams. [1] - https://twitter.com/OpenAI/status/1037765547427954688 https://twitter.com/OpenAI/status/1037765547427954688
- qwert7890 8y ago
  > They managed to coordinate simply by looking at what the others were doing. This bit seems incorrect, https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-bots-communicate-8b3b83064a95 https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b....
  - Ajedi32 8y ago
    In Dota you have perfect information of the state of all allied units, so I think it's debatable whether sharing input (observation) data between the bots really counts as "communication". Though that same fact also means communication shouldn't really be necessary; the bots are all exact copies of each other and share a copy of the game state, so they should all have similar ideas of what actions are optimal at any given point in the game.
    - qwert7890 8y ago
      Thank you Ajedi32, I updated the post https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-bots-communicate-8b3b83064a95 https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b....
abdullahkhalids 8y ago
Lots of OpenAI Five bashing going on in this thread. I propose a gaming-bot-realism Turing test: when a group of rank 100 or better players cannot discriminate between human teams and the bot team by watching the game, only then are the bots playing "real Dota 2".
- backpropaganda 8y ago
  Great test. The bots won't be able to pass. Both fast reaction times and lack of cohesive strategy will give them away in the first 5 minutes.

62 points by caliber 8y ago

> I think it is amazing that one relatively simple mathematical construct can produce such a complicated behavior. No complicated behavior was produced. > Or, I don’t know, maybe it says something about the complexity of Dota 2 game? Dota 2 game was not played. A tiny subset of the game was attempted. Since people seem to be just buying whatever OpenAI propaganda sells them, let me be specific: only 18 heroes are in the game. The combinatorics explode when you go from 18 to 110. Go on a 5x5 board is a joke compared to 19x19 Go. This is not hard to understand. > Do short term tactics combined with fast reaction time beat long-term strategy? In a game designed for humans, perhaps yes, because the game wouldn't be tested against extremely fast reaction times. The game was meant to be a strategic game for humans. Just because OpenAI Five appears to play Dota 2 (still doesn't beat any serious players though) doesn't imply anything fundamental about tactics beating strategy.

tlb 8y ago
It was a subset in early demos, but not any more: https://blog.openai.com/the-international-2018-results/ https://blog.openai.com/the-international-2018-results/
- deleted 8y ago
  [deleted]
- backpropaganda 8y ago
  This is the kind of misinformation OpenAI is so great at. Otherwise smart people like tlb are led to believe that this was the full game even though it's not even close to the full game. Thanks for helping me make my point.
  - chibg10 8y ago
    It's an Elon Musk outfit; misinformation is in its DNA.
prestonh 8y ago
>> No complicated behavior was produced. The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. >> Dota 2 game was not played. A tiny subset of the game was attempted. Since people seem to be just buying whatever OpenAI propaganda sells them, let me be specific: only 18 heroes are in the game. The combinatorics explode when you go from 18 to 110. Go on a 5x5 board is a joke compared to 19x19 Go. This is not hard to understand. Already mentioned that the hero pool has been opened up. Along with the removal of the other restrictions (invincible courier, items) this is basically pure DotA. >> Just because OpenAI Five appears to play Dota 2 (still doesn't beat any serious players though) doesn't imply anything fundamental about tactics beating strategy. The bots beat a team of 5 casters (granted, with some of the older restrictions in place) who are individually in the top 1% of DotA players by MMR. You're flat out wrong or at the very least inaccurate in all three statements that you made.
- gsich 8y ago
  >The bots beat a team of 5 casters (granted, with some of the older restrictions in place) who are individually in the top 1% of DotA players by MMR. And without the 5 couriers the caster team probably would have won.
- backpropaganda 8y ago
  > The bots were routinely pulling off coordinated team behavior that players couldn't figure out, but that worked. This has to qualify as complicated. This is just not true. Do you know the game? Are you speaking as a player? Or are you telling us what's written in the OpenAI blog post? There is nothing a player couldn't figure out. The caster team lost because of the broken game and because few of them were rusted (Merlini hadn't played for months). The bots were garbage at the TI, and got beaten without any problem by the pro teams. > Already mentioned that the hero pool has been opened up. Along with the removal of the other restrictions (invincible courier, items) this is basically pure DotA. The hero pool is still 18 heroes. Dota 2 has over 110 heroes. Can you please try to think what makes you say something so wrong with so much confidence? > The bots beat a team of 5 casters (granted, with some of the older restrictions in place) who are individually in the top 1% of DotA players by MMR. Casters don't count as serious players. Would you regard any sports commentators as good players? Would you put 5 of them randomly in a team and say that's representative of the best players of that sport?

kenhwang 8y ago

Going to echo what I said last time before everyone falls for the propaganda. The bots did not play at a top tier level, the bots didn't even play at a level that could be considered median level. They did not discover new mechanics. They couldn't even understand basic mechanics. Their coordination was no more sophisticated than "wander around together and press all the buttons on the first enemy". They were showing some signs of competitiveness because of their consistently better than human mechanical skills. You can go a long way in Dota2 off solid mechanical skills.

debacle 8y ago
I think that smart teams are going to extract what they can out of these bots. Even top players had difficulties dealing with the inhumanness of the bots - people are generally trained to compete against something that acts to some degree on instinct, not probability. The bots lack if fear in e.g. diving a tower before 5 minutes definitely paid off to a certain degree. I also personally feel that the caster stack of players intentionally lost their matches to hype OpenAI. The limitations of the bots were very apparent, and the pro players at TI easily outsmarted the bots by either: - Split pushing with 1 hero and fighting with 4. - Baiting a lower priority hero (e.g. Lion). The AI would very often vastly overcommit for kills in the lategame.
- kenhwang 8y ago
  Not even lategame, they overcommitted at all points in the game. Those tower dives for trades early game rarely lead to an objective or advantage. They were able to trade early because superior mechanical advantage matters more early game. By the time midgame rolled around, it was pretty clear how naive their strategy was. It has an element of surprise to it since it's not a very human strategy, but just because it's not human doesn't make it remotely good. It's like watching a a car drive on a sidewalk in reverse uphill and honking to avoid pedestrians. It's very impressive that the car figured out that driving on sidewalks reduces collisions with other vehicles, and honking reduces the chance of hitting pedestrians, and it's doing that all while driving in reverse which is very hard for a human to do. But no one in their right mind would call that good driving.

nindalf 8y ago

Some of the comments in the thread are the usual HN contrarianism - "oh, this isn't impressive because X, Y, Z". This is not the case. What OpenAI have accomplished is extraordinary, ~~especially the part where there is no explicit communication between the bots~~ (not true, see below). They managed to coordinate simply by looking at what the others were doing. Did they make elementary errors? Yes they did. Did they fail to grasp the strategy of avoiding fights that the humans used? Yes they did. The fact is that even a rank amateur like me (90th percentile when I was playing) wouldn't have made those mistakes. But I still strongly believe that what has been achieved so far is nothing short of extraordinary. Judging by their rate of progress[1] I would not be surprised to see them come back in a few months with these problems figured out, capable of beating the best teams. [1] - https://twitter.com/OpenAI/status/1037765547427954688 https://twitter.com/OpenAI/status/1037765547427954688

qwert7890 8y ago
> They managed to coordinate simply by looking at what the others were doing. This bit seems incorrect, https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-bots-communicate-8b3b83064a95 https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b....
- Ajedi32 8y ago
  In Dota you have perfect information of the state of all allied units, so I think it's debatable whether sharing input (observation) data between the bots really counts as "communication". Though that same fact also means communication shouldn't really be necessary; the bots are all exact copies of each other and share a copy of the game state, so they should all have similar ideas of what actions are optimal at any given point in the game.
  - qwert7890 8y ago
    Thank you Ajedi32, I updated the post https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-bots-communicate-8b3b83064a95 https://medium.com/@stelmaszczykadam/do-openai-five-dota-2-b....

abdullahkhalids 8y ago

Lots of OpenAI Five bashing going on in this thread. I propose a gaming-bot-realism Turing test: when a group of rank 100 or better players cannot discriminate between human teams and the bot team by watching the game, only then are the bots playing "real Dota 2".

backpropaganda 8y ago
Great test. The bots won't be able to pass. Both fast reaction times and lack of cohesive strategy will give them away in the first 5 minutes.