4 ms·

The thing is, OpenAI are terrible at the game. If they try to coach too much, they'll ruin their chances at beating the top Dota team. They have to let the bot

by shawn 8y ago

The thing is, OpenAI are terrible at the game. If they try to coach too much, they'll ruin their chances at beating the top Dota team. They have to let the bot discover its own winning strategies. For example: There are an infinite number of places to place a ward. One way to train the bot is to preselect all possible ward locations, reducing it to 30 or so common ones. Another way is to make an optimization algorithm where the bot focuses on trying to maximize the "strategic vision" (if it's possible to come up with a measure for "strategic") and then let the bot place wards wherever it wants. After hundreds of years of self-play, it should figure out the best place and times. As I write this out, I think you're probably right. There are too many aspects of the game for a purely-random algorithm to be effective... E.g. item builds. But I'm holding out hope that's just because they haven't figured out a good way to encode all dota items into a distance measure. AI has proven over and over that humans aren't so special. And humans know how to adapt to the game. That said, I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. :) Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate?

dx87 8y ago
During the OpenAI stream where they first played against 5 other players, they said that the AI isn't picking items at all, it's just using pre-made recommended item lists. They also said that the match was being played on an older patch so they may just not update if the upcoming patch makes large changes.
Benjammer 8y ago
>I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. Totally agree with you here. This would also help actual DOTA players to analyze what is truly emergent behavior that might open new doors to high level play styles, and what is just weird coaching/niche optimization choices by the OpenAI team.
- ufo 8y ago
  For what is worth, they have been very clear about that kind of stuff in their blogposts and interviews. It just happens that the information is a bit disjointed due to that format.
hoseja 8y ago
>Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate? The bots aren't even playing real Dota, let alone current patch.
- shawn 8y ago
  That's not really true. There are restrictions, but it's real dota. The fact that people haven't had time to adapt to the rules is irrelevant; the bots' strategy is very similar to TI4, when deathball meta was in fashion and games would end in 13 minutes. OpenAI are here to compete on equal footing. They aren't going to stick with the old patch, I would guess.
  - delusional 8y ago
    Watching the showmatch as a dota player, it was clear it's not "real" dota. I played Turbo while it was new, and the game they are playing is much more similar to that. Having _almost_ instant regen whenever you need it is a gamechanger. It's sort of a "Theseus's paradox" situation.
    - Ntrails 8y ago
      Until the AI is thinking about how to deal with a possible 5th pick brood/huskar/meepo it's not real drafting. Until it's coping properly with courier snipes mid, courier use distribution etc. It's just playing a variant of dota. It's still really fucking good. But it's not dota _yet_
andreyk 8y ago
re "That said, I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. :)" This is a pretty good summary - the main bit to know is reward shaping https://medium.com/@evanthebouncy/understanding-openai-five-16f8d177a957 https://medium.com/@evanthebouncy/understanding-openai-five-...
delusional 8y ago
> AI has proven over and over that humans aren't so special. I disagree. I think AI has shows that we are very special.

by shawn 8y ago

dx87 8y ago

During the OpenAI stream where they first played against 5 other players, they said that the AI isn't picking items at all, it's just using pre-made recommended item lists. They also said that the match was being played on an older patch so they may just not update if the upcoming patch makes large changes.

Benjammer 8y ago

>I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. Totally agree with you here. This would also help actual DOTA players to analyze what is truly emergent behavior that might open new doors to high level play styles, and what is just weird coaching/niche optimization choices by the OpenAI team.

ufo 8y ago
For what is worth, they have been very clear about that kind of stuff in their blogposts and interviews. It just happens that the information is a bit disjointed due to that format.

hoseja 8y ago

>Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate? The bots aren't even playing real Dota, let alone current patch.

shawn 8y ago
That's not really true. There are restrictions, but it's real dota. The fact that people haven't had time to adapt to the rules is irrelevant; the bots' strategy is very similar to TI4, when deathball meta was in fashion and games would end in 13 minutes. OpenAI are here to compete on equal footing. They aren't going to stick with the old patch, I would guess.
- delusional 8y ago
  Watching the showmatch as a dota player, it was clear it's not "real" dota. I played Turbo while it was new, and the game they are playing is much more similar to that. Having _almost_ instant regen whenever you need it is a gamechanger. It's sort of a "Theseus's paradox" situation.
  - Ntrails 8y ago
    Until the AI is thinking about how to deal with a possible 5th pick brood/huskar/meepo it's not real drafting. Until it's coping properly with courier snipes mid, courier use distribution etc. It's just playing a variant of dota. It's still really fucking good. But it's not dota _yet_

andreyk 8y ago

re "That said, I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. :)" This is a pretty good summary - the main bit to know is reward shaping https://medium.com/@evanthebouncy/understanding-openai-five-16f8d177a957 https://medium.com/@evanthebouncy/understanding-openai-five-...

delusional 8y ago

> AI has proven over and over that humans aren't so special. I disagree. I think AI has shows that we are very special.