4 ms·

Keep in mind that it's not just pure learning from zero, with these crazy emergent behaviors coming out. There is a fair amount of "coaching" involved. The Open

Keep in mind that it's not just pure learning from zero, with these crazy emergent behaviors coming out. There is a fair amount of "coaching" involved. The OpenAI team explained this last year when they revealed the SF mid 1v1 version of their Dota bot. People were astounded that the SF mid was creep blocking at an extremely high level, since wave positioning/management is a very complex behavior/process that doesn't pay off directly right away. The OpenAI team said that they basically taught the SF specifically how to creep block, and they've done this with other behaviors as well, like denying allied creeps, as well as a lot of the meta-level strategic play like warding/vision and item builds/usage.

antidesitter 8y ago
Perhaps one day they’ll be able to create the analogue of what AlphaGo Zero (learning from scratch) was to the original AlphaGo (learning from human play). That would be very impressive.
- Benjammer 8y ago
  Imo, going from OpenAI Five to some kind of OpenAI FiveZero or whatever is like 1000x the amount of progress of AlphaGo to AlphaGo Zero, just based on the complexity of game mechanics and the fact that it's 5v5 and not 1v1.
  - kirillseva 8y ago
    and perfect vs imperfect information - the policy network has to forecast enemy positions and deduce enemy goals. At least in Go you know full game state
shawn 8y ago
The thing is, OpenAI are terrible at the game. If they try to coach too much, they'll ruin their chances at beating the top Dota team. They have to let the bot discover its own winning strategies. For example: There are an infinite number of places to place a ward. One way to train the bot is to preselect all possible ward locations, reducing it to 30 or so common ones. Another way is to make an optimization algorithm where the bot focuses on trying to maximize the "strategic vision" (if it's possible to come up with a measure for "strategic") and then let the bot place wards wherever it wants. After hundreds of years of self-play, it should figure out the best place and times. As I write this out, I think you're probably right. There are too many aspects of the game for a purely-random algorithm to be effective... E.g. item builds. But I'm holding out hope that's just because they haven't figured out a good way to encode all dota items into a distance measure. AI has proven over and over that humans aren't so special. And humans know how to adapt to the game. That said, I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. :) Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate?
- dx87 8y ago
  During the OpenAI stream where they first played against 5 other players, they said that the AI isn't picking items at all, it's just using pre-made recommended item lists. They also said that the match was being played on an older patch so they may just not update if the upcoming patch makes large changes.
- Benjammer 8y ago
  >I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. Totally agree with you here. This would also help actual DOTA players to analyze what is truly emergent behavior that might open new doors to high level play styles, and what is just weird coaching/niche optimization choices by the OpenAI team.
  - ufo 8y ago
    For what is worth, they have been very clear about that kind of stuff in their blogposts and interviews. It just happens that the information is a bit disjointed due to that format.
- hoseja 8y ago
  >Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate? The bots aren't even playing real Dota, let alone current patch.
  - shawn 8y ago
    That's not really true. There are restrictions, but it's real dota. The fact that people haven't had time to adapt to the rules is irrelevant; the bots' strategy is very similar to TI4, when deathball meta was in fashion and games would end in 13 minutes. OpenAI are here to compete on equal footing. They aren't going to stick with the old patch, I would guess.
    - 8y ago

antidesitter 8y ago

Perhaps one day they’ll be able to create the analogue of what AlphaGo Zero (learning from scratch) was to the original AlphaGo (learning from human play). That would be very impressive.

Benjammer 8y ago
Imo, going from OpenAI Five to some kind of OpenAI FiveZero or whatever is like 1000x the amount of progress of AlphaGo to AlphaGo Zero, just based on the complexity of game mechanics and the fact that it's 5v5 and not 1v1.
- kirillseva 8y ago
  and perfect vs imperfect information - the policy network has to forecast enemy positions and deduce enemy goals. At least in Go you know full game state

shawn 8y ago

The thing is, OpenAI are terrible at the game. If they try to coach too much, they'll ruin their chances at beating the top Dota team. They have to let the bot discover its own winning strategies. For example: There are an infinite number of places to place a ward. One way to train the bot is to preselect all possible ward locations, reducing it to 30 or so common ones. Another way is to make an optimization algorithm where the bot focuses on trying to maximize the "strategic vision" (if it's possible to come up with a measure for "strategic") and then let the bot place wards wherever it wants. After hundreds of years of self-play, it should figure out the best place and times. As I write this out, I think you're probably right. There are too many aspects of the game for a purely-random algorithm to be effective... E.g. item builds. But I'm holding out hope that's just because they haven't figured out a good way to encode all dota items into a distance measure. AI has proven over and over that humans aren't so special. And humans know how to adapt to the game. That said, I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. :) Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate?

dx87 8y ago
During the OpenAI stream where they first played against 5 other players, they said that the AI isn't picking items at all, it's just using pre-made recommended item lists. They also said that the match was being played on an older patch so they may just not update if the upcoming patch makes large changes.
Benjammer 8y ago
>I wish OpenAI would be completely transparent as to what's emergent behavior and what's not. Totally agree with you here. This would also help actual DOTA players to analyze what is truly emergent behavior that might open new doors to high level play styles, and what is just weird coaching/niche optimization choices by the OpenAI team.
- ufo 8y ago
  For what is worth, they have been very clear about that kind of stuff in their blogposts and interviews. It just happens that the information is a bit disjointed due to that format.
hoseja 8y ago
>Oh, one last interesting thing: Icefrog is going to roll out a big patch after this TI, just like he always does. I wonder how much of the bots' knowledge will transfer over? Or if they'd be better off training from a clean slate? The bots aren't even playing real Dota, let alone current patch.
- shawn 8y ago
  That's not really true. There are restrictions, but it's real dota. The fact that people haven't had time to adapt to the rules is irrelevant; the bots' strategy is very similar to TI4, when deathball meta was in fashion and games would end in 13 minutes. OpenAI are here to compete on equal footing. They aren't going to stick with the old patch, I would guess.
  - 8y ago