4 ms·

Very impressive... but it seems like the AI relies entirely on abusing blink stalkers which with perfect micro is basically impossible to counter. It is no sur

by kmnc 8y ago

Very impressive... but it seems like the AI relies entirely on abusing blink stalkers which with perfect micro is basically impossible to counter. It is no surprise it can crush pros when it has perfect timing and zero mistakes in using these units. I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed (Micro harrasing early, early expansions, a very good understanding of when to attack/retreat). It looks just like your average pro..until you see its god tier micro. Well, after seeing Mana crush it in the live game it seems the AI had zero clue as to what to do... it seems like it calculated it couldn't win a fight so let Mana destroy its entire base. So, just like with the Dota AI we see pros can exploit and win easily once they play around the micro advantages.

samfriedman 8y ago
As the commentators mentioned, it's no use building units to counter your enemy's army (Immortals over Stalkers) when the enemy can control their army so much more effectively. I have to wonder if future competitive games will need to take into account the abilities of reinforcement learning algorithms when releasing balance patches.
- dllu 8y ago
  But there IS use building units to counter your enemy's army. In the last live match when Mana won, his immortal archon zealot composition was what sealed the deal in the end.
  - orbifold 8y ago
    And his far superior positioning.
  - dragontamer 8y ago
    I mean, the AI in that last match couldn't see the whole map at once. Fog of War was enabled in all cases, but the last match was them enabling the "Scrolling window" that humans are forced to look at the game with. The AI in all of the other games could see and control all of its own units on the map simultaneously. No human has this ability due to the limitation of the screen.
    - Dylan16807 8y ago
      But they did say that the way the AI focused was roughly equivalent to moving the screen every couple seconds, not very different from pros, and that adding the limit didn't affect its performance against the old version.
devilmoon 8y ago
>I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed I mean, the agents have learned the game from pro replays so to me it seems obvious it would evolve to play the same strats human pros use.
- Calms 8y ago
  My understanding is that Alphastar is trained on reinforcement learning. I suspect this is unsupervised so it would have learned this behaviour independently without pro replays.
  - w1 8y ago
    Per DeepMind's blog post[1], an agent was initially trained via supervised learning on pro matches. Then, the agent was forked repeatedly, as the population of agents learned via tournament-style self-play. So, while initial strategies could have been seeded by pro play styles, the final models were the result of models learning from games with other models. [1] https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/ https://deepmind.com/blog/alphastar-mastering-real-time-stra...
    - devilmoon 8y ago
      Meaning that the final models evolved by playing against each other and they all started by using pro strategies, so, again, to me it seems kind of obvious that they would end up using pro strategies and in the best case just try to make them better
gambler 8y ago
Stuff like that is why StarCraft is not a very good game to test AI on. It's good for publicity, because it is well-known and has a pro scene so you can claim to "beat humans", but it's too complex in some ways (ruleset) and too simplistic in others (mostly just killing stuf). APM and micromanagement are huge, while long-term strategies are fairly limited compared to many other strategy games. With all of this, it's really hard to see how much/which parts of the game the AI really "understands". I'd love to see an AI that can play MAX or MoO from raw pixel inputs, even if it sucks at it.
- Topgamer7 8y ago
  There are limitations they put on the AI to try ti restrict to human levels. Such as having an action counter. And in the demonstration they filmed today, they actually limited the information the AI knows about to the screen space, which is probably along the lines of what you were wanting.
  - kzrdude 8y ago
    Their action counter rationale seemed fallacious, allowing 300 actions per minute sounds like it would give the bot an edge, presumably the bot has a much better ratio of meaningful actions vs all actions.
  - gambler 8y ago
    >There are limitations they put on the AI to try ti restrict to human levels. Such as having an action counter. Which is exactly why StarCraft is not a very good game to test AI on. It's absurd to put arbitrary limitation on something to make the game "fair" and then pat yourself on the back simply because the algorithm won. If it can already win through pure micromanagement, why There are tons of strategy games which don't revolve around micromanagement where APM simply doesn't matter. All turn-based games, for example. Or real-time games where building stuff is more important than combat.
    - gpm 8y ago
      > All turn-based games, for example. You mean like chess? And go? I think turning to a real time game with a complex rule set after showing they mastered turn based games with simple rule sets was very sensible. > Or real-time games where building stuff is more important than combat. Can you name one that is played professionally (important for balance and comparison to humans) where this is more true than starcraft 2? I think of starcraft 2 as very macro focused as games go. I used to be ~80th percentile in North America (worst region) and I'm certain at the time any pro could have beat me without clicking anything outside of their own base (except the minimap).
Dylan16807 8y ago
It had a variety of strategies, not always making a lot of stalkers. And humans can do some really impressive blink micro too up to a certain number of units. So that's one aspect of it but isn't the main strength.
- orbifold 8y ago
  Those were not one and the same agent. If you look at the figure they released on their website the given agent would probably always have gone for a lot of blink stalkers.
  - JeremyBanks 8y ago
    I was amused to see on their website that one of the top-rated agents we didn't get to see built almost nothing but Void Rays.
  - Dylan16807 8y ago
    We're talking about the whole package of the AI, what the entire system is capable of, so that technical detail doesn't affect kmnc's point or mine. The biggest effect is that it's harder to change strategy midgame, which is not all that critical.
    - orbifold 8y ago
      It makes the trained agent much less like a human, individual agents are not creative, they‘ve just learned to execute a certain build order. It might turn out that this is enough coupled with flawless execution and micro.
      - Dylan16807 8y ago
        "A human agent isn't creative, it just has a set of initial build orders it developed and picks one before the game starts, and after that it adapts to the situation weighted by the units it prefers." Also the execution of the build order is nowhere near flawless.
Symmetry 8y ago
It looks like a human/AI centaur[1] would far exceed the abilities of either alone. In chess that lasted for about a decade, before the human became superfluous. In Go it seems to have happened already? But given the complexity of Starcraft I expect it to take longer than Go. [1]https://en.wikipedia.org/wiki/Advanced_Chess https://en.wikipedia.org/wiki/Advanced_Chess
xster 8y ago
Ya, that one game was a bit... 'fresh'. But saying that it relies 'entirely' on abusing blink stalkers isn't quite right. It did it in 1 game of the 5 they were showing.

by kmnc 8y ago

samfriedman 8y ago

As the commentators mentioned, it's no use building units to counter your enemy's army (Immortals over Stalkers) when the enemy can control their army so much more effectively. I have to wonder if future competitive games will need to take into account the abilities of reinforcement learning algorithms when releasing balance patches.

dllu 8y ago
But there IS use building units to counter your enemy's army. In the last live match when Mana won, his immortal archon zealot composition was what sealed the deal in the end.
- orbifold 8y ago
  And his far superior positioning.
- dragontamer 8y ago
  I mean, the AI in that last match couldn't see the whole map at once. Fog of War was enabled in all cases, but the last match was them enabling the "Scrolling window" that humans are forced to look at the game with. The AI in all of the other games could see and control all of its own units on the map simultaneously. No human has this ability due to the limitation of the screen.
  - Dylan16807 8y ago
    But they did say that the way the AI focused was roughly equivalent to moving the screen every couple seconds, not very different from pros, and that adding the limit didn't affect its performance against the old version.

devilmoon 8y ago

>I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed I mean, the agents have learned the game from pro replays so to me it seems obvious it would evolve to play the same strats human pros use.

Calms 8y ago
My understanding is that Alphastar is trained on reinforcement learning. I suspect this is unsupervised so it would have learned this behaviour independently without pro replays.
- w1 8y ago
  Per DeepMind's blog post[1], an agent was initially trained via supervised learning on pro matches. Then, the agent was forked repeatedly, as the population of agents learned via tournament-style self-play. So, while initial strategies could have been seeded by pro play styles, the final models were the result of models learning from games with other models. [1] https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/ https://deepmind.com/blog/alphastar-mastering-real-time-stra...
  - devilmoon 8y ago
    Meaning that the final models evolved by playing against each other and they all started by using pro strategies, so, again, to me it seems kind of obvious that they would end up using pro strategies and in the best case just try to make them better

gambler 8y ago

Stuff like that is why StarCraft is not a very good game to test AI on. It's good for publicity, because it is well-known and has a pro scene so you can claim to "beat humans", but it's too complex in some ways (ruleset) and too simplistic in others (mostly just killing stuf). APM and micromanagement are huge, while long-term strategies are fairly limited compared to many other strategy games. With all of this, it's really hard to see how much/which parts of the game the AI really "understands". I'd love to see an AI that can play MAX or MoO from raw pixel inputs, even if it sucks at it.

Topgamer7 8y ago
There are limitations they put on the AI to try ti restrict to human levels. Such as having an action counter. And in the demonstration they filmed today, they actually limited the information the AI knows about to the screen space, which is probably along the lines of what you were wanting.
- kzrdude 8y ago
  Their action counter rationale seemed fallacious, allowing 300 actions per minute sounds like it would give the bot an edge, presumably the bot has a much better ratio of meaningful actions vs all actions.
- gambler 8y ago
  >There are limitations they put on the AI to try ti restrict to human levels. Such as having an action counter. Which is exactly why StarCraft is not a very good game to test AI on. It's absurd to put arbitrary limitation on something to make the game "fair" and then pat yourself on the back simply because the algorithm won. If it can already win through pure micromanagement, why There are tons of strategy games which don't revolve around micromanagement where APM simply doesn't matter. All turn-based games, for example. Or real-time games where building stuff is more important than combat.
  - gpm 8y ago
    > All turn-based games, for example. You mean like chess? And go? I think turning to a real time game with a complex rule set after showing they mastered turn based games with simple rule sets was very sensible. > Or real-time games where building stuff is more important than combat. Can you name one that is played professionally (important for balance and comparison to humans) where this is more true than starcraft 2? I think of starcraft 2 as very macro focused as games go. I used to be ~80th percentile in North America (worst region) and I'm certain at the time any pro could have beat me without clicking anything outside of their own base (except the minimap).

Dylan16807 8y ago

It had a variety of strategies, not always making a lot of stalkers. And humans can do some really impressive blink micro too up to a certain number of units. So that's one aspect of it but isn't the main strength.

orbifold 8y ago
Those were not one and the same agent. If you look at the figure they released on their website the given agent would probably always have gone for a lot of blink stalkers.
- JeremyBanks 8y ago
  I was amused to see on their website that one of the top-rated agents we didn't get to see built almost nothing but Void Rays.
- Dylan16807 8y ago
  We're talking about the whole package of the AI, what the entire system is capable of, so that technical detail doesn't affect kmnc's point or mine. The biggest effect is that it's harder to change strategy midgame, which is not all that critical.
  - orbifold 8y ago
    It makes the trained agent much less like a human, individual agents are not creative, they‘ve just learned to execute a certain build order. It might turn out that this is enough coupled with flawless execution and micro.
    - Dylan16807 8y ago
      "A human agent isn't creative, it just has a set of initial build orders it developed and picks one before the game starts, and after that it adapts to the situation weighted by the units it prefers." Also the execution of the build order is nowhere near flawless.

Symmetry 8y ago

It looks like a human/AI centaur[1] would far exceed the abilities of either alone. In chess that lasted for about a decade, before the human became superfluous. In Go it seems to have happened already? But given the complexity of Starcraft I expect it to take longer than Go. [1]https://en.wikipedia.org/wiki/Advanced_Chess https://en.wikipedia.org/wiki/Advanced_Chess

xster 8y ago

Ya, that one game was a bit... 'fresh'. But saying that it relies 'entirely' on abusing blink stalkers isn't quite right. It did it in 1 game of the 5 they were showing.