3 ms·

> Alphastar also didn't play with the same limitations that a human has. Even after removing its ability to see the entire map and finally forcing it to scroll

> Alphastar also didn't play with the same limitations that a human has. Even after removing its ability to see the entire map and finally forcing it to scroll around, alphastar never misclicks (so its APM==EPM) and can still blast nearly unlimited APM for short bursts as long as its "average APM" over an x-second period matched human's APM. > I believe Alphastar would generate more interesting strategies if we limited alphastar to a bit below human APM... No Alphastar definitely had misclicks, and it had a maximum cap on APM regardless of average that was far lower than the max burst of APM (or even EPM) of top players. When I have the time I can go dig up some games where Alphastar definitely has misclicks, and I believe the Deep Mind team has said before that it will misclick. Its APM limits are already lower than pros both on average and in bursts (and are reflected in its play, Alphastar will often mis-micro units in larger, more frantic battles such as allowing disruptor shots to destroy its own units, but it will never make the same mistake with much smaller numbers of units). > Currently its main strategy is "perfectly juggle stalkers" Definitely not. That was its strategy in its early iterations against MaNa and is no longer feasible with the stricter limitations in place. Its Protoss strategy is significantly more advanced than that now (see its impressive series of games against Serral with an amazing comeback here: https://www.youtube.com/watch?v=jELuQ6XEtEc https://www.youtube.com/watch?v=jELuQ6XEtEc and a powerful defense against multi-pronged aggression here: https://www.youtube.com/watch?v=C6qmPNyKRGw https://www.youtube.com/watch?v=C6qmPNyKRGw) (and of course by "now" I mean when Deep Mind took it off the ladder). Both of these involve an eclectic mix of units with Alphastar effectively using each type of unit and varying it in response to what Serral puts out and its own resource constraints. A lot of commentators have difficulty distinguishing Alphastar from humans when the former plays as Protoss (its Terran and Zerg play is weaker and often more mechanical). > I mean strategies that humans could learn to adopt. My main takeaways from watching Alphastar were "pros undervalue static defense and often have a less than optimal number of workers (where Alphastar's seeming overproduction of workers lets it shrug off aggressive harassment)," but I don't know if those have picked up in the meta.

pretendscholar 5y ago
Why is it far worse with Terran and Zerg?
- dwohnitmok 5y ago
  I don't know enough to answer "what mechanisms of how the AI works would cause it to be worse than Terran and Zerg." If the question is rather "what characteristics of Alphastar's Terran and Zerg play style make me say that its Terran and Zerg play is worse than its Protoss play," the simplest answer is that Alphastar just feels a lot more like a bot. Unlike when playing Protoss, it seems to get into certain "ruts" of unit composition and tactics that are a bad match for the opponent it's facing and can't seem to reactively change based on the game is going, whereas with Protoss it seems more than happy to change its play style over the course of the game based on what the opponent is doing.
runnerup 5y ago
Edit 2: Reading through the "supplementary data" of the 2019 paper, it definitely appears that the AlphaStar which reached grandmaster was not limited in the same ways as the 2017 paper would suggest. x/y positions of units are not determined visually, but fed directly from the API. So AlphaStar absolutely can just run Attack(Position: carrier_of_interest->pos.x) and not mis-click. It's "map" / "vision" is really just a bounding box of an array of every entity/unit on the map and all the things that a human would have to spend APM to manually check (precise energy level, precise hit points remaining, exact location of invisible units, etc). See [7]. DeepMind showed they have some fixed time delays to emulate human experience, if they had a position fuzzer, they would have mentioned it. I'm reasonably convinced they gave AlphaStar huge advantages even in the 2019 version that was 'nerfed' from the 2018 version. The 2017 paper was a more ambitious project IMO that didn't quite get fully developed. Edit: 7 minutes after writing this I re-read the original paper[-1]. https://arxiv.org/pdf/1708.04782.pdf https://arxiv.org/pdf/1708.04782.pdf page 6 and 7 make it clear that DeepMind limited themselves to SpatialActions, so they cannot tell units "Attack Carrier" but have to say "Attack point x,y" (and x,y also has to be determined visually, not through carrier_of_interest->pos.x ). It's still not clear in the paper if any randomness is added to Attack(x,y). Additionally, I have some serious concerns about assuming that the design decisions made in this 2017 paper were actually used in the implementation of the 2019 Alphastar demo vs TLO and MaNa. The paper claims "In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players." I would agree with this choice! But [5][6] indicates that Alphastar's APM spiked to over 1500 APM in 2019! And even in moments when a human reaches that APM, their EPM would be an order or magnitude lower, whereas Alphastar's EPM matches its APM. Original post: Thank you so, so much for adding to the discussion! Would love to chat more about this if you see my reply and feel like it. Regarding "mis-clicking", my understanding was that AlphaStar used Deepmind's PySC2[0][1], which in turn exposes Blizzard's SC2 API[2][3]. Here is the example for how to tell an SCV to build a supply depot: Actions()->UnitCommand( unit_to_build, ability_type_for_structure, Point2D( unit_to_build->pos.x + rx * 15.0f, unit_to_build->pos.y + ry * 15.0f ) ); where unit_to_build->pos.x and unit_to_build->pos.x are the current position of the SCV and rx and ry are offsets. It's possible to fuzz this with some randomness, and indeed in the example, rx and ry are actually random (because the toy example just wants to create a supply depot in a truly random nearby spot, it doesn't care where). But the API doesn't attempt to "click" on an SCV and then use a hotkey and then "click" somewhere else. The API will never fail to select the correct SCV. It will also build precisely at the coordinates provided. Point 1: Even if DeepMind added a fuzz to this method to make it so AlphaStar can "misclick" where the depot gets built, it cannot accidentally select the wrong SCV to build that depot. (Possibly wrong, as they could be using SpatialActions, see below) Point 2: Most bot-makers wouldn't add a random fuzz to the depot placement coordinates to make their AI worse and I'd be super surprised if there was hard evidence somewhere that Alphastar had such a fuzz. (This is my main concern.) My personal conclusion was that anything which looks like a "misclick" is, in fact, a "mis-decision". A human can decide "I want my marines to attack that carrier" but accidentally click the attack onto a nearby interceptor. I didn't think Alphastar could do that because I assumed it would use the Attack(Target: Unit) method instead of Attack(Target: Point) in that scenario -- and even if they used Attack(Target: Point) it would be used as Attack(Target: carrier->pos.x). However, I realize now that they could be doing everything with SpatialActions (edit: it does, see paper[-1] pp. 6-7) (select point, select rect's)[4], and that they could have a implemented a randomness layer to make alpha star literally mis-click. I suppose I would need to test this API and dive into the replay files to first see if its possible to discern the different between Attack(Target: carrier_of_interest) and Attack(Target: carrier_of_interest->pos.x). Then, even if Alphastar is using the latter, it's still not clear that there's an additional element of randomness outside of the AI/ML control. Has anyone already done an analysis of the replay files on this level, or has DeepMind released hard info on how they're controlling the bot? -1: https://arxiv.org/pdf/1708.04782.pdf https://arxiv.org/pdf/1708.04782.pdf 0: https://www.youtube.com/watch?v=-fKUyT14G-8 https://www.youtube.com/watch?v=-fKUyT14G-8 1: https://github.com/deepmind/pysc2 https://github.com/deepmind/pysc2 2: https://github.com/Blizzard/s2client-proto https://github.com/Blizzard/s2client-proto 3: https://blizzard.github.io/s2client-api/index.html https://blizzard.github.io/s2client-api/index.html 4: https://blizzard.github.io/s2client-api/structsc2_1_1_spatial_actions.html https://blizzard.github.io/s2client-api/structsc2_1_1_spatia... 5: https://www.alexirpan.com/2019/02/22/alphastar.html https://www.alexirpan.com/2019/02/22/alphastar.html 6: https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii https://deepmind.com/blog/article/alphastar-mastering-real-t... 7: https://ychai.uk/notes/2019/07/21/RL/DRL/Decipher-AlphaStar-on-StarCraft-II/ https://ychai.uk/notes/2019/07/21/RL/DRL/Decipher-AlphaStar-...

pretendscholar 5y ago

Why is it far worse with Terran and Zerg?

dwohnitmok 5y ago
I don't know enough to answer "what mechanisms of how the AI works would cause it to be worse than Terran and Zerg." If the question is rather "what characteristics of Alphastar's Terran and Zerg play style make me say that its Terran and Zerg play is worse than its Protoss play," the simplest answer is that Alphastar just feels a lot more like a bot. Unlike when playing Protoss, it seems to get into certain "ruts" of unit composition and tactics that are a bad match for the opponent it's facing and can't seem to reactively change based on the game is going, whereas with Protoss it seems more than happy to change its play style over the course of the game based on what the opponent is doing.

runnerup 5y ago

Edit 2: Reading through the "supplementary data" of the 2019 paper, it definitely appears that the AlphaStar which reached grandmaster was not limited in the same ways as the 2017 paper would suggest. x/y positions of units are not determined visually, but fed directly from the API. So AlphaStar absolutely can just run Attack(Position: carrier_of_interest->pos.x) and not mis-click. It's "map" / "vision" is really just a bounding box of an array of every entity/unit on the map and all the things that a human would have to spend APM to manually check (precise energy level, precise hit points remaining, exact location of invisible units, etc). See [7]. DeepMind showed they have some fixed time delays to emulate human experience, if they had a position fuzzer, they would have mentioned it. I'm reasonably convinced they gave AlphaStar huge advantages even in the 2019 version that was 'nerfed' from the 2018 version. The 2017 paper was a more ambitious project IMO that didn't quite get fully developed. Edit: 7 minutes after writing this I re-read the original paper[-1]. https://arxiv.org/pdf/1708.04782.pdf https://arxiv.org/pdf/1708.04782.pdf page 6 and 7 make it clear that DeepMind limited themselves to SpatialActions, so they cannot tell units "Attack Carrier" but have to say "Attack point x,y" (and x,y also has to be determined visually, not through carrier_of_interest->pos.x ). It's still not clear in the paper if any randomness is added to Attack(x,y). Additionally, I have some serious concerns about assuming that the design decisions made in this 2017 paper were actually used in the implementation of the 2019 Alphastar demo vs TLO and MaNa. The paper claims "In all our RL experiments, we act every 8 game frames, equivalent to about 180 APM, which is a reasonable choice for intermediate players." I would agree with this choice! But [5][6] indicates that Alphastar's APM spiked to over 1500 APM in 2019! And even in moments when a human reaches that APM, their EPM would be an order or magnitude lower, whereas Alphastar's EPM matches its APM. Original post: Thank you so, so much for adding to the discussion! Would love to chat more about this if you see my reply and feel like it. Regarding "mis-clicking", my understanding was that AlphaStar used Deepmind's PySC2[0][1], which in turn exposes Blizzard's SC2 API[2][3]. Here is the example for how to tell an SCV to build a supply depot: Actions()->UnitCommand( unit_to_build, ability_type_for_structure, Point2D( unit_to_build->pos.x + rx * 15.0f, unit_to_build->pos.y + ry * 15.0f ) ); where unit_to_build->pos.x and unit_to_build->pos.x are the current position of the SCV and rx and ry are offsets. It's possible to fuzz this with some randomness, and indeed in the example, rx and ry are actually random (because the toy example just wants to create a supply depot in a truly random nearby spot, it doesn't care where). But the API doesn't attempt to "click" on an SCV and then use a hotkey and then "click" somewhere else. The API will never fail to select the correct SCV. It will also build precisely at the coordinates provided. Point 1: Even if DeepMind added a fuzz to this method to make it so AlphaStar can "misclick" where the depot gets built, it cannot accidentally select the wrong SCV to build that depot. (Possibly wrong, as they could be using SpatialActions, see below) Point 2: Most bot-makers wouldn't add a random fuzz to the depot placement coordinates to make their AI worse and I'd be super surprised if there was hard evidence somewhere that Alphastar had such a fuzz. (This is my main concern.) My personal conclusion was that anything which looks like a "misclick" is, in fact, a "mis-decision". A human can decide "I want my marines to attack that carrier" but accidentally click the attack onto a nearby interceptor. I didn't think Alphastar could do that because I assumed it would use the Attack(Target: Unit) method instead of Attack(Target: Point) in that scenario -- and even if they used Attack(Target: Point) it would be used as Attack(Target: carrier->pos.x). However, I realize now that they could be doing everything with SpatialActions (edit: it does, see paper[-1] pp. 6-7) (select point, select rect's)[4], and that they could have a implemented a randomness layer to make alpha star literally mis-click. I suppose I would need to test this API and dive into the replay files to first see if its possible to discern the different between Attack(Target: carrier_of_interest) and Attack(Target: carrier_of_interest->pos.x). Then, even if Alphastar is using the latter, it's still not clear that there's an additional element of randomness outside of the AI/ML control. Has anyone already done an analysis of the replay files on this level, or has DeepMind released hard info on how they're controlling the bot? -1: https://arxiv.org/pdf/1708.04782.pdf https://arxiv.org/pdf/1708.04782.pdf 0: https://www.youtube.com/watch?v=-fKUyT14G-8 https://www.youtube.com/watch?v=-fKUyT14G-8 1: https://github.com/deepmind/pysc2 https://github.com/deepmind/pysc2 2: https://github.com/Blizzard/s2client-proto https://github.com/Blizzard/s2client-proto 3: https://blizzard.github.io/s2client-api/index.html https://blizzard.github.io/s2client-api/index.html 4: https://blizzard.github.io/s2client-api/structsc2_1_1_spatial_actions.html https://blizzard.github.io/s2client-api/structsc2_1_1_spatia... 5: https://www.alexirpan.com/2019/02/22/alphastar.html https://www.alexirpan.com/2019/02/22/alphastar.html 6: https://deepmind.com/blog/article/alphastar-mastering-real-time-strategy-game-starcraft-ii https://deepmind.com/blog/article/alphastar-mastering-real-t... 7: https://ychai.uk/notes/2019/07/21/RL/DRL/Decipher-AlphaStar-on-StarCraft-II/ https://ychai.uk/notes/2019/07/21/RL/DRL/Decipher-AlphaStar-...