9 ms·
StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer
- taylorswift_ 8y agoInteresting, outside of the academic merits of the research, I wonder if they could partner with Blizzard (or other similar RTS game studios) and start shipping games with AI that is very difficult to beat? That would usher in a new era of solo and offline game play that could be very fun and challenging! Plus we'd be training Skynet, err, I mean the neural network in the process!
- loser777 8y agoMy understanding is that a micromanagement-based AI is likely not to be an "interesting" difficult AI to beat. The DeepMind project is aimed more towards (with its apm caps) at building something that makes good decisions (e.g., unit composition, build orders, crisis management) instead of relying on brute force optimal control of units. While it is likely possible to completely break the balance of the game with perfect unit control, playing against such an AI (which is really just a bot at this point) would get stale pretty quickly.
- taurath 8y agoJust like playing against MarineKing would get stale quickly for most players :P The problem with starcraft is that its like chess - for a lot of players it really DOES get stale pretty quickly, with only one or two interesting moments in a game (assuming no early blunders). The games that are interesting involve near perfect play for 10-20 minutes and then frequently one split second decision. Macro changes or new strategies are quickly picked up and interated on, and I think thats what the hardcore still stick around for
- loser777 8y agoThe difference is that an interesting AI could likely be scaled back meaningfully for lower skill levels. Make MarineKing play with only the mouse against a lower skilled opponent? The results might be less stale. Rate limiting a brute-force AI would make it trivially bad.
- acomar 8y agoNot really. Stopping MarineKing from sending tons of APM is only part of the challenge. His choices for what to do in any given frame (yes, frame) are also just better than basically every player on the ladder. If you remove his keyboard, you'll find that he still wins most games against folks who don't know the game.
- AmericanChopper 8y agoA 'perfectly' micro-ing AI is both boring to implement and boring to play against. Somebody made one for SC2 back when that was popular, take a look at these videos to see how uninteresting it would become after the first attempt. https://www.youtube.com/watch?v=0EYH-csTttw https://www.youtube.com/watch?v=0EYH-csTttw https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=DXUOWXidcY0 https://www.youtube.com/watch?v=DXUOWXidcY0
- make3 8y agoyeah but I'm pretty sure these bots still get crushed by humans because of macro, planning and positioning, which is why it takes top firms on the planet so long to solve the problem
- AmericanChopper 8y agoIf the AI wasn't very smart, then you'd expect there to be some optimal path that would beat them. But Starcraft has always been a game with very little skill ceiling on micro. Well micro'd units have the potential overcome huge macro advantages. A famous real-world example would be Boxer's Immortal Marines: https://www.youtube.com/watch?v=WJp0t9n8DWk https://www.youtube.com/watch?v=WJp0t9n8DWk
- TulliusCicero 8y agoBut even the best AIs in Starcraft are still quite bad compared to competitive players.
- partycoder 8y agoThe thing with StarCraft is that there are many types of units and types of damage, you need to alternate between moving and attacking, different units have different roles in a formation, etc. When I pick targets I prefer to kill units that deal the most damage or have low HP first.
- daemonk 8y agoI want an AI assisted mode with API hooks where people can code their own sophisticated "macros" that can be activated with a command/button. Limit the code to X number of characters. It would be awesome to see a tournament with this.
- axxl 8y agoThere is some stuff like this, you would see it on the ladder in SC2. Hacks that would perfectly split your marines against incoming banelings (they explode and do splash damage so the more you split the less effective they were), or auto burrowing/unburrowing roaches (which hides them from the enemy and they heal underground). It sounds cool but it’s way cooler when you saw a pro doing all that stuff themselves almost as good and still going back and building units etc.
- bottled_poe 8y agoCheck this out. 100 zerglings vs 20 siege tanks - https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs
- taurath 8y agoInteresting. It seems that this is possible mostly by the lag time between target acquisition and firing by the siege tanks, compounded with the relatively high speed of the zerglings - most any other unit in the game wouldn't be able to react in time.
- daemonk 8y agoI appreciate the manual dexterity, but imagine the amount of strategy that AI assisted play would open up.
- d1zzy 8y agoIf you're looking for more strategic gameplay instead of micromanagement, Starcraft isn't the best game to pick. Lowering the importance of high APM for a game like Starcraft through the use of smart macros is just a hack, why not just play (or design) a game that doesn't need any of that. Of course, the obvious choice is any turn based game. But even if you want to stay in RTS land, one of the more recent titles that comes to mind where micromanagement is not as important and strategy/planning is very important is Ashes of the Singularity. Spoiler: the game may seem "slow" compared to Starcraft but that's also part of reducing the importance of high APM.
- calebh 8y agoFor those who don't know Starcraft: microing refers to the small scale/precise maneuvering of units during a battle, as well as activating special unit abilities at certain times. In the lower level leagues microing is not as important as having a good overall large-scale "macro" strategy. The microing really starts coming into play at the higher levels where it can give the edge needed to win a battle. I am not very surprised that machine learning has been able to successfully execute micromanagement. In my opinion the macro decisions are more interesting, since they typically require higher level reasoning (and comprehension of what the opponent is doing). We have yet to see an AI system that can successfully execute macro strategy when playing against a human opponent.
- s_m_t 8y agoHow can this be so? In my experience playing Starcraft the decision space for macro is very, very, very limited. You are essentially locked into a handful of viable build orders for the early game, a handful of timing attack opportunities in the early and mid game, and after that things veer so off track that very few players do more than wing it.
- tialaramex 8y agoThis is sometimes called the game's "meta" but it's very unclear how much of this is real and how much is basically just fashion. Some good but not quite star quality players like Ketroc deliberately ignore the meta. Ketroc plays with loads of Ravens, he did this when people said Ravens were strong but hard to use, he still did it when the accepted wisdom was that they were now useless, and last time I looked he was still at it after yet further balance changes to SC2. How he uses them changes, but he likes Ravens and they never stopped being viable. Games like Chess have meta too. A year ago humans believed that materiel was essential and strong position made up only for small differences like losing a pawn or two. Then Google's Alpha Zero AI says hold my beer, sacrificing loads of my pieces for positional advantage is fine, because I can use that advantage to pin your extra pieces and still mate you away.
- s_m_t 8y ago
- mLuby 8y agoThree great short videos showing an old StarCraft bot in action. I recommend watching #3 then #2. The second shows a pro player for comparison. https://www.youtube.com/channel/UCOBfAiPhRlKOhdJHaAJ2OpQ https://www.youtube.com/channel/UCOBfAiPhRlKOhdJHaAJ2OpQ
- wesleytodd 8y agoBut can your AI learn to drop mules so it can capture some escaping ultras? Better link: https://clips.twitch.tv/BusyDrabFennelRedCoat https://clips.twitch.tv/BusyDrabFennelRedCoat
- wesleytodd 8y agoFunny enough, after I posted this as a joke I realized this is exactly the kind of thing an AI would be incredibly good at finding. It makes it all that much more impressive seeing Buyn do it.
- taurath 8y agoI've always felt like macro is pretty easy to automate, once you've identified a lot of the potentials in the early game you can get going pretty well. Is AI playing against each other getting to some sort of endgame consistently (against equal opponents), or are they having blowouts because of some small difference in midgame? Also wondering what the interface is - are they creating a virtual mouse/keyboard to enter inputs, are they playing in some sort of headless mode? How do they deal with recognizing units?
- loser777 8y agoIf this project uses the BWAPI (which it says it does in the paper), then I assume that this doesn't need to recognize units as that state is available from the game. On the other hand, the DeepMind "research environment" (https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/ https://deepmind.com/blog/deepmind-and-blizzard-open-starcra...), presents a simplified view of the map that requires AIs to recognize units as simplified blobs on the map and issue commands that more closely resemble what a human would do via mouse and keyboard.
- WhompingWindows 8y agoCreating a micromanagement AI is only the first step. This must be piped into a macro AI, which produces the units with with micromanagement can be done. Then, another critical layer beyond micro or macro is scouting the opponent and predicting what their plan is. Next, you need to adapt your own strategy, which must also be planned at the start of the match since you can't scout instantaneously or perfectly. Finally, your own initial strategy choice must be tailored to the map, to the player you're playing against, to recent games/strategies other pros have exposed as powerful/weak. There are an incredible number of layers of complexity to what a truly fierce Starcraft AI would need to do. And yet, something similar may have been said about Go, Chess, Checkers, Poker, Jeopardy, or many other games before this. If enough talented minds and powerful devices focus on Starcraft, how would humans compete? Does anyone here think top pros would win in a best of 7 or a best of 11, for instance? Which top pros, and BW or SC2? To me, I just think it'd be arrogant to assume SC is different from all the other realms in which humans have been bested by machines.
- TulliusCicero 8y agoI don't think anyone doubts computers will eventually beat pros in Starcraft. I do think it will take quite a bit longer though, even with recent advancements seen in the likes of AlphaGo. From a human perspective, Starcraft doesn't seem "harder" than Chess or Go, but it has more fluidity and ambiguity, both things computers struggle with. And the ruleset, as it were, is VASTLY more complicated.
- MikkoFinell 8y agoI disagree that Starcraft doesn't seem harder. The strategic depth is arguably comparable, but the difference is you need the dexterity of a master pianist to execute.
- jon_richards 8y agoSC2 has a lot of things that help the player over an AI, like imperfect information, a very large ruleset, and continuous movement and time. The main advantage AI can exploit is "infinite micro" units. There are certain units in SC2 that are primarily limited by someone's ability to pay attention to them. Terran Reapers, for instance, in early patches could dance back and forth just within range Zerg Roaches, but out of range of being counterattacked (though the reapers would quickly die if player stopped paying attention). This required such fast movement that it was almost never employed except by professionals during the early game when not much was going on. If an AI is not limited by the number of actions a person can take, these "infinite micro" units become extremely powerful. In particular, most of these types of units are within the Terran race and most powerful in the early game. The best AI in the near future will likely focus on early all-in attacks using these sorts of units (primarily reapers, widow mines, hellions, and medivacs). The human player would have to rely on defensive structures that would put them at a major disadvantage and then catch up in the mid-late game, relying on the fact that the AI gets worse as the game goes on and becomes less predictable. Another thing to note is that Terran strategy relies quite heavily on killing the enemy workers in the early game, but the inbuilt controls for targeting workers are quite clunky and prioritise military targets. Players can target individual workers manually, but generally do so with their whole army at once, while an AI could select the perfect number of units required to 1-shot a worker, give them the command, then select another group, and so on.
- spectaclepiece 8y agoAnybody know if the military in any country is looking into using similar strategies to manage real wars?