19 ms·

DeepMind and Blizzard Open StarCraft II as an AI Research Environment

603 points by nijynot 9y ago

arcanus 9y ago
I also want to see the algorithm win on unorthodox maps. Perhaps a map they have never seen before, or one where the map is the same as before but the resources have moved. Don't tell the player or the algorithm this, and see how both react, and adapt. This tells us a great deal about the resiliency of abilities.
- jmcmahon443 9y ago
  I am considering a random map generator for just this reason.
ipnon 9y ago
Any predictions for how long it will take for an AI to win against the world's best player?
- ktRolster 9y ago
  AI can already win against the world's best player now. It's just a question of whether they can win with human-level micro.
  - obastani 9y ago
    Do you have a citation? As far as I know, existing AIs do not stand much of a chance again humans [1]. My understanding is that even if AIs can beat humans is small skirmishes due to superior micro, they lose so badly at macro that it simply doesn't matter. [1] http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/custom-ai-programs-take-on-top-ranked-humans-in-starcraft http://spectrum.ieee.org/automaton/robotics/artificial-intel...
    - sidusknight 9y ago
      Yeah, you're right. Not sure what the guy is talking about.
      - keerthiko 9y ago
        > the guy is talking about I don't want to be overly semantic or PC on HN, but just saying the GP may be female, judging by their name on profile. Being misgendered could be very offputting and discourages participation, so you may have wanted to say "this person" even if it doesn't sound as offhand as you'd have liked it to come across.
        cm2012 9y ago
        In the NorthEast US, guy is often a gender neutral term. It's totally fine to say, "Hey you guys" to a group of only women.
        figgis 9y ago
        Personal experience tells me it's a pretty common thing across the US. > I don't want to be overly semantic or PC on HN Proceeds to be overly semantic or PC on HN...
        suchow 9y ago
        Calling a group of women "you guys" is different from calling a person of unknown gender "guy" because in the former case the gender-neutral usage is the only possible interpretation, whereas in the latter it might either be gender-neutral usage or an assumption of the commenter's gender. Given that this ambiguity is nearly always a distraction from the primary purpose of the conversation (as it is here), rightfully or not, pragmatically it is better to use terms that steer clear of it. See Bryan Garner's Modern English Usage for fuller advice on safely navigating this linguistic and sociocultural terrain.
        mpfundstein 9y ago
        I am so happy to be European... you guys in America always take everything to the total extreme.
  - sidusknight 9y ago
    There isn't any SC2 AI that's anywhere near as good as a player in an actual game. Sure, they can demonstrate some sick unit control, but that alone isn't enough to win games.
  - cm2012 9y ago
    Cool Starcraft related username!
- aerovistae 9y ago
  Awhile. This just isn't like Go or Chess. The gap from perfect information to imperfect information is quite a chasm, and from turn-based to real-time is even more vast. I play Age of Empires 2 semi-competitively, and I just can't imagine the research progress that would have to be made for a pro to lose to an APM-limited AI agent. So much of the game comes down to intuiting what your opponent is planning without being able to see what they're doing, and more importantly intuiting what your opponent isn't ready for. The biggest difference, though, is the "RT" in "RTS"-- real time. This isn't turn-based anymore, where at a given moment you have a single choice to make, a single piece to move as in Chess and Go, and can then wait for the singular and visible reaction your opponent makes before making your next choice. My understanding it that the moves a program like AlphaGo makes are not interconnected-- it picks each move individually as an ideal move for that board state. It could take over halfway through the game for someone else and would make the same move that it would have made at that point if it had been in control the whole time and arrived at that board state on its own. But that doesn't work in a real-time game, since you and your opponent are now moving simultaneously and the "board" is never static. Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last. There is no "one" move for a given state. Another facet of real-time play is the idea of distraction. It's very important in RTS's to keep your opponent distracted, to disrupt their plans and their focus, by coming from unexpected directions at unexpected times, sometimes concurrently with other operations against them. This can't happen in Chess or Go, where the demands on your focus are far less urgent and two things can't happen at once in a literal sense. Can an AI agent learn to appreciate the power of distraction? Can it learn to intuit what will be most disruptive to a human, and what won't be disruptive at all? How can you teach a computer to learn to be annoying? I will say, of course, that nobody saw AlphaGo coming. And I hope it's the same with RTS's. That would be so exciting. I would love to see an AI blow us away with previously unthought-of strategies. That would be the coolest thing ever. So I hope it happens. But I'd be astonished. RTS is just such a whole new level of thinking for AIs.
  - artursapek 9y ago
    Speaking of AoE 2, I really hope this research ends up benefiting that game. The AI in it has always been so bad but it's still my favorite game of all time.
- gwern 9y ago
  Churchill, of the SC AI competition, guesses it might take as much as 5 years: https://www.wired.com/story/googles-ai-declares-galactic-war-on-starcraft- https://www.wired.com/story/googles-ai-declares-galactic-war...
  - bytefactory 9y ago
    Ooh, this'll be interesting to see, as with AlphaGo, a lot of people disputed the "experts believed it would take ~10 more years" claim retrospectively. With SC2, no AI even comes close to beating even a silver level player, so even a 5 year timeline seems really soon. Let's see if DeepMind can beat it! What's your totally unscientific guess, Gwern?
    - gwern 9y ago
      I think it is doable in under 5 years, but this critically depends on the resources invested by DM and other DL orgs. Deep RL is hugely demanding of computational resources to iterate your designs - for example, the first AlphaGo took something like 3 GPU-years to train it once (2 or 3 months parallelized); however, with much more iteration, DM was able to get Master's from-scratch training down to under 1 month. Now an AG researcher can iterate rapidly with small-scale hobbyist or researcher resources, but if they had had to do it all themselves, Ke Jie would still be waiting for a worthy adversary... When I look at all the recent deep RL research ( https://www.reddit.com/r/reinforcementlearning/ https://www.reddit.com/r/reinforcementlearning/ ) I definitely feel that we can't be far from an architecture which could solve SC2, but I don't know if anyone is going to invest the team+GPUs to do it within that timeframe. (It might not even be as complex as people think: some well-tuned mix of imitation learning on those 500k+ human games, self-play, residual RNNs for memory/POMDP-solving, and use of recent work on planning over high-level environment modeling\, might well be enough.) \ "Learning model-based planning from scratch" https://arxiv.org/abs/1707.06170 https://arxiv.org/abs/1707.06170 , Pascanu et al 2017; "Imagination-Augmented Agents for Deep Reinforcement Learning" https://arxiv.org/abs/1707.06203 https://arxiv.org/abs/1707.06203 , Weber et al 2017 (blog: https://deepmind.com/blog/agents-imagine-and-plan/ https://deepmind.com/blog/agents-imagine-and-plan/ "Agents that imagine and plan"); "Path Integral Networks: End-to-End Differentiable Optimal Control" https://arxiv.org/abs/1706.09597 https://arxiv.org/abs/1706.09597 , Okada et al 2017; "Value Prediction Network" https://arxiv.org/abs/1707.03497 https://arxiv.org/abs/1707.03497 , Oh et al 2017; "Prediction and Control with Temporal Segment Models" https://arxiv.org/abs/1703.04070 https://arxiv.org/abs/1703.04070 , Mishra et al 2017
ktRolster 9y ago
When Watson won at Jeopardy, one of its prime advantages was the faster reaction time at pushing the buzzer. The fairness of that has already been hashed out elsewhere, but..... We already know that computers can have superior micro and beat humans at Starcraft through that(1). Is DeepMind going to win by giving themselves a micro advantage that is beyond what reasonable humans can do? (1)https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs as one example
- sidusknight 9y ago
  Alternatively, http://www.youtube.com/watch?v=DXUOWXidcY0&t=0m50s http://www.youtube.com/watch?v=DXUOWXidcY0&t=0m50s
- obastani 9y ago
  My understanding is that in a full match, AIs still have no hope against humans, since even though they can crush humans at micro, their macro is still abysmal [1]. I'm not aware of a match where any AI has beat a pro human player at Starcraft -- I'd be interested in learning otherwise! [1] http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/custom-ai-programs-take-on-top-ranked-humans-in-starcraft http://spectrum.ieee.org/automaton/robotics/artificial-intel...
  - flamedoge 9y ago
    would you love to be proven wrong?
    - sidusknight 9y ago
      Of course.
    - deleted 9y ago
      [deleted]
  - apetresc 9y ago
    That's because there hasn't been too much concentrated effort on this problem yet, since you'd have to spend quite a bit of effort just integrating with the game engine. Certainly a lot less research has been done on computer SC2 than computer go, and nobody expected a pro to be beaten there 1.5 years ago, either.
siegecraft 9y ago
The API Blizzard is exposing is really nice. Sadly most of the advantages AI had in SC1 were just due to the fact that an automated process could micro-manage the tasks the game didn't automate for you (a lot of boring, repetitive work). SC2 got rid of a lot of that while still allowing room for innovative and overpowered tactics to be discovered (MarineKing's insane marine micro, SlayerS killing everyone with blue flame hellions, some more recent stuff I'm sure from the newest expansions). Hopefully the API lets AIs converge on optimal resource management and get to exploring new and innovative timings, transitions, army makeups, etc.
- captainmuon 9y ago
  > SC2 got rid of a lot of that You think so? My impression is that SC2 had a lot more of repetitive tasks you had to do. E.g. wall off the ramp, send a worker scouting, ... and you have to perform certain actions every X seconds (like using chrono boost). A lot or mastering the game is rote learning, and polishing a build order. Another big part is constantly scouting and reacting to what the enemy is doing. Due to those reasons I found SC2 a bit tedious (it was still fun, just felt more like work than SC1). Granted, this is maybe because I played SC1 more on LAN, and there wasn't all the metagame going on. But I think SC2 really does focus on "grinding" and rote learning to get better, probably this was chosen to make it more "eSports"-like. If I would get to design a SC2.5 or SC3, I would remove all the rote - the actions you always have to perform - and I would give the player the opportunity to trade off more between macro and micro. Actually, it would be cool if you could "research" certain AI features in game for a cost. For example, have one upgrade that micros your marines like a pro, or positions your units in sensible formations. Another player could counter this with a "radio jam" ability, that would make your units in an area take bad formations, or be controlled by a very simple AI. And if you are good at micro, you could save the update, or invest it in an update that makes macro simpler. And so on, I think there are a lot of things one could explore there. Maybe opening SC2 to AI exploration can lead to such gameplay innovations.
  - CuriouslyC 9y ago
    I really like the idea of a helper AI that takes high level commands and handles the manual dexterity aspect of the game. That would make RTS games a lot more attractive for someone like me who prefers simple, thoughtful games rather than complex games with a steep learning curve.
daemonk 9y ago
Blizzard should put in an AI-assisted play mode where players are limited to X lines of code that can be launched with keyboard commands.
- neuronexmachina 9y ago
  That would be quite interesting, having humans handle the macro while the AI focuses on the micro. I'm reminded of "Advanced Chess": https://en.wikipedia.org/wiki/Advanced_Chess https://en.wikipedia.org/wiki/Advanced_Chess > Advanced Chess is a relatively new form of chess, wherein each human player uses a computer chess program to help him explore the possible results of candidate moves. The human players, despite this computer assistance, are still fully in control of what moves their "team" (of one human and one computer) makes.
- ajkjk 9y ago
  I know that, as a player, the high mechanical limitations of Starcraft are part of why it's such a difficult, high-skill-ceiling game. But.. I've tried to enjoy watching SC2 on Twitch, and while it's kinda fun, it's just so disappointing when a complicated strategic game is thrown away because a player doesn't react fast enough to workers being sniped or a drop being shot down. I wish the individual units had some automatic behavior -- for example, marines would could run in spread out formations near tanks or banelings; workers would flee from hazards; flying units would avoid turrets unless specifically directed to fly over them. It would require a lot of rebalancing, of course, but it would make the game so much more tactical and strategic and (imo) enjoyable to watch.
  - jbob2000 9y ago
    You would like Company of Heroes 2, the units require much less micro and the game is a bit slower paced. If your infantry come under fire, they all dive into cover to protect themselves. Defense is a bit more automated (you can set artillery to automatically return fire on enemy artillery once it's exposed) and you have more options for static emplacements to defend critical areas without needing your attention. I have the same complaints about SC2 as you - way too fast and intense - and I really enjoy Company of Heroes. The whole game is basically set up to get rid of "nags" so you can just play around with the units and have fun.
Havoc 9y ago
That's surprising. I thought Bliz didn't want anyone near sc2 but approved of sc1 being used for this purpose.
- yflu 9y ago
  SC1 really doesn't make sense for this, 80% of the skill is just keeping on top of the mindless but mechanically intensive stuff, which is trivial beyond trivial for an AI. SC2's automated away most of this (pretty much everything but production cycles), which makes it a better measure for AI vs human.
  - eatitraw 9y ago
    I agree that SC2 is much easier for a human player. However, top players still have to do quite a bit of micromanagement.
  - jon_richards 9y ago
    A lot of the balance of SC2 will immediately be destroyed by a semi-competent AI. One of the races has especially powerful early tactics that are primarily limited by a human's inability to multitask (reapers, medivacs, and liberators). I honestly don't think it will be possible for a human to beat an AI that just focuses on those strategies.
  - solicode 9y ago
    If they're limiting APM to that of human levels, I don't see it being much of an issue though. APM would just be a limited resource like any other. In fact, I sort of want to see how a strong AI would choose to spend its limited pool of actions. How different would it look compared to a pro? Maybe not much, but I'm not actually certain. In SC1, just the act of moving a large army is a commitment and takes quite a few resources. Moving your armies under fog of war and not letting your opponent know exactly how you're set up in order to get a good angle on you is incredibly important. I want to see how much of an importance a great AI puts on that vs the other things it could be doing instead. Are the strongest AIs going to be more methodological, safer, and slow moving? Or will the best AIs try to exploit the imperfect information aspect of the game and try to lure the opponent into making a wrong decision? I feel like AIs tend to excel at the former, but the latter has been a huge component for the very best pros in SC1. I don't follow SC2, so I don't know much of this also applies there. I just feel like SC1 isn't as mechanical as it's made out to be. There's definitely that huge initial barrier, but once passed that, the game actually feels very delicate and is about good use of resources (including mouse/keyboard actions), timing, and transitions in unit composition to catch your opponent off-balance.
toisanji 9y ago
great they opened it up. I'm sure reinforcement learning / Deep learning will solve this. It has been a tough problem before, but honestly doesnt seem that tough compared to all the harder AI problems.
- Synaesthesia 9y ago
  Such as?
dpflan 9y ago
Related: Today I learned that a group of AI researchers has released a paper called: STARDATA: A StarCraft AI Research Dataset. According to one of the authors: "We're releasing a dataset of 65k StarCraft: Brood War games, 1.5b frames, 500m actions, 400GB of data. Check it out!" > Article: https://arxiv.org/abs/1708.02139 https://arxiv.org/abs/1708.02139 > Github: https://github.com/TorchCraft/StarData https://github.com/TorchCraft/StarData
- tgb 9y ago
  The great thing about this is that it includes the game state throughout the game. It's been pretty easy to find lots of Starcraft replays, but the replays only include enough information to recreate the game (basically just the player actions). If you wanted to know what was happening in the game at the time the player made an action, you had to load up Starcraft and simulate out the game until that point. This dataset has already run the game for you and provided the data!
  - wfunction 9y ago
    Is it that much computation to simulate an entire game? You obviously don't need to render the graphics or anything, it should just be a list of events that occur, which doesn't seem all that slow to process.
    - gwern 9y ago
      Until today's release of the headless Linux client, you still had to run the full StarCraft program, which gets expensive fast. And it massively complicates the workflow to have to play through every game serially to recreate the state rather than simply reading random rows of data from a 300GB dataframe on disk.
      - wfunction 9y ago
        Oh I see, thanks, I didn't know. But man, 300 GB per game sounds completely nuts!
JefeChulo 9y ago
"so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”." I am really glad they are limiting APM because otherwise things just get stupid.
- usaphp 9y ago
  Would not actions per second be a better limit tho?
  - yannyu 9y ago
    Why? Speed is in MPH or KPH, would it be better to go by m/s? The convention in gaming is APM, so they're just using the nomenclature that is already understood.
    - pavedwalden 9y ago
      Maybe what usaphp is getting at is that the AI could still gain an advantage by doing a set of actions much faster than humanly possible in just a fraction of a second as long as it kept its total number of actions that minute below the cap.
      - vmasto 9y ago
        Exactly. Most actions in idle game mode (i.e. when there's no active battle, micro or macro to be done) are null, the gamers simply repeat random meaningless keystrokes; this helps with being always alert and ready for anything that requires a rapid response. An AI doesn't need to do this. Source: Ex pro Starcraft (Brood War) gamer.
        sidusknight 9y ago
        There no were BW pro-gamers outside of Korea.
        taneq 9y ago
        > this helps with being always alert and ready for anything that requires a rapid response. I always wondered why they do that, just spam meaningless things like open/close the stats window or click on the background. Some WoW PVPers did it too (like Laintime, I think it was - played a warrior and ran around spamming the character window open/closed an mashing 'weapon swap' even when there was no earthly reason to do so). I figured it was just the result of too much caffeine for 18 hours a day. This sounds like it's more an equivalent of tennis players' dancing/hopping around, or martial artists doing their ducking/weaving thing?
hitekker 9y ago
This seems all in good fun but I wonder if it's come too late. Starcraft 2 is at its twilight. The biggest leagues of South Korea have disbanded. [1] The prolific progamers who transitioned to Starcraft 2 have gone back to Broodwar. [2] Blizzard itself has scrubbed all references to Starcraft 2 on the very home page of Starcraft. [3] Except for the twitter embed, it has only only one "2" character... in the copyright statement. My take is that the future for the Starcraft franchise will be through remastered and potential expansion packs following it. Starcraft 2 had a good run but, with the entire RTS genre stagnating [4], I don't think Blizzard wants to bet on anything less than the top horse. [1] https://www.kotaku.com.au/2016/10/the-end-of-an-era-for-starcraft-and-south-korea/ https://www.kotaku.com.au/2016/10/the-end-of-an-era-for-star... [2] http://www.espn.com/esports/story/_/id/18935988/starcraft-brood-war-glory-days-jaedong-best-bisu-talk-starcraft http://www.espn.com/esports/story/_/id/18935988/starcraft-br... [3] http://starcraft.com http://starcraft.com [4]http://www.pcgamer.com/the-decline-evolution-and-future-of-the-rts/ http://www.pcgamer.com/the-decline-evolution-and-future-of-t... (Aside from MOBAs)
- cjbprime 9y ago
  I don't quite agree, FWIW. SC2 does seem to be at its twilight in Korea, and I agree progamers and fans there are super interested in Remastered. But I don't think Remastered will be very popular outside KR. The SC2 "war chest" promo appears to have made more money than expected, as measured by hitting its funding ceiling within a few days. So I don't think it's "Remastered replaces SC2", I think it's a divergence into KR playing Remastered and non-KR playing SC2, and the number of progamers and players doesn't have to be zero-sum: it could enlarge the population playing either game, too.
  - doppel 9y ago
    As an esport / spectator sport, SC2 has been waning for a long time, and similarly the War Chest was capped at 200.000 for prize pool money and an unknown amount after that (which, compared to Dota 2 or League of Legends prize pools, is not a lot). I think, given the matchmaking update for remastered, that SC1 will see a resurgence both inside and outside of KR, but I am not sure either SC1 or SC2 will stay competitive in the long run.
deleted 9y ago
[deleted]
blobbers 9y ago
YESSSSSSS!!!!!!!! --why are there not more fanboy comments?!
- hughes 9y ago
  Probably because they don't contribute much to the conversation.
qub1t 9y ago
A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much harder problem than go. Not to mention that the game itself is more complex, in the sense that go, despite being a very hard game for humans to master, is composed of a few very simple and well defined rules. Starcraft is much more open-ended, has many more rules, and as a result its much harder to build a representation of game state that is conducive to effective deep learning. I do think that eventually we will get an AI that can beat humans, but it will be a non-trivial problem to solve, and it may take some time to get there. I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning.
- strgrd 9y ago
  I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the game starts, and attempt to out micro the human opponent before they can build an army-producing building.
  - ionforce 9y ago
    I know this isn't the exact same as the article, but when genetic algorithms were introduced to solve for build orders, the "seven roach rush" was in vogue, something that was unexpected at the time and "discovered" using GA. I think there is a space for finding strategies that have more leeway in execution and thus are more suitable for humans to pilot rather than have machine level micro.
    - 9y ago
SiempreZeus 9y ago
It's a bit too bad they're having to move towards supervised learning and imitation learning. I totally understand why they need to do that given the insane decision trees, but I was really hoping to see what the AI would learn to do without any human example, simply because it would be inhuman and interesting. I'm really interested in particular if an unsupervised AI would use very strange building placements and permanently moving ungrouped units. One thing that struck me in the video was the really actively weird mining techniques in one clip and then another clip where it blocked its mineral line with 3 raised depots...
- Synaesthesia 9y ago
  Well the unsupervised ai couldn’t even do basic tasks from the video I saw, so looks like we have a long way to go.
- dontreact 9y ago
  They can always finetune using RL later. Superversied training was the first step at making AlphaGo work.
naveen99 9y ago
> even strong baseline agents, such as A3C, cannot win a single game against even the easiest built-in AI. Then, why not release code for the built in ai, and improve on it ? Or is the built in ai cheating ?
- Synaesthesia 9y ago
  The built in ai is scripted whereas they’re trying to teach agents to learn the game from scratch with some sort of reward-based/machine learning approach.
  - naveen99 9y ago
    still, why not use machine learning to tweak the script instead of starting from scratch.
- mattnewton 9y ago
  The goal is to "grow" an AI through reinforcement learning and other techniques that are broadly applicable outside starcraft. The existing rules based systems in the game A) are extremely Starcraft specific and the methods have comparatively limited utility outside video game AI, B) unlikely to be scalable to fairly beat a professional human (which introduces too much variability to capture reliably into rules)
DefNotARogueAI 9y ago
This gives me great ideas
arnioxux 9y ago
Are there any known arbitrary code injection for starcraft? Like how you can use a regular controller to reprogram super mario world to play pong? https://www.reddit.com/r/programming/comments/1v5mqg/using_bugs_in_super_mario_world_to_inject_new/ https://www.reddit.com/r/programming/comments/1v5mqg/using_b... https://bulbapedia.bulbagarden.net/wiki/Arbitrary_code_execution https://bulbapedia.bulbagarden.net/wiki/Arbitrary_code_execu... Is this how we are going to accidentally let AGI loose into the world!? /s On a more realistic note I think this will degenerate into a game of who can fuzz test for the best game breaking glitch. Think of all the programming bugs that turned into game mechanics in BW that we haven't discovered for SC2 yet: http://www.codeofhonor.com/blog/the-starcraft-path-finding-hack http://www.codeofhonor.com/blog/the-starcraft-path-finding-h...
hacker_9 9y ago
There's something funny about a company that is actively developing bleeding edge AI technology, but who can't design a webpage that works on mobile without crashing.
- chii 9y ago
  Just goes to show how complicated web tech is, even ai researchers can't get it right!
deleted 9y ago
[deleted]
siliconc0w 9y ago
The SCAI bots I've seen are more hardcoded tactics engines rather than machine learning models. They're still impressive, but their logic isn't quite 'learned' it's hand coded which is a crucial difference.
captn3m0 9y ago
Someone needs to link this to FB's ELF platform (An End-To-End, Lightweight and Flexible Platform for Game Research). That was specifically made for RTS games like SC.
deleted 9y ago
[deleted]
bretepete 9y ago
ATTN: We Provide Medical Marijuana to persons in good need of it.we strive to keep both faith and hope alive and most of good health of mind and body.please for more info,reach us on dr.rickerbs0@india.com
onorton 9y ago
I think I know what my final year project will be.
Ntrails 9y ago
I'd be really interested in how differently tiered data sets (ladder rank) would work as sources for teaching. Is it possible that training on diamond players is less effective than training on, say, silver? Is that actually even an interesting thing to look at?
krasi0 9y ago
The StarCraft 1 BroodWar AI scene has been thriving for a few years now: https://sscaitournament.com/ https://sscaitournament.com/ You can watch 24/7 live AI vs AI games on Twitch at: https://www.twitch.tv/sscait https://www.twitch.tv/sscait Support for voting on who to play next and even a betting system are in place, too. For those who wish to get their feet wet with BW AI development, here are the Java / C++ tutorials: https://sscaitournament.com/index.php?action=tutorial https://sscaitournament.com/index.php?action=tutorial
- krasi0 9y ago
  Some thoughts and analysis on why Starcraft AI by one of the active AI developers Dan: https://dangant.com/2017/08/09/why-starcraft-ai/ https://dangant.com/2017/08/09/why-starcraft-ai/
Magnificents 9y ago
Can we have an AlphaGo tournament with the StarCraft II AI, where pros fight the bot, lose horribly, and make them question their life decisions?
deleted 9y ago
[deleted]
Companion 9y ago
It's a bit too bad they're having to move towards supervised learning and imitation learning. I totally understand why they need to do that given the insane decision trees, but I was really hoping to see what the AI would learn to do without any human example, simply because it would be inhuman and interesting. I'm really interested in particular if an unsupervised AI would use very strange building placements and permanently moving ungrouped units. One thing that struck me in the video was the really actively weird mining techniques in one clip and then another clip where it blocked its mineral line with 3 raised depots...
convefefe 9y ago
I thought this was already happening. Right after AlphaGo beat Lee, I remember hearing about it. Did they give up on having their AI playing SC2? I wondered if that would work, since it seemed to take turns in Go at the same speed as a normal player, I wondered if it was trying to compute the most likely winning move each turn and the late game implications of those moves. If it tried that in a fast paced game how it would deal with the speed. It obviously would need to develop a pattern of pre-baked strategies that would win it the game. Would it play the same build every round or would it realize that changing things up each match wins it more games?
Outrageous 9y ago
Novice here: I really want to try this Starcraft API but I don't know how to start. I believe this uses more reinforcement learning and agent-based models (which honestly I am not familiar with yet) What are good papers to get started on this?
Lambent 9y ago
It's not like this is going to create fantastic AI. Keep in mind there's been an amateur AI project for broodwar for almost 7 years now. Even after such a long learning period, the games are very primitive, and the AI's still couldn't pose a threat to even a beginner human player. Sometimes the games take hours. Trying to build strategy and decision making into an AI is incredibly complicated. There have been teams working at the SSCAIT for many years now, and the product is still fairly primitive. So what CA did was instead write up a simpler AI that mimics strategy and decision making. We all know it's not great, but I'd be really skeptical that 3rd parties would magically create an AI that can think strategically.
JabavuAdams 9y ago
When I used to play a lot of StarCraft, and then later with Total Annihilation, I wished for the ability to customize the AI. So then BWAPI came along ... and ... AI is hard. The best SCBW bots are still pretty pathetic compared to a human player, never mind an expert human player.

603 points by nijynot 9y ago

I also want to see the algorithm win on unorthodox maps. Perhaps a map they have never seen before, or one where the map is the same as before but the resources have moved. Don't tell the player or the algorithm this, and see how both react, and adapt. This tells us a great deal about the resiliency of abilities.

jmcmahon443 9y ago
I am considering a random map generator for just this reason.

ipnon 9y ago

Any predictions for how long it will take for an AI to win against the world's best player?

ktRolster 9y ago
AI can already win against the world's best player now. It's just a question of whether they can win with human-level micro.
- obastani 9y ago
  Do you have a citation? As far as I know, existing AIs do not stand much of a chance again humans [1]. My understanding is that even if AIs can beat humans is small skirmishes due to superior micro, they lose so badly at macro that it simply doesn't matter. [1] http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/custom-ai-programs-take-on-top-ranked-humans-in-starcraft http://spectrum.ieee.org/automaton/robotics/artificial-intel...
  - sidusknight 9y ago
    Yeah, you're right. Not sure what the guy is talking about.
    - keerthiko 9y ago
      > the guy is talking about I don't want to be overly semantic or PC on HN, but just saying the GP may be female, judging by their name on profile. Being misgendered could be very offputting and discourages participation, so you may have wanted to say "this person" even if it doesn't sound as offhand as you'd have liked it to come across.
      - cm2012 9y ago
        In the NorthEast US, guy is often a gender neutral term. It's totally fine to say, "Hey you guys" to a group of only women.
        figgis 9y ago
        Personal experience tells me it's a pretty common thing across the US. > I don't want to be overly semantic or PC on HN Proceeds to be overly semantic or PC on HN...
        suchow 9y ago
        Calling a group of women "you guys" is different from calling a person of unknown gender "guy" because in the former case the gender-neutral usage is the only possible interpretation, whereas in the latter it might either be gender-neutral usage or an assumption of the commenter's gender. Given that this ambiguity is nearly always a distraction from the primary purpose of the conversation (as it is here), rightfully or not, pragmatically it is better to use terms that steer clear of it. See Bryan Garner's Modern English Usage for fuller advice on safely navigating this linguistic and sociocultural terrain.
      - mpfundstein 9y ago
        I am so happy to be European... you guys in America always take everything to the total extreme.
- sidusknight 9y ago
  There isn't any SC2 AI that's anywhere near as good as a player in an actual game. Sure, they can demonstrate some sick unit control, but that alone isn't enough to win games.
- cm2012 9y ago
  Cool Starcraft related username!
aerovistae 9y ago
Awhile. This just isn't like Go or Chess. The gap from perfect information to imperfect information is quite a chasm, and from turn-based to real-time is even more vast. I play Age of Empires 2 semi-competitively, and I just can't imagine the research progress that would have to be made for a pro to lose to an APM-limited AI agent. So much of the game comes down to intuiting what your opponent is planning without being able to see what they're doing, and more importantly intuiting what your opponent isn't ready for. The biggest difference, though, is the "RT" in "RTS"-- real time. This isn't turn-based anymore, where at a given moment you have a single choice to make, a single piece to move as in Chess and Go, and can then wait for the singular and visible reaction your opponent makes before making your next choice. My understanding it that the moves a program like AlphaGo makes are not interconnected-- it picks each move individually as an ideal move for that board state. It could take over halfway through the game for someone else and would make the same move that it would have made at that point if it had been in control the whole time and arrived at that board state on its own. But that doesn't work in a real-time game, since you and your opponent are now moving simultaneously and the "board" is never static. Your moves must be cohesive and planned and flow continuously without time to ponder, each connected to the last. There is no "one" move for a given state. Another facet of real-time play is the idea of distraction. It's very important in RTS's to keep your opponent distracted, to disrupt their plans and their focus, by coming from unexpected directions at unexpected times, sometimes concurrently with other operations against them. This can't happen in Chess or Go, where the demands on your focus are far less urgent and two things can't happen at once in a literal sense. Can an AI agent learn to appreciate the power of distraction? Can it learn to intuit what will be most disruptive to a human, and what won't be disruptive at all? How can you teach a computer to learn to be annoying? I will say, of course, that nobody saw AlphaGo coming. And I hope it's the same with RTS's. That would be so exciting. I would love to see an AI blow us away with previously unthought-of strategies. That would be the coolest thing ever. So I hope it happens. But I'd be astonished. RTS is just such a whole new level of thinking for AIs.
- artursapek 9y ago
  Speaking of AoE 2, I really hope this research ends up benefiting that game. The AI in it has always been so bad but it's still my favorite game of all time.
gwern 9y ago
Churchill, of the SC AI competition, guesses it might take as much as 5 years: https://www.wired.com/story/googles-ai-declares-galactic-war-on-starcraft- https://www.wired.com/story/googles-ai-declares-galactic-war...
- bytefactory 9y ago
  Ooh, this'll be interesting to see, as with AlphaGo, a lot of people disputed the "experts believed it would take ~10 more years" claim retrospectively. With SC2, no AI even comes close to beating even a silver level player, so even a 5 year timeline seems really soon. Let's see if DeepMind can beat it! What's your totally unscientific guess, Gwern?
  - gwern 9y ago
    I think it is doable in under 5 years, but this critically depends on the resources invested by DM and other DL orgs. Deep RL is hugely demanding of computational resources to iterate your designs - for example, the first AlphaGo took something like 3 GPU-years to train it once (2 or 3 months parallelized); however, with much more iteration, DM was able to get Master's from-scratch training down to under 1 month. Now an AG researcher can iterate rapidly with small-scale hobbyist or researcher resources, but if they had had to do it all themselves, Ke Jie would still be waiting for a worthy adversary... When I look at all the recent deep RL research ( https://www.reddit.com/r/reinforcementlearning/ https://www.reddit.com/r/reinforcementlearning/ ) I definitely feel that we can't be far from an architecture which could solve SC2, but I don't know if anyone is going to invest the team+GPUs to do it within that timeframe. (It might not even be as complex as people think: some well-tuned mix of imitation learning on those 500k+ human games, self-play, residual RNNs for memory/POMDP-solving, and use of recent work on planning over high-level environment modeling\, might well be enough.) \ "Learning model-based planning from scratch" https://arxiv.org/abs/1707.06170 https://arxiv.org/abs/1707.06170 , Pascanu et al 2017; "Imagination-Augmented Agents for Deep Reinforcement Learning" https://arxiv.org/abs/1707.06203 https://arxiv.org/abs/1707.06203 , Weber et al 2017 (blog: https://deepmind.com/blog/agents-imagine-and-plan/ https://deepmind.com/blog/agents-imagine-and-plan/ "Agents that imagine and plan"); "Path Integral Networks: End-to-End Differentiable Optimal Control" https://arxiv.org/abs/1706.09597 https://arxiv.org/abs/1706.09597 , Okada et al 2017; "Value Prediction Network" https://arxiv.org/abs/1707.03497 https://arxiv.org/abs/1707.03497 , Oh et al 2017; "Prediction and Control with Temporal Segment Models" https://arxiv.org/abs/1703.04070 https://arxiv.org/abs/1703.04070 , Mishra et al 2017

ktRolster 9y ago

When Watson won at Jeopardy, one of its prime advantages was the faster reaction time at pushing the buzzer. The fairness of that has already been hashed out elsewhere, but..... We already know that computers can have superior micro and beat humans at Starcraft through that(1). Is DeepMind going to win by giving themselves a micro advantage that is beyond what reasonable humans can do? (1)https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs as one example

sidusknight 9y ago
Alternatively, http://www.youtube.com/watch?v=DXUOWXidcY0&t=0m50s http://www.youtube.com/watch?v=DXUOWXidcY0&t=0m50s
obastani 9y ago
My understanding is that in a full match, AIs still have no hope against humans, since even though they can crush humans at micro, their macro is still abysmal [1]. I'm not aware of a match where any AI has beat a pro human player at Starcraft -- I'd be interested in learning otherwise! [1] http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/custom-ai-programs-take-on-top-ranked-humans-in-starcraft http://spectrum.ieee.org/automaton/robotics/artificial-intel...
- flamedoge 9y ago
  would you love to be proven wrong?
  - sidusknight 9y ago
    Of course.
  - deleted 9y ago
    [deleted]
- apetresc 9y ago
  That's because there hasn't been too much concentrated effort on this problem yet, since you'd have to spend quite a bit of effort just integrating with the game engine. Certainly a lot less research has been done on computer SC2 than computer go, and nobody expected a pro to be beaten there 1.5 years ago, either.

siegecraft 9y ago

The API Blizzard is exposing is really nice. Sadly most of the advantages AI had in SC1 were just due to the fact that an automated process could micro-manage the tasks the game didn't automate for you (a lot of boring, repetitive work). SC2 got rid of a lot of that while still allowing room for innovative and overpowered tactics to be discovered (MarineKing's insane marine micro, SlayerS killing everyone with blue flame hellions, some more recent stuff I'm sure from the newest expansions). Hopefully the API lets AIs converge on optimal resource management and get to exploring new and innovative timings, transitions, army makeups, etc.

captainmuon 9y ago
> SC2 got rid of a lot of that You think so? My impression is that SC2 had a lot more of repetitive tasks you had to do. E.g. wall off the ramp, send a worker scouting, ... and you have to perform certain actions every X seconds (like using chrono boost). A lot or mastering the game is rote learning, and polishing a build order. Another big part is constantly scouting and reacting to what the enemy is doing. Due to those reasons I found SC2 a bit tedious (it was still fun, just felt more like work than SC1). Granted, this is maybe because I played SC1 more on LAN, and there wasn't all the metagame going on. But I think SC2 really does focus on "grinding" and rote learning to get better, probably this was chosen to make it more "eSports"-like. If I would get to design a SC2.5 or SC3, I would remove all the rote - the actions you always have to perform - and I would give the player the opportunity to trade off more between macro and micro. Actually, it would be cool if you could "research" certain AI features in game for a cost. For example, have one upgrade that micros your marines like a pro, or positions your units in sensible formations. Another player could counter this with a "radio jam" ability, that would make your units in an area take bad formations, or be controlled by a very simple AI. And if you are good at micro, you could save the update, or invest it in an update that makes macro simpler. And so on, I think there are a lot of things one could explore there. Maybe opening SC2 to AI exploration can lead to such gameplay innovations.
- CuriouslyC 9y ago
  I really like the idea of a helper AI that takes high level commands and handles the manual dexterity aspect of the game. That would make RTS games a lot more attractive for someone like me who prefers simple, thoughtful games rather than complex games with a steep learning curve.

daemonk 9y ago

Blizzard should put in an AI-assisted play mode where players are limited to X lines of code that can be launched with keyboard commands.

neuronexmachina 9y ago
That would be quite interesting, having humans handle the macro while the AI focuses on the micro. I'm reminded of "Advanced Chess": https://en.wikipedia.org/wiki/Advanced_Chess https://en.wikipedia.org/wiki/Advanced_Chess > Advanced Chess is a relatively new form of chess, wherein each human player uses a computer chess program to help him explore the possible results of candidate moves. The human players, despite this computer assistance, are still fully in control of what moves their "team" (of one human and one computer) makes.
ajkjk 9y ago
I know that, as a player, the high mechanical limitations of Starcraft are part of why it's such a difficult, high-skill-ceiling game. But.. I've tried to enjoy watching SC2 on Twitch, and while it's kinda fun, it's just so disappointing when a complicated strategic game is thrown away because a player doesn't react fast enough to workers being sniped or a drop being shot down. I wish the individual units had some automatic behavior -- for example, marines would could run in spread out formations near tanks or banelings; workers would flee from hazards; flying units would avoid turrets unless specifically directed to fly over them. It would require a lot of rebalancing, of course, but it would make the game so much more tactical and strategic and (imo) enjoyable to watch.
- jbob2000 9y ago
  You would like Company of Heroes 2, the units require much less micro and the game is a bit slower paced. If your infantry come under fire, they all dive into cover to protect themselves. Defense is a bit more automated (you can set artillery to automatically return fire on enemy artillery once it's exposed) and you have more options for static emplacements to defend critical areas without needing your attention. I have the same complaints about SC2 as you - way too fast and intense - and I really enjoy Company of Heroes. The whole game is basically set up to get rid of "nags" so you can just play around with the units and have fun.

Havoc 9y ago

That's surprising. I thought Bliz didn't want anyone near sc2 but approved of sc1 being used for this purpose.

yflu 9y ago
SC1 really doesn't make sense for this, 80% of the skill is just keeping on top of the mindless but mechanically intensive stuff, which is trivial beyond trivial for an AI. SC2's automated away most of this (pretty much everything but production cycles), which makes it a better measure for AI vs human.
- eatitraw 9y ago
  I agree that SC2 is much easier for a human player. However, top players still have to do quite a bit of micromanagement.
- jon_richards 9y ago
  A lot of the balance of SC2 will immediately be destroyed by a semi-competent AI. One of the races has especially powerful early tactics that are primarily limited by a human's inability to multitask (reapers, medivacs, and liberators). I honestly don't think it will be possible for a human to beat an AI that just focuses on those strategies.
- solicode 9y ago
  If they're limiting APM to that of human levels, I don't see it being much of an issue though. APM would just be a limited resource like any other. In fact, I sort of want to see how a strong AI would choose to spend its limited pool of actions. How different would it look compared to a pro? Maybe not much, but I'm not actually certain. In SC1, just the act of moving a large army is a commitment and takes quite a few resources. Moving your armies under fog of war and not letting your opponent know exactly how you're set up in order to get a good angle on you is incredibly important. I want to see how much of an importance a great AI puts on that vs the other things it could be doing instead. Are the strongest AIs going to be more methodological, safer, and slow moving? Or will the best AIs try to exploit the imperfect information aspect of the game and try to lure the opponent into making a wrong decision? I feel like AIs tend to excel at the former, but the latter has been a huge component for the very best pros in SC1. I don't follow SC2, so I don't know much of this also applies there. I just feel like SC1 isn't as mechanical as it's made out to be. There's definitely that huge initial barrier, but once passed that, the game actually feels very delicate and is about good use of resources (including mouse/keyboard actions), timing, and transitions in unit composition to catch your opponent off-balance.

toisanji 9y ago

great they opened it up. I'm sure reinforcement learning / Deep learning will solve this. It has been a tough problem before, but honestly doesnt seem that tough compared to all the harder AI problems.

Synaesthesia 9y ago
Such as?

dpflan 9y ago

Related: Today I learned that a group of AI researchers has released a paper called: STARDATA: A StarCraft AI Research Dataset. According to one of the authors: "We're releasing a dataset of 65k StarCraft: Brood War games, 1.5b frames, 500m actions, 400GB of data. Check it out!" > Article: https://arxiv.org/abs/1708.02139 https://arxiv.org/abs/1708.02139 > Github: https://github.com/TorchCraft/StarData https://github.com/TorchCraft/StarData

tgb 9y ago
The great thing about this is that it includes the game state throughout the game. It's been pretty easy to find lots of Starcraft replays, but the replays only include enough information to recreate the game (basically just the player actions). If you wanted to know what was happening in the game at the time the player made an action, you had to load up Starcraft and simulate out the game until that point. This dataset has already run the game for you and provided the data!
- wfunction 9y ago
  Is it that much computation to simulate an entire game? You obviously don't need to render the graphics or anything, it should just be a list of events that occur, which doesn't seem all that slow to process.
  - gwern 9y ago
    Until today's release of the headless Linux client, you still had to run the full StarCraft program, which gets expensive fast. And it massively complicates the workflow to have to play through every game serially to recreate the state rather than simply reading random rows of data from a 300GB dataframe on disk.
    - wfunction 9y ago
      Oh I see, thanks, I didn't know. But man, 300 GB per game sounds completely nuts!

JefeChulo 9y ago

"so agents must interact with the game within limits of human dexterity in terms of “Actions Per Minute”." I am really glad they are limiting APM because otherwise things just get stupid.

usaphp 9y ago
Would not actions per second be a better limit tho?
- yannyu 9y ago
  Why? Speed is in MPH or KPH, would it be better to go by m/s? The convention in gaming is APM, so they're just using the nomenclature that is already understood.
  - pavedwalden 9y ago
    Maybe what usaphp is getting at is that the AI could still gain an advantage by doing a set of actions much faster than humanly possible in just a fraction of a second as long as it kept its total number of actions that minute below the cap.
    - vmasto 9y ago
      Exactly. Most actions in idle game mode (i.e. when there's no active battle, micro or macro to be done) are null, the gamers simply repeat random meaningless keystrokes; this helps with being always alert and ready for anything that requires a rapid response. An AI doesn't need to do this. Source: Ex pro Starcraft (Brood War) gamer.
      - sidusknight 9y ago
        There no were BW pro-gamers outside of Korea.
      - taneq 9y ago
        > this helps with being always alert and ready for anything that requires a rapid response. I always wondered why they do that, just spam meaningless things like open/close the stats window or click on the background. Some WoW PVPers did it too (like Laintime, I think it was - played a warrior and ran around spamming the character window open/closed an mashing 'weapon swap' even when there was no earthly reason to do so). I figured it was just the result of too much caffeine for 18 hours a day. This sounds like it's more an equivalent of tennis players' dancing/hopping around, or martial artists doing their ducking/weaving thing?

hitekker 9y ago

This seems all in good fun but I wonder if it's come too late. Starcraft 2 is at its twilight. The biggest leagues of South Korea have disbanded. [1] The prolific progamers who transitioned to Starcraft 2 have gone back to Broodwar. [2] Blizzard itself has scrubbed all references to Starcraft 2 on the very home page of Starcraft. [3] Except for the twitter embed, it has only only one "2" character... in the copyright statement. My take is that the future for the Starcraft franchise will be through remastered and potential expansion packs following it. Starcraft 2 had a good run but, with the entire RTS genre stagnating [4], I don't think Blizzard wants to bet on anything less than the top horse. [1] https://www.kotaku.com.au/2016/10/the-end-of-an-era-for-starcraft-and-south-korea/ https://www.kotaku.com.au/2016/10/the-end-of-an-era-for-star... [2] http://www.espn.com/esports/story/_/id/18935988/starcraft-brood-war-glory-days-jaedong-best-bisu-talk-starcraft http://www.espn.com/esports/story/_/id/18935988/starcraft-br... [3] http://starcraft.com http://starcraft.com [4]http://www.pcgamer.com/the-decline-evolution-and-future-of-the-rts/ http://www.pcgamer.com/the-decline-evolution-and-future-of-t... (Aside from MOBAs)

cjbprime 9y ago
I don't quite agree, FWIW. SC2 does seem to be at its twilight in Korea, and I agree progamers and fans there are super interested in Remastered. But I don't think Remastered will be very popular outside KR. The SC2 "war chest" promo appears to have made more money than expected, as measured by hitting its funding ceiling within a few days. So I don't think it's "Remastered replaces SC2", I think it's a divergence into KR playing Remastered and non-KR playing SC2, and the number of progamers and players doesn't have to be zero-sum: it could enlarge the population playing either game, too.
- doppel 9y ago
  As an esport / spectator sport, SC2 has been waning for a long time, and similarly the War Chest was capped at 200.000 for prize pool money and an unknown amount after that (which, compared to Dota 2 or League of Legends prize pools, is not a lot). I think, given the matchmaking update for remastered, that SC1 will see a resurgence both inside and outside of KR, but I am not sure either SC1 or SC2 will stay competitive in the long run.

deleted 9y ago

[deleted]

blobbers 9y ago

YESSSSSSS!!!!!!!! --why are there not more fanboy comments?!

hughes 9y ago
Probably because they don't contribute much to the conversation.

qub1t 9y ago

A lot of people here seem to be underestimating the difficulty of this problem. There are several incorrect comments saying that in SC1 AIs have already been able to beat professionals - right now they are nowhere near that level. Go is a discrete game where the game state is 100% known at all times. Starcraft is a continuous game and the game state is not 100% known at any given time. This alone makes it a much harder problem than go. Not to mention that the game itself is more complex, in the sense that go, despite being a very hard game for humans to master, is composed of a few very simple and well defined rules. Starcraft is much more open-ended, has many more rules, and as a result its much harder to build a representation of game state that is conducive to effective deep learning. I do think that eventually we will get an AI that can beat humans, but it will be a non-trivial problem to solve, and it may take some time to get there. I think a big component is not really machine learning but more related to how to represent state at any given time, which will necessarily involve a lot of human-tweaking of distilling down what really are the important things that influence winning.

strgrd 9y ago
I wonder if we will see any advanced cheese strats come out of this. I'm assuming some implementations will eventually develop micro control that is far beyond any human player's capabilities, which would make things like all-in probe rushing much more viable. Instead of playing the normal meta in a computer-vs-human, I imagine an advanced AI would simply send all of its workers off the mineral line as soon as the game starts, and attempt to out micro the human opponent before they can build an army-producing building.
- ionforce 9y ago
  I know this isn't the exact same as the article, but when genetic algorithms were introduced to solve for build orders, the "seven roach rush" was in vogue, something that was unexpected at the time and "discovered" using GA. I think there is a space for finding strategies that have more leeway in execution and thus are more suitable for humans to pilot rather than have machine level micro.
  - 9y ago

SiempreZeus 9y ago

It's a bit too bad they're having to move towards supervised learning and imitation learning. I totally understand why they need to do that given the insane decision trees, but I was really hoping to see what the AI would learn to do without any human example, simply because it would be inhuman and interesting. I'm really interested in particular if an unsupervised AI would use very strange building placements and permanently moving ungrouped units. One thing that struck me in the video was the really actively weird mining techniques in one clip and then another clip where it blocked its mineral line with 3 raised depots...

Synaesthesia 9y ago
Well the unsupervised ai couldn’t even do basic tasks from the video I saw, so looks like we have a long way to go.
dontreact 9y ago
They can always finetune using RL later. Superversied training was the first step at making AlphaGo work.

naveen99 9y ago

> even strong baseline agents, such as A3C, cannot win a single game against even the easiest built-in AI. Then, why not release code for the built in ai, and improve on it ? Or is the built in ai cheating ?

Synaesthesia 9y ago
The built in ai is scripted whereas they’re trying to teach agents to learn the game from scratch with some sort of reward-based/machine learning approach.
- naveen99 9y ago
  still, why not use machine learning to tweak the script instead of starting from scratch.
mattnewton 9y ago
The goal is to "grow" an AI through reinforcement learning and other techniques that are broadly applicable outside starcraft. The existing rules based systems in the game A) are extremely Starcraft specific and the methods have comparatively limited utility outside video game AI, B) unlikely to be scalable to fairly beat a professional human (which introduces too much variability to capture reliably into rules)

DefNotARogueAI 9y ago

This gives me great ideas

arnioxux 9y ago

Are there any known arbitrary code injection for starcraft? Like how you can use a regular controller to reprogram super mario world to play pong? https://www.reddit.com/r/programming/comments/1v5mqg/using_bugs_in_super_mario_world_to_inject_new/ https://www.reddit.com/r/programming/comments/1v5mqg/using_b... https://bulbapedia.bulbagarden.net/wiki/Arbitrary_code_execution https://bulbapedia.bulbagarden.net/wiki/Arbitrary_code_execu... Is this how we are going to accidentally let AGI loose into the world!? /s On a more realistic note I think this will degenerate into a game of who can fuzz test for the best game breaking glitch. Think of all the programming bugs that turned into game mechanics in BW that we haven't discovered for SC2 yet: http://www.codeofhonor.com/blog/the-starcraft-path-finding-hack http://www.codeofhonor.com/blog/the-starcraft-path-finding-h...

hacker_9 9y ago

There's something funny about a company that is actively developing bleeding edge AI technology, but who can't design a webpage that works on mobile without crashing.

chii 9y ago
Just goes to show how complicated web tech is, even ai researchers can't get it right!

deleted 9y ago

[deleted]

siliconc0w 9y ago

The SCAI bots I've seen are more hardcoded tactics engines rather than machine learning models. They're still impressive, but their logic isn't quite 'learned' it's hand coded which is a crucial difference.

captn3m0 9y ago

Someone needs to link this to FB's ELF platform (An End-To-End, Lightweight and Flexible Platform for Game Research). That was specifically made for RTS games like SC.

deleted 9y ago

[deleted]

bretepete 9y ago

ATTN: We Provide Medical Marijuana to persons in good need of it.we strive to keep both faith and hope alive and most of good health of mind and body.please for more info,reach us on dr.rickerbs0@india.com

onorton 9y ago

I think I know what my final year project will be.

Ntrails 9y ago

I'd be really interested in how differently tiered data sets (ladder rank) would work as sources for teaching. Is it possible that training on diamond players is less effective than training on, say, silver? Is that actually even an interesting thing to look at?

krasi0 9y ago

The StarCraft 1 BroodWar AI scene has been thriving for a few years now: https://sscaitournament.com/ https://sscaitournament.com/ You can watch 24/7 live AI vs AI games on Twitch at: https://www.twitch.tv/sscait https://www.twitch.tv/sscait Support for voting on who to play next and even a betting system are in place, too. For those who wish to get their feet wet with BW AI development, here are the Java / C++ tutorials: https://sscaitournament.com/index.php?action=tutorial https://sscaitournament.com/index.php?action=tutorial

krasi0 9y ago
Some thoughts and analysis on why Starcraft AI by one of the active AI developers Dan: https://dangant.com/2017/08/09/why-starcraft-ai/ https://dangant.com/2017/08/09/why-starcraft-ai/

Magnificents 9y ago

Can we have an AlphaGo tournament with the StarCraft II AI, where pros fight the bot, lose horribly, and make them question their life decisions?

deleted 9y ago

[deleted]

Companion 9y ago

convefefe 9y ago

I thought this was already happening. Right after AlphaGo beat Lee, I remember hearing about it. Did they give up on having their AI playing SC2? I wondered if that would work, since it seemed to take turns in Go at the same speed as a normal player, I wondered if it was trying to compute the most likely winning move each turn and the late game implications of those moves. If it tried that in a fast paced game how it would deal with the speed. It obviously would need to develop a pattern of pre-baked strategies that would win it the game. Would it play the same build every round or would it realize that changing things up each match wins it more games?

Outrageous 9y ago

Novice here: I really want to try this Starcraft API but I don't know how to start. I believe this uses more reinforcement learning and agent-based models (which honestly I am not familiar with yet) What are good papers to get started on this?

Lambent 9y ago

It's not like this is going to create fantastic AI. Keep in mind there's been an amateur AI project for broodwar for almost 7 years now. Even after such a long learning period, the games are very primitive, and the AI's still couldn't pose a threat to even a beginner human player. Sometimes the games take hours. Trying to build strategy and decision making into an AI is incredibly complicated. There have been teams working at the SSCAIT for many years now, and the product is still fairly primitive. So what CA did was instead write up a simpler AI that mimics strategy and decision making. We all know it's not great, but I'd be really skeptical that 3rd parties would magically create an AI that can think strategically.

JabavuAdams 9y ago

When I used to play a lot of StarCraft, and then later with Total Annihilation, I wished for the ability to customize the AI. So then BWAPI came along ... and ... AI is hard. The best SCBW bots are still pretty pathetic compared to a human player, never mind an expert human player.