18 ms·

DeepMind and Blizzard to release StarCraft II as an AI research environment

917 points by madspindel 10y ago

Miner49er 10y ago
Looks like they are going to limit the APM of the AI. I wonder how they are going to decide the limit? I've never played StarCraft, but from what I understand very high APM is needed to play the game at the highest levels.
- nip 10y ago
  Probably an average of the best players.
- tylerpachal 10y ago
  I've followed the professional StarCraft 2 scene for a long time now. There were a few players who were famous for having high APMs, here is an over-the-sholder view of one of them [1]. I don't have a reference but believe he used to play in the ~500APM range. That would probably be the upper limit of a human's ability. [1] https://youtu.be/WHcl6Cs1FAM?t=7m57s https://youtu.be/WHcl6Cs1FAM?t=7m57s
  - hughes 10y ago
    I wouldn't be surprised if a 500APM player is making closer to 100 meaningful actions per minute. It looks like most of the actions are spam that may register on the APM meter but not affect the outcome of the game. An AI limited to even 200 decision-driven actions per minute would probably have a significant advantage over a "500APM" player.
- exelius 10y ago
  I imagine they'll have separate AI brackets, and they could potentially simulate at multiple APM levels.
- areign 10y ago
  probably limit it to human levels. 400 to 600 apm is around the limit for brief periods of time. It makes for an interesting optimization problem with how to use them though.
- Aweorih 10y ago
  High apm is required, yes. But most people push it by pushing it through spamming 1-2-3 (groups of your army). There was a pro player who didnt do it and he was around +- 70 APM.
  - Cafey 10y ago
    It would be interesting to see what percentage of a pro player's APM is actually useful. Like you said, most of the time they are spamming buttons to keep their momentum up.
    - placeybordeaux 10y ago
      There is a replay analyzer that attempts to figure this out, generally the effective APM will go up during a fight, but the players will keep the APM fairly high even when the eAPM is not high just because it is easier to go from many useless actions to many useful actions than few useful actions to many useful actions.
    - Aweorih 10y ago
      I think they are doing it to stay "warm". E.g. if something happens that the fingers can react quick and you don't need a "warmup" time.
- fazzone 10y ago
  There's probably no reason to restrict the computer the below human pro players (200-300 APM). My guess is the engine just wasn't designed with superhuman APM in mind and could be exploited. If that's the case then the AI would in some sense not really be playing the same game as the humans.
  - stale2002 10y ago
    An apm limit is needed because impossibly high APM results in stupid strategies that the game wasn't designed for. The point is to build ai that compete with humans using human strategies. Not to cheese with borked game mechanic exploits.
    - TulliusCicero 10y ago
      In practice right now the AI still loses even with unlimited APM. It's just too dumb to really take advantage of its execution speed.
      - nogbit 10y ago
        Buts it's intentionally dumbed down to keep the game fun, loosing all the time would get frustrating real quick. Theres no ROI for AI devs to kick the humans ass (except for maybe now with our AI/Cloud/VR/ craze).
        stale2002 10y ago
        No it is not. People were making custom AIs for starcraft 2 since it was in beta. They can't beat any human worth their salt.
        TulliusCicero 10y ago
        It is not intentionally dumbed down. Current AIs are just too stupid at strategy and tactics to beat competent human players.
  - SerLava 10y ago
- DontGiveTwoFlux 10y ago
  There was a Starcraft brood war AI competition that did not have a limit on APM. The top contestants were employing strategies that required APM an order of magnitude higher than what players could do. For example, in brood war, SCVs can repair any Terran building or vehicle but it is not worth the effort to repair goliaths or tanks for a player. However, the AI could easily manage repairing it's tanks which makes the Terran army much more cost effective. In SCII there is auto repair and other improved UX that probably limit the ability of novel strategies like repairing vehicles.
  - placeybordeaux 10y ago
    This was an example of extreme micro that a very simple AI can do that changes the game entirely: https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs
    - wlesieutre 10y ago
      How does it seem to know which zergling is about to get hit?
      - acomar 10y ago
        If you slow it down, it would be apparent to you too. High-level human players do this kind of thing to, just not this fast. But effectively, it's watching where the attacking unit is facing and it knows when the next attack is coming, so it knows what the viable targets are and can react when it becomes obvious. (it's also possible the system is receiving information directly from the game engine, but it's definitely possible to figure this out from nothing but the information on screen)
      - caylus 10y ago
        Because the tanks are using the default unit AI, which AFAIK amounts to "when your weapon is ready to fire, shoot the closest enemy in range". The zergling AI knows the attack range and rate of each tank so can ensure that the closest zergling is always on its own when a tank is ready to fire. What would be really interesting is putting a similar smart AI on the tanks, which could make decisions like shooting at the highest concentration of zerglings instead of the closest one, or holding its fire if one lone zergling is in range, but a clump of them is about to enter range. Human players already do both optimizations with widow mines, which are a unit that takes several seconds to prepare its shot so it's feasible to retarget. It's not feasible for humans to do this with tanks though as they fire immediately and automatically. You could even see counterevolution as two AI algorithms become better at predicting what the other will do.
- nharada 10y ago
  Back when I played SC2, you really didn't need a high APM to succeed, unlike SC1. SC2 offered some helpful mechanics that SC1 did not, such as the ability to select infinite units at once, or give commands to multiple structures at one. I think Sheth, a high level Zerg player, was famous for having under 100 APM.
  - thaumasiotes 10y ago
    I'll consider APM "not high" when it gets down to 12. Two actions per second is comfortably "high".
    - uw_rob 10y ago
      That may be an ok definition for you, but it most certainly isn't for SC2. I wasn't anywhere near the pro level and I would peak at 300 apm and would spend most of the game around 200. 12 APM is borderline not playing.
      - thaumasiotes 10y ago
        Consider it a statement about what I'm looking for in a game. I'm not really interested in competing over who can work their mouse button harder, or in driving myself to excel in giving myself repetitive motion injuries. Those are things that take away from starcraft; they don't enhance it.
        halflings 10y ago
        I would strongly encourage you to go against your preconceived idea of what StarCraft is (a game where people compete on who's the fastest to bash their keys) and see a professional game and all the strategy it entails, such as in the semi-finals and grand final that will be on today. Pro players do have valid reasons for having such a high APM (even if a portion of it, maybe 30%, is just "spamming" to keep theirselves active and ready), but you can be an excellent player with rather low APM (~120 maybe?). Why would you need to do 2 action per second? A lot of it is simply building units periodically (e.g pressing 1 then "S" every 10 seconds) but then APM really gets high when there is a battle and you need to give actions to multiple sets of units to get a good engagement.
- JabavuAdams 10y ago
  The best SCBW bots still aren't very good, so I don't see unrestricted APM as an unfair advantage.
- lanius 10y ago
  Unfortunate. I personally would love play against an AI with strong game sense as well as insane micro a la https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs
- Retr0spectrum 10y ago
  For anyone else wondering, APM = Actions Per Minute.
ChrisAntaki 10y ago
> We’re particularly pleased that the environment we’ve worked with Blizzard to construct will be open and available to all researchers next year. This is awesome. I've only ever reached the Platinum league in Starcraft II (1v1), but I'd almost feel more driven to create bots to (hopefully) surpass that skill level, than actually playing the game.
luka-birsa 10y ago
Anybody else sees a problem with training the AI to move troops around a battlefield, with the purpose to exterminate the opponent?
- harigov 10y ago
  It's going to happen anyway. I think we are better off doing such things in the open and having some visibility into it rather than be surprised when something happens.
  - treehau5 10y ago
    I'm so sick of this argument. How about we have a spine for once and say no?
    - danielpatrick 10y ago
      The problem is we is not everyone... it's only those who agree with you. Everyone else will just keep on pushing ahead, hence the less-bad option of not relegating that advancement to dark corners.
    - webkike 10y ago
      All 7 billion of us? Good luck with that.
      - Mtinie 10y ago
        I agree with you, it's a virtually impossible task. The commenter I was responding to has strong ideas about it however, so I wanted to provide the opportunity for an explanation.
    - Mtinie 10y ago
      Ok, for conversation sake we all come together and say, "No, AI cannot be used for offensive or defensive military actions". How do you propose that ban be enforced?
    - 10y ago
KZeillmann 10y ago
This is so exciting. I've always wanted to program bots to play online games -- mainly for learning purposes. (Can I make a bot that plays better than me?) But I've never done it because of the risk of bans. I'm glad that Blizzard has opened it up for people to experiment with this. I wonder how it will interact with any sort of anti-cheat systems in place, etc.
- tylerpachal 10y ago
  This has been around for awhile [1]. They give you an API to use and then you upload your JAR. They have a Twitch channel [2] that is constantly streaming AIs playing against each other in their tournament. [1] http://sscaitournament.com http://sscaitournament.com [2] https://www.twitch.tv/certicky https://www.twitch.tv/certicky
  - pdimitar 10y ago
    This has to be the most interesting link in this whole thread. Thank you so much!
- formula1 10y ago
  Bans should never keep you back. A fake email is pennies to how much is gained from learning or building something incredible
  - garrettgrimsley 10y ago
    A fake email won't help when you are banned by your unique ID that you paid, in the case of SC2: Legacy of the Void, at least $40 for.
    - formula1 10y ago
      Is $40 really not worth a year or more of experimentation? You can also do a lot of learning through replays or vs AI. I personally am willing to pay $40 for an environment to test my ai on for a couple months.
formula1 10y ago
I suspect this will eventually lead to AI as a service for games. Rather than build a terrible AI that delays a game by months, approaching a company that can build a decent AI initially which gets better overtime would probably be ideal and create better experiences. Im curious if a startup can be built from this.
- doikor 10y ago
  Is it harder to make an AI that is mediocre at playing the game vs one that is good? That is the main issue with making good AIs for games now. We want AIs (if you can call them that) that play kinda like humans but worse then the player as the player wants to win. Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
  - Voloskaya 10y ago
    Having checkpoints along the way as you train your model could be a good way to allow fine-grained level selection. I guess :)
    - LesZedCB 10y ago
      That would be funny. The difficulty slider changes to "How many hours of playtime" instead of easy, medium, hard. Actually it could even just scale it automatically based on a metric like steam hours played. that would be awesome.
    - aero142 10y ago
      I don't think so. Many of the fun things AIs do in games is attack at different stages of the game. They send a small force at the beginning, and then a little larger force a few minutes later, and then a big attack at the end. No AI trying to win would end up at this strategy. You would have to define the fun goals and have the AI train on that for it's success criteria.
Savageman 10y ago
That's so cool! I wish they could start doing AI for team based competitive games like League of Legends where meta-play and team decision making is important. Is that too complicated to tackle yet?
- rcthompson 10y ago
  Are you imagining a single AI playing all 5 heroes on a team, or 5 independent AIs each playing a single hero with limited communication between them, or a single AI controlling a single hero on a team with 4 humans? I would say the first scenario (1 AI controlling entire team) is roughly equivalent to the the RTS case. It's just fewer units with more abilities, and character progression instead of base building. The other two cases are probably a lot more complicated.
  - Savageman 10y ago
    Either one could work. If the single AI that controls 5 heroes is easier to program, let's imagine that. I have the feeling that it's more complicated than a 1vs1 RTS game, but I don't know why...
    - CydeWeys 10y ago
      Controlling five units is a lot easier than playing an RTS. In an RTS you don't just control five units, you control potentially hundreds of units, and you have base-building and an economy to manage.
  - manux 10y ago
    Honestly all these scenarios are interesting research avenues. Inter-agent communication, especially in a machine learning context, is an unsolved problem, exploring it in such a scenario would be nice. Agent-human communication is probably as hard, and also largely unsolved, although there is a push right now for dialog systems to achieve human level performance.
  - SketchySeaBeast 10y ago
    I can't wait to see the first team of 5 separate AI learn to insult each other. It'll be a watershed moment in history when the first vote to kick another AI occurs.
randyrand 10y ago
I hope the AI will have a handicap on the speed of mouse movement. IRL you can't just teleport the mouse around the screen.
simpfai 10y ago
Does anyone know of any resources for someone looking to learn more about Game AI for real time strategy games?
- luminiferous 10y ago
  The Berkeley Overmind was designed for a 2010 Brood War competition. The website (http://overmind.cs.berkeley.edu/ http://overmind.cs.berkeley.edu/), although no longer updated, still contains many videos and links to articles that describe the internals of the AI and the design process. The Ars Technica article in particular goes quite in depth.
- PetahNZ 10y ago
  http://aigamedev.com/ http://aigamedev.com/
andrew3726 10y ago
This is really good news! Lets hope DeepMind can improve even further on their Differentiable neural computers (DNC) which seems like an requirement for this kind of AI to work (exploiting long-term dependencies). I also hope that other research/industry teams will join on the competition to create competing AIs. Very exciting!
lanius 10y ago
I can't wait to see how far DeepMind can go in this area. I was initially skeptical that AlphaGo could defeat top human players, but then it happened. Who knows, perhaps one day AI can compete against progamers in GSL!
- placeybordeaux 10y ago
  Competitive AI in broodwar already can destroy the best humans. AI can use strategies and tactics that human can't effectively do. Most notable is perfect splitting which changes the game quite a bit.
  - erik 10y ago
    Do you have a reference on this? Last I looked into it broodwar AI bots were easily beaten by competent humans.
  - lanius 10y ago
    >Competitive AI in broodwar already can destroy the best humans Who do you mean by "the best humans"? I've never heard of top-tier progamers such as Flash, Jaedong, Bisu, etc. losing to AI. In fact, I doubt any B-teamer or above would lose to an AI in a Bo3.
felix_thursday 10y ago
OpenAI vs DeepMind?
flamedoge 10y ago
This is dangerous. Overmind will come to life.
- luminiferous 10y ago
  The UC Berkeley AI for Brood War (designed for the 2010 AIIDE StarCraft competition) was literally named the Berkeley Overmind. http://overmind.cs.berkeley.edu/ http://overmind.cs.berkeley.edu/
- dmichulke 10y ago
  And this time it can't be drugged.
brylie 10y ago
It would be cool to see a project like this for an open source game, such as OpenTransportTycoonDeluxe (http://www.openttd.org/ http://www.openttd.org/). The AI developed by interacting with the OpenTTD economy might even prove useful for urban planning of real geographic regions.
deleted 10y ago
[deleted]
komaromy 10y ago
My mostly-uneducated o/u on the time to beat the best human players is 8 years.
yaboyhud 10y ago
plug: https://medium.com/new-game/comp-stomp-b8fe687fbeb6 https://medium.com/new-game/comp-stomp-b8fe687fbeb6
partycoder 10y ago
That's great. AIs on SC1 relied on many hacks. Initially I thought that DeepMind was going to create a bot for the original SC. I hope some of the advances in SC2 AI can be integrated into the in-game AI. e.g: a trained neural network that plays better than the "hard" AI, but can run on a consumer box and not on a massive cluster.
rkuykendall-com 10y ago
This would be so much more fun with a turn-based game where speed isn't a variable, like Civilization. I'd love to play against several AIs that were better than me because of code, not because they get a bunch of extra in-game bonuses. With a nice AI API, you could have online competitions where AIs battled every month.
noiv 10y ago
Hopefully there's a JS/Spidermonkey interface. I'd be happy to port Hannibal from 0AD.
- tgb 10y ago
  What does Hannibal do that makes it portable? My impression is most game AI is essentially completely game specific, outside of, eg, pathfinding.
bluebeard 10y ago
This will be good for games moving forward due to the meta changing for players as the AI adapts to their tactics and vice versa. Lessons learned from this can then be applied to other areas. And as an added bonus it creates more interest in AI research.
tdaltonc 10y ago
I'd love to see a e-sport league where the teams are AI human hybrids (Centaur teams). We know that AI human hybrid teams are great at chess [1], and I'd love to see rts games played by 'Centaur' teams. In the same way the innovations made in F1 often trickle down to consumer cars, can you imagine the advances that could be made in human-machine interactions in the crucible of a real-time Centaur gaming league? [1] http://bloomreach.com/2014/12/centaur-chess-brings-best-humans-machines/ http://bloomreach.com/2014/12/centaur-chess-brings-best-huma...
- bradleyjg 10y ago
  If they allowed you to script micro unit behavior in some way that'd be first step in a hybrid game.
  - tdaltonc 10y ago
    Either that or a custom alerts system.
  - hooloovoo_zoo 10y ago
    It would also be cool if it was an upgrade you unlock. So, instead of just stat upgrades, you could also purchase AI upgrades.
    - make3 10y ago
      I'm sure in the next decade we will see the start of modern ai in video games & all ai applied to all kinds of gameplay mechanics, like this one
    - bradleyjg 10y ago
      I'd rather that the coding was something that was left to the player or to the third party market. Right now programming games are kind of boring, more like a way of prettying up homework than anything else. A really fun game where programming could give you a competitive edge would be something to see. The closest thing I can think of is add-ons for MMOs, but there the game developers specifically try to prevent add-ons that are too performance enhancing instead of making the coding an integral part of the game.
simopaa 10y ago
I would love to keep a 24/7 stream open where the AI is continuously playing against the game's own bots in order to improve its playstyle.
- wesleytodd 10y ago
  I would probably watch that
- Houshalter 10y ago
  Typically they train the AI against itself. That gives it a continuous skill gradient. Otherwise it would lose all of the games without learning anything. Since there are no successes to learn from. Or it would win all the games after it figures out how to exploit the built-in bot. With strategies that probably won't apply to humans.
tylerpachal 10y ago
For anyone looking for more information about Starcraft 2, the world championship is on this weekend and the broadcast for today has just started (16:30EST) https://www.twitch.tv/starcraft https://www.twitch.tv/starcraft
l34ch 10y ago
Pretty soon AI for games will be so good that we won't have to play online multiplayer for a really good authentic experience. Once we can't tell the difference between the AI and an actual human playing.
fitzwatermellow 10y ago
Training data? After all, AlphaGo trained on a database of over 30M expert human moves. I suspect one championship round from Team EnVy is worth billions of iterations of random exploration ;) Kudos to both Blizzard and DeepMind. Anticipating a lot of fun with this. StarCraft 2 could indeed become the AI pedagogy standard.
pizza 10y ago
Nice! I cracked a joke about how SC2 was nothing more than an AI testbed just last week, lol. Very glad to see it's becoming a real thing!
theptip 10y ago
This is pretty interesting. DeepMind’s last triumph (beating the best human Go players with AlphaGo) is impressive, but Go is a great fit for neural networks as they stand today; it’s stateless, so you can fully evaluate your position based on the state of the board at a given turn. That’s not a good fit for most real-world problems, where you have to remember something that happened in the past. E.g. the ‘fog of war’ in a strategy game. This is a big open problem in AI research right now. I saw a post around the time of AlphaGo challenging DeepMind to tackle StarCraft next, so it is very cool that they have gone in this direction. When Google’s AI can beat a human at StarCraft, it’s time to be very afraid.
- andrew3726 10y ago
  > When Google’s AI can beat a human at StarCraft, it’s time to be very afraid. More like a time to be very excited! But I totally agree, long-term dependencies are an difficult problem. The DNC[0] by DeepMind is in my opinion a very important step in the correct direction to tackle this. [0] https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/
  - chmod775 10y ago
    Honestly you can solve fog of war using regular computing (TM) to remember the last known state of a particular area, then feed that to the network along with information that A. it's an area as it was seen some time ago B. how much time elapsed since it was last uncovered. I can't imagine that this is where DNCs come in unless you want to needlessly complicate the matter (and create a network that is much more expensive to train).
    - thomasahle 10y ago
      Yes, it's interesting what help will be given to the network. From the article it seems the answer is: very little. It is allowed to use only the pixels on the screen. (During play, not training apparently.) It will even have to spend actions on moving the screen/window to inspect things. Very cool stuff :)
andr 10y ago
I wonder if AI will take over eSports (Twitch, competitions, etc.), as well. It could be a variant of the Turing test.
xg15 10y ago
I applaud the idea, but I'm worried about how open the results of the research will be in the end. I think the worst possible outcome for society would be if we ended up with capable AI but with the algorithms only accessible for a handful "AI-as-a-service" companies. The second concern is understandability of the algorithms: from what I've read, it's already hard to deduce "why" an ANN in a traditional problem behaved like it did. An AI that is capable of coping with dynamic, unpredictable situations like an SC game (with only pixels as input) is impressive but it seems less useful to me if it's not understood how that is done.
- mememachine 10y ago
  Well unfortunately im fairly certain that outcome is the source of funding for these research projects, and i dont think socialized funding could hold a match to it nor potentially even provide as much value
- halflings 10y ago
  DeepMind always published papers about their research (the AlphaGo paper goes into a lot of details on how everything works). Not sure why you're doubting the openness of this initiative when they are pushing an open API for everybody to do research on, not just DeepMind.
chrishare 10y ago
Very interesting stuff. Allowing researchers to build AIs that operate on either game state or visual data is a great step, IMO. Being able to limit actions-per-minute is also very wise. The excellent BWAPI library for Starcraft Broodwar that is referenced (https://github.com/bwapi/bwapi https://github.com/bwapi/bwapi) provides game state - and was presumably used by Facebook to do their research earlier this year (http://arxiv.org/abs/1609.02993 http://arxiv.org/abs/1609.02993). For mine, the significant new challenges here not present in Go are the partial observability of the game and the limited time window in which decisions need to be made. Even at 24 frames per second, the AI would only have 40 milliseconds to decide what to do in response to that frame. This is more relevant to online learning architectures. The open questions here are how freely this will be available - and in what form. Will I need to buy a special version of the game? Clearly there will be some protection or AI detection - to ensure that competitive human play is not ruined either by human-level bots, if they can truly be developed, or by sub-par bots. Starcraft 2 (presumably the latest expansion, Legacy of the Void, will be used here) does not run on Linux, whereas most GPU-based deep learning toolkits are, so having a bridge between the game and AI server may be necessary for some researchers. Besides being great for AI researchers, this is probably good for Blizzard too, since it will bring more interest to the Starcraft series. 2017 can't come soon enough.
- samstave 10y ago
  >2017 can't come soon enough. Let's just get through Tuesday and see what happens! ;-)
- endgame 10y ago
  I wonder if this would have ever been built or released if bwapi wasn't a thing.
  - ygra 10y ago
    BWAPI asked in the past, I think, and got pointed to the game's EULA, which forbids it. It may be a different thing if Google asks. Especially with a somewhat more remarkable result (AlphaGo) than "just" a bunch of student AI experiments.
Havoc 10y ago
This is awesome. I know there is a happy AI community on SC1 front so glad to see Blizzard anything on SC2 front.
cryptrash 10y ago
I'm pretty excited about this. I think some kids out there will really enjoy an environment like this to mess with, and maybe learn a thing or two about machine learning along the way. Starcraft is a really fun game, and I think it's enough to engage kids a little more than something like Minecraft where there's plenty of room for some cool ML hacking, but not enough stimulation from it. Instead of just seeing blocks here or there or whatever, starcraft has hard goals that will force them to use critical thinking skills, their knowledge of the game, their own personal strategic insights, and the ML skills they accrue. So exciting! Love the feature layer idea also, well done!
- pdimitar 10y ago
  I agree. SC2 is a very strategic game and has tons of potential for scouting, fake attacks, distractions, diversions and all sorts of deceitful play. And that's coming after having a good micro-management of units, using perfect timing for expanding your economy, upgrading your units, proxy offense, and a ton of other mechanics. DeepMind really chose well. SC2 has to be the most demanding game nowadays in terms of strategy and execution.
plg 10y ago
I understand the challenge, and the importance of the proof-of-principle. But again? Having done Atari games, then Go, at what point exactly does google deepmind start attacking some real world problems of consequence? Or maybe the answer is never, other companies are supposed to do the hard work? We only play games?
- Ph0X 10y ago
  Machine learning is currently being used all over the place, in almost every single of Google's products. Projects like these may have a "fun" goal, but while reaching towards that objective, a lot of new techniques and theory is developed, which then gets used in real products. Systems like this are much easier to use as a playground and test in than the real world. But as they mention, the messiness and stateful nature of games like SC2 will bring a lot of innovation which will be very relevant to the real world.
- wsetchell 10y ago
  They are publicly working on medicine and energy problems. Games just happen to be a convenient place to do research. https://deepmind.com/blog/putting-patients-heart-deepmind-health/ https://deepmind.com/blog/putting-patients-heart-deepmind-he... https://deepmind.com/blog/applying-machine-learning-radiotherapy-planning-head-neck-cancer/ https://deepmind.com/blog/applying-machine-learning-radiothe... https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/ https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...
- empath75 10y ago
  Solving toy problems in a limited domain is pretty important before you can tackle real world problems.
- Houshalter 10y ago
  What real world problems do you have in mind? Games are convenient because they can be easily simulated, and you can compare the performance to expert humans. The reinforcement learning they work on is extremely general and can be applied to many other types of problems. Particularly robot control, which is sort of the same sort of task as a video game.
deleted 10y ago
[deleted]
jasikpark 10y ago
Comparing AI and humans in games is not useful unless all limitations are controlled for both parties. The artificial intelligence only gets the video output of the game and output to simple controls with a human reflex - like lag on how long it takes for the controls to take effect. It just comes down to the scientific method.
rezashirazian 10y ago
If DeepMind is planning on building an AI that can beat the best human SCII player, they have their work cut out for them. I'm not sure how familiar people are with StarCraft II, but at the highest levels of the game, where player have mastered the mechanics, it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style.
- cjbprime 10y ago
  I agree with you -- this requires first creating the equivalent of a pro poker player, and then adding SC2 mechanics and strategy on top.
- fantasticsid 10y ago
  I believe the same thing could be said about Go :)
  - jdoliner 10y ago
    No the same thing very much could not be said about Go. Go is a perfect information game so by definition it can't involve deception. It's a completely different problem for AIs.
- seanwilson 10y ago
  > I'm not sure how familiar people are with StarCraft II, but at the highest levels of the game, where player have mastered the mechanics, it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style. I wouldn't be surprised if an AI comes along that shows this human perception of what it takes to win isn't required at all. I'm sure similar things were said about Go and Chess.
  - andrepd 10y ago
    Both Go and Chess are perfect information games with a (mind bogglingly huge but) vanishingly small number of states/permutations compared to Starcraft. It's an entirely different beast, altogether.
wodenokoto 10y ago
This paper is a few years old, but it gives a good overview of the problems faced when building an AI for starcraft and the methods used. Ontanón, Santiago, et al. "A survey of real-time strategy game ai research and competition in starcraft." IEEE Transactions on Computational Intelligence and AI in games 5.4 (2013): 293-311. http://webdocs.cs.ualberta.ca/~cdavid/pdf/starcraft_survey.pdf http://webdocs.cs.ualberta.ca/~cdavid/pdf/starcraft_survey.p...
seanwilson 10y ago
Can a simulation of a complete Starcraft game be done quickly and assuming it can't doesn't that present problems to training an AI? For example, I'm guessing complete games of Go are order of magnitudes faster to simulate which makes it more practical to do things like getting AlphaGo to play against itself to train.
- dsjoerg 10y ago
  No, the fastest they can do right now is 8x. And yes, that will become a substantial bottleneck for AI training. I imagine they're going to have to have a large sim-farm for just running the games at 8x for training.
simonebrunozzi 10y ago
This is a wasted opportunity for other strategy games to become the most played game on the planet. I sometimes play strategy games and I always find the AI disappointing. Any game with a great AI would be my favorite for years. Heck, I would even pay a few dozen cents/hour to be able to compete against a proper AI.
kleigenfreude 10y ago
First we teach it Atari games, and now strategy and war? Why not give it lots of data to solve real problems? Training it on useless games will have no benefit.
deepnotderp 10y ago
By the ways, if anyone's interested, there was a deluge of deep learning papers today and one of them basically used deep learning to make deep learning models and it did better than humans.
- beaner 10y ago
  Source?
  - deepnotderp 10y ago
    "Neural architecture search with reinforcement learning"
- deleted 10y ago
  [deleted]
- daveguy 10y ago
  Seriously... Source? I have looked at new papers indexed by Google scholar and I don't see anything that boasts this. It would be the feature of the abstract and would probably be a Nature/Science paper. So... Source??
ambar123 10y ago
I will consider AI to human level.. When it can fully play gta san-andreas
deleted 10y ago
[deleted]
prawn 10y ago
Makes me wonder if any game companies have seeded empty servers with bots, acting as humans, to give their games a sense of popularity.
- pdimitar 10y ago
  I don't see why not, unless the game eventually becomess a boss-infested cesspool where players can't have any fun of course.
oblio 10y ago
I wonder when we'll be at a point that a small, portable AI (such as the one included with games) is actually competitive with decent humans.

917 points by madspindel 10y ago

Looks like they are going to limit the APM of the AI. I wonder how they are going to decide the limit? I've never played StarCraft, but from what I understand very high APM is needed to play the game at the highest levels.

nip 10y ago
Probably an average of the best players.
tylerpachal 10y ago
I've followed the professional StarCraft 2 scene for a long time now. There were a few players who were famous for having high APMs, here is an over-the-sholder view of one of them [1]. I don't have a reference but believe he used to play in the ~500APM range. That would probably be the upper limit of a human's ability. [1] https://youtu.be/WHcl6Cs1FAM?t=7m57s https://youtu.be/WHcl6Cs1FAM?t=7m57s
- hughes 10y ago
  I wouldn't be surprised if a 500APM player is making closer to 100 meaningful actions per minute. It looks like most of the actions are spam that may register on the APM meter but not affect the outcome of the game. An AI limited to even 200 decision-driven actions per minute would probably have a significant advantage over a "500APM" player.
exelius 10y ago
I imagine they'll have separate AI brackets, and they could potentially simulate at multiple APM levels.
areign 10y ago
probably limit it to human levels. 400 to 600 apm is around the limit for brief periods of time. It makes for an interesting optimization problem with how to use them though.
Aweorih 10y ago
High apm is required, yes. But most people push it by pushing it through spamming 1-2-3 (groups of your army). There was a pro player who didnt do it and he was around +- 70 APM.
- Cafey 10y ago
  It would be interesting to see what percentage of a pro player's APM is actually useful. Like you said, most of the time they are spamming buttons to keep their momentum up.
  - placeybordeaux 10y ago
    There is a replay analyzer that attempts to figure this out, generally the effective APM will go up during a fight, but the players will keep the APM fairly high even when the eAPM is not high just because it is easier to go from many useless actions to many useful actions than few useful actions to many useful actions.
  - Aweorih 10y ago
    I think they are doing it to stay "warm". E.g. if something happens that the fingers can react quick and you don't need a "warmup" time.
fazzone 10y ago
There's probably no reason to restrict the computer the below human pro players (200-300 APM). My guess is the engine just wasn't designed with superhuman APM in mind and could be exploited. If that's the case then the AI would in some sense not really be playing the same game as the humans.
- stale2002 10y ago
  An apm limit is needed because impossibly high APM results in stupid strategies that the game wasn't designed for. The point is to build ai that compete with humans using human strategies. Not to cheese with borked game mechanic exploits.
  - TulliusCicero 10y ago
    In practice right now the AI still loses even with unlimited APM. It's just too dumb to really take advantage of its execution speed.
    - nogbit 10y ago
      Buts it's intentionally dumbed down to keep the game fun, loosing all the time would get frustrating real quick. Theres no ROI for AI devs to kick the humans ass (except for maybe now with our AI/Cloud/VR/ craze).
      - stale2002 10y ago
        No it is not. People were making custom AIs for starcraft 2 since it was in beta. They can't beat any human worth their salt.
      - TulliusCicero 10y ago
        It is not intentionally dumbed down. Current AIs are just too stupid at strategy and tactics to beat competent human players.
- SerLava 10y ago
DontGiveTwoFlux 10y ago
There was a Starcraft brood war AI competition that did not have a limit on APM. The top contestants were employing strategies that required APM an order of magnitude higher than what players could do. For example, in brood war, SCVs can repair any Terran building or vehicle but it is not worth the effort to repair goliaths or tanks for a player. However, the AI could easily manage repairing it's tanks which makes the Terran army much more cost effective. In SCII there is auto repair and other improved UX that probably limit the ability of novel strategies like repairing vehicles.
- placeybordeaux 10y ago
  This was an example of extreme micro that a very simple AI can do that changes the game entirely: https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs
  - wlesieutre 10y ago
    How does it seem to know which zergling is about to get hit?
    - acomar 10y ago
      If you slow it down, it would be apparent to you too. High-level human players do this kind of thing to, just not this fast. But effectively, it's watching where the attacking unit is facing and it knows when the next attack is coming, so it knows what the viable targets are and can react when it becomes obvious. (it's also possible the system is receiving information directly from the game engine, but it's definitely possible to figure this out from nothing but the information on screen)
    - caylus 10y ago
      Because the tanks are using the default unit AI, which AFAIK amounts to "when your weapon is ready to fire, shoot the closest enemy in range". The zergling AI knows the attack range and rate of each tank so can ensure that the closest zergling is always on its own when a tank is ready to fire. What would be really interesting is putting a similar smart AI on the tanks, which could make decisions like shooting at the highest concentration of zerglings instead of the closest one, or holding its fire if one lone zergling is in range, but a clump of them is about to enter range. Human players already do both optimizations with widow mines, which are a unit that takes several seconds to prepare its shot so it's feasible to retarget. It's not feasible for humans to do this with tanks though as they fire immediately and automatically. You could even see counterevolution as two AI algorithms become better at predicting what the other will do.
nharada 10y ago
Back when I played SC2, you really didn't need a high APM to succeed, unlike SC1. SC2 offered some helpful mechanics that SC1 did not, such as the ability to select infinite units at once, or give commands to multiple structures at one. I think Sheth, a high level Zerg player, was famous for having under 100 APM.
- thaumasiotes 10y ago
  I'll consider APM "not high" when it gets down to 12. Two actions per second is comfortably "high".
  - uw_rob 10y ago
    That may be an ok definition for you, but it most certainly isn't for SC2. I wasn't anywhere near the pro level and I would peak at 300 apm and would spend most of the game around 200. 12 APM is borderline not playing.
    - thaumasiotes 10y ago
      Consider it a statement about what I'm looking for in a game. I'm not really interested in competing over who can work their mouse button harder, or in driving myself to excel in giving myself repetitive motion injuries. Those are things that take away from starcraft; they don't enhance it.
      - halflings 10y ago
        I would strongly encourage you to go against your preconceived idea of what StarCraft is (a game where people compete on who's the fastest to bash their keys) and see a professional game and all the strategy it entails, such as in the semi-finals and grand final that will be on today. Pro players do have valid reasons for having such a high APM (even if a portion of it, maybe 30%, is just "spamming" to keep theirselves active and ready), but you can be an excellent player with rather low APM (~120 maybe?). Why would you need to do 2 action per second? A lot of it is simply building units periodically (e.g pressing 1 then "S" every 10 seconds) but then APM really gets high when there is a battle and you need to give actions to multiple sets of units to get a good engagement.
JabavuAdams 10y ago
The best SCBW bots still aren't very good, so I don't see unrestricted APM as an unfair advantage.
lanius 10y ago
Unfortunate. I personally would love play against an AI with strong game sense as well as insane micro a la https://www.youtube.com/watch?v=IKVFZ28ybQs https://www.youtube.com/watch?v=IKVFZ28ybQs
Retr0spectrum 10y ago
For anyone else wondering, APM = Actions Per Minute.

ChrisAntaki 10y ago

> We’re particularly pleased that the environment we’ve worked with Blizzard to construct will be open and available to all researchers next year. This is awesome. I've only ever reached the Platinum league in Starcraft II (1v1), but I'd almost feel more driven to create bots to (hopefully) surpass that skill level, than actually playing the game.

luka-birsa 10y ago

Anybody else sees a problem with training the AI to move troops around a battlefield, with the purpose to exterminate the opponent?

harigov 10y ago
It's going to happen anyway. I think we are better off doing such things in the open and having some visibility into it rather than be surprised when something happens.
- treehau5 10y ago
  I'm so sick of this argument. How about we have a spine for once and say no?
  - danielpatrick 10y ago
    The problem is we is not everyone... it's only those who agree with you. Everyone else will just keep on pushing ahead, hence the less-bad option of not relegating that advancement to dark corners.
  - webkike 10y ago
    All 7 billion of us? Good luck with that.
    - Mtinie 10y ago
      I agree with you, it's a virtually impossible task. The commenter I was responding to has strong ideas about it however, so I wanted to provide the opportunity for an explanation.
  - Mtinie 10y ago
    Ok, for conversation sake we all come together and say, "No, AI cannot be used for offensive or defensive military actions". How do you propose that ban be enforced?
  - 10y ago

KZeillmann 10y ago

This is so exciting. I've always wanted to program bots to play online games -- mainly for learning purposes. (Can I make a bot that plays better than me?) But I've never done it because of the risk of bans. I'm glad that Blizzard has opened it up for people to experiment with this. I wonder how it will interact with any sort of anti-cheat systems in place, etc.

tylerpachal 10y ago
This has been around for awhile [1]. They give you an API to use and then you upload your JAR. They have a Twitch channel [2] that is constantly streaming AIs playing against each other in their tournament. [1] http://sscaitournament.com http://sscaitournament.com [2] https://www.twitch.tv/certicky https://www.twitch.tv/certicky
- pdimitar 10y ago
  This has to be the most interesting link in this whole thread. Thank you so much!
formula1 10y ago
Bans should never keep you back. A fake email is pennies to how much is gained from learning or building something incredible
- garrettgrimsley 10y ago
  A fake email won't help when you are banned by your unique ID that you paid, in the case of SC2: Legacy of the Void, at least $40 for.
  - formula1 10y ago
    Is $40 really not worth a year or more of experimentation? You can also do a lot of learning through replays or vs AI. I personally am willing to pay $40 for an environment to test my ai on for a couple months.

formula1 10y ago

I suspect this will eventually lead to AI as a service for games. Rather than build a terrible AI that delays a game by months, approaching a company that can build a decent AI initially which gets better overtime would probably be ideal and create better experiences. Im curious if a startup can be built from this.

doikor 10y ago
Is it harder to make an AI that is mediocre at playing the game vs one that is good? That is the main issue with making good AIs for games now. We want AIs (if you can call them that) that play kinda like humans but worse then the player as the player wants to win. Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
- Voloskaya 10y ago
  Having checkpoints along the way as you train your model could be a good way to allow fine-grained level selection. I guess :)
  - LesZedCB 10y ago
    That would be funny. The difficulty slider changes to "How many hours of playtime" instead of easy, medium, hard. Actually it could even just scale it automatically based on a metric like steam hours played. that would be awesome.
  - aero142 10y ago
    I don't think so. Many of the fun things AIs do in games is attack at different stages of the game. They send a small force at the beginning, and then a little larger force a few minutes later, and then a big attack at the end. No AI trying to win would end up at this strategy. You would have to define the fun goals and have the AI train on that for it's success criteria.

Savageman 10y ago

That's so cool! I wish they could start doing AI for team based competitive games like League of Legends where meta-play and team decision making is important. Is that too complicated to tackle yet?

rcthompson 10y ago
Are you imagining a single AI playing all 5 heroes on a team, or 5 independent AIs each playing a single hero with limited communication between them, or a single AI controlling a single hero on a team with 4 humans? I would say the first scenario (1 AI controlling entire team) is roughly equivalent to the the RTS case. It's just fewer units with more abilities, and character progression instead of base building. The other two cases are probably a lot more complicated.
- Savageman 10y ago
  Either one could work. If the single AI that controls 5 heroes is easier to program, let's imagine that. I have the feeling that it's more complicated than a 1vs1 RTS game, but I don't know why...
  - CydeWeys 10y ago
    Controlling five units is a lot easier than playing an RTS. In an RTS you don't just control five units, you control potentially hundreds of units, and you have base-building and an economy to manage.
- manux 10y ago
  Honestly all these scenarios are interesting research avenues. Inter-agent communication, especially in a machine learning context, is an unsolved problem, exploring it in such a scenario would be nice. Agent-human communication is probably as hard, and also largely unsolved, although there is a push right now for dialog systems to achieve human level performance.
- SketchySeaBeast 10y ago
  I can't wait to see the first team of 5 separate AI learn to insult each other. It'll be a watershed moment in history when the first vote to kick another AI occurs.

randyrand 10y ago

I hope the AI will have a handicap on the speed of mouse movement. IRL you can't just teleport the mouse around the screen.

simpfai 10y ago

Does anyone know of any resources for someone looking to learn more about Game AI for real time strategy games?

luminiferous 10y ago
The Berkeley Overmind was designed for a 2010 Brood War competition. The website (http://overmind.cs.berkeley.edu/ http://overmind.cs.berkeley.edu/), although no longer updated, still contains many videos and links to articles that describe the internals of the AI and the design process. The Ars Technica article in particular goes quite in depth.
PetahNZ 10y ago
http://aigamedev.com/ http://aigamedev.com/

andrew3726 10y ago

This is really good news! Lets hope DeepMind can improve even further on their Differentiable neural computers (DNC) which seems like an requirement for this kind of AI to work (exploiting long-term dependencies). I also hope that other research/industry teams will join on the competition to create competing AIs. Very exciting!

lanius 10y ago

I can't wait to see how far DeepMind can go in this area. I was initially skeptical that AlphaGo could defeat top human players, but then it happened. Who knows, perhaps one day AI can compete against progamers in GSL!

placeybordeaux 10y ago
Competitive AI in broodwar already can destroy the best humans. AI can use strategies and tactics that human can't effectively do. Most notable is perfect splitting which changes the game quite a bit.
- erik 10y ago
  Do you have a reference on this? Last I looked into it broodwar AI bots were easily beaten by competent humans.
- lanius 10y ago
  >Competitive AI in broodwar already can destroy the best humans Who do you mean by "the best humans"? I've never heard of top-tier progamers such as Flash, Jaedong, Bisu, etc. losing to AI. In fact, I doubt any B-teamer or above would lose to an AI in a Bo3.

felix_thursday 10y ago

OpenAI vs DeepMind?

flamedoge 10y ago

This is dangerous. Overmind will come to life.

luminiferous 10y ago
The UC Berkeley AI for Brood War (designed for the 2010 AIIDE StarCraft competition) was literally named the Berkeley Overmind. http://overmind.cs.berkeley.edu/ http://overmind.cs.berkeley.edu/
dmichulke 10y ago
And this time it can't be drugged.

brylie 10y ago

It would be cool to see a project like this for an open source game, such as OpenTransportTycoonDeluxe (http://www.openttd.org/ http://www.openttd.org/). The AI developed by interacting with the OpenTTD economy might even prove useful for urban planning of real geographic regions.

deleted 10y ago

[deleted]

komaromy 10y ago

My mostly-uneducated o/u on the time to beat the best human players is 8 years.

yaboyhud 10y ago

plug: https://medium.com/new-game/comp-stomp-b8fe687fbeb6 https://medium.com/new-game/comp-stomp-b8fe687fbeb6

partycoder 10y ago

That's great. AIs on SC1 relied on many hacks. Initially I thought that DeepMind was going to create a bot for the original SC. I hope some of the advances in SC2 AI can be integrated into the in-game AI. e.g: a trained neural network that plays better than the "hard" AI, but can run on a consumer box and not on a massive cluster.

rkuykendall-com 10y ago

This would be so much more fun with a turn-based game where speed isn't a variable, like Civilization. I'd love to play against several AIs that were better than me because of code, not because they get a bunch of extra in-game bonuses. With a nice AI API, you could have online competitions where AIs battled every month.

noiv 10y ago

Hopefully there's a JS/Spidermonkey interface. I'd be happy to port Hannibal from 0AD.

tgb 10y ago
What does Hannibal do that makes it portable? My impression is most game AI is essentially completely game specific, outside of, eg, pathfinding.

bluebeard 10y ago

This will be good for games moving forward due to the meta changing for players as the AI adapts to their tactics and vice versa. Lessons learned from this can then be applied to other areas. And as an added bonus it creates more interest in AI research.

tdaltonc 10y ago

I'd love to see a e-sport league where the teams are AI human hybrids (Centaur teams). We know that AI human hybrid teams are great at chess [1], and I'd love to see rts games played by 'Centaur' teams. In the same way the innovations made in F1 often trickle down to consumer cars, can you imagine the advances that could be made in human-machine interactions in the crucible of a real-time Centaur gaming league? [1] http://bloomreach.com/2014/12/centaur-chess-brings-best-humans-machines/ http://bloomreach.com/2014/12/centaur-chess-brings-best-huma...

bradleyjg 10y ago
If they allowed you to script micro unit behavior in some way that'd be first step in a hybrid game.
- tdaltonc 10y ago
  Either that or a custom alerts system.
- hooloovoo_zoo 10y ago
  It would also be cool if it was an upgrade you unlock. So, instead of just stat upgrades, you could also purchase AI upgrades.
  - make3 10y ago
    I'm sure in the next decade we will see the start of modern ai in video games & all ai applied to all kinds of gameplay mechanics, like this one
  - bradleyjg 10y ago
    I'd rather that the coding was something that was left to the player or to the third party market. Right now programming games are kind of boring, more like a way of prettying up homework than anything else. A really fun game where programming could give you a competitive edge would be something to see. The closest thing I can think of is add-ons for MMOs, but there the game developers specifically try to prevent add-ons that are too performance enhancing instead of making the coding an integral part of the game.

simopaa 10y ago

I would love to keep a 24/7 stream open where the AI is continuously playing against the game's own bots in order to improve its playstyle.

wesleytodd 10y ago
I would probably watch that
Houshalter 10y ago
Typically they train the AI against itself. That gives it a continuous skill gradient. Otherwise it would lose all of the games without learning anything. Since there are no successes to learn from. Or it would win all the games after it figures out how to exploit the built-in bot. With strategies that probably won't apply to humans.

tylerpachal 10y ago

For anyone looking for more information about Starcraft 2, the world championship is on this weekend and the broadcast for today has just started (16:30EST) https://www.twitch.tv/starcraft https://www.twitch.tv/starcraft

l34ch 10y ago

Pretty soon AI for games will be so good that we won't have to play online multiplayer for a really good authentic experience. Once we can't tell the difference between the AI and an actual human playing.

fitzwatermellow 10y ago

Training data? After all, AlphaGo trained on a database of over 30M expert human moves. I suspect one championship round from Team EnVy is worth billions of iterations of random exploration ;) Kudos to both Blizzard and DeepMind. Anticipating a lot of fun with this. StarCraft 2 could indeed become the AI pedagogy standard.

pizza 10y ago

Nice! I cracked a joke about how SC2 was nothing more than an AI testbed just last week, lol. Very glad to see it's becoming a real thing!

theptip 10y ago

This is pretty interesting. DeepMind’s last triumph (beating the best human Go players with AlphaGo) is impressive, but Go is a great fit for neural networks as they stand today; it’s stateless, so you can fully evaluate your position based on the state of the board at a given turn. That’s not a good fit for most real-world problems, where you have to remember something that happened in the past. E.g. the ‘fog of war’ in a strategy game. This is a big open problem in AI research right now. I saw a post around the time of AlphaGo challenging DeepMind to tackle StarCraft next, so it is very cool that they have gone in this direction. When Google’s AI can beat a human at StarCraft, it’s time to be very afraid.

andrew3726 10y ago
> When Google’s AI can beat a human at StarCraft, it’s time to be very afraid. More like a time to be very excited! But I totally agree, long-term dependencies are an difficult problem. The DNC[0] by DeepMind is in my opinion a very important step in the correct direction to tackle this. [0] https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/
- chmod775 10y ago
  Honestly you can solve fog of war using regular computing (TM) to remember the last known state of a particular area, then feed that to the network along with information that A. it's an area as it was seen some time ago B. how much time elapsed since it was last uncovered. I can't imagine that this is where DNCs come in unless you want to needlessly complicate the matter (and create a network that is much more expensive to train).
  - thomasahle 10y ago
    Yes, it's interesting what help will be given to the network. From the article it seems the answer is: very little. It is allowed to use only the pixels on the screen. (During play, not training apparently.) It will even have to spend actions on moving the screen/window to inspect things. Very cool stuff :)

andr 10y ago

I wonder if AI will take over eSports (Twitch, competitions, etc.), as well. It could be a variant of the Turing test.

xg15 10y ago

I applaud the idea, but I'm worried about how open the results of the research will be in the end. I think the worst possible outcome for society would be if we ended up with capable AI but with the algorithms only accessible for a handful "AI-as-a-service" companies. The second concern is understandability of the algorithms: from what I've read, it's already hard to deduce "why" an ANN in a traditional problem behaved like it did. An AI that is capable of coping with dynamic, unpredictable situations like an SC game (with only pixels as input) is impressive but it seems less useful to me if it's not understood how that is done.

mememachine 10y ago
Well unfortunately im fairly certain that outcome is the source of funding for these research projects, and i dont think socialized funding could hold a match to it nor potentially even provide as much value
halflings 10y ago
DeepMind always published papers about their research (the AlphaGo paper goes into a lot of details on how everything works). Not sure why you're doubting the openness of this initiative when they are pushing an open API for everybody to do research on, not just DeepMind.

chrishare 10y ago

Very interesting stuff. Allowing researchers to build AIs that operate on either game state or visual data is a great step, IMO. Being able to limit actions-per-minute is also very wise. The excellent BWAPI library for Starcraft Broodwar that is referenced (https://github.com/bwapi/bwapi https://github.com/bwapi/bwapi) provides game state - and was presumably used by Facebook to do their research earlier this year (http://arxiv.org/abs/1609.02993 http://arxiv.org/abs/1609.02993). For mine, the significant new challenges here not present in Go are the partial observability of the game and the limited time window in which decisions need to be made. Even at 24 frames per second, the AI would only have 40 milliseconds to decide what to do in response to that frame. This is more relevant to online learning architectures. The open questions here are how freely this will be available - and in what form. Will I need to buy a special version of the game? Clearly there will be some protection or AI detection - to ensure that competitive human play is not ruined either by human-level bots, if they can truly be developed, or by sub-par bots. Starcraft 2 (presumably the latest expansion, Legacy of the Void, will be used here) does not run on Linux, whereas most GPU-based deep learning toolkits are, so having a bridge between the game and AI server may be necessary for some researchers. Besides being great for AI researchers, this is probably good for Blizzard too, since it will bring more interest to the Starcraft series. 2017 can't come soon enough.

samstave 10y ago
>2017 can't come soon enough. Let's just get through Tuesday and see what happens! ;-)
endgame 10y ago
I wonder if this would have ever been built or released if bwapi wasn't a thing.
- ygra 10y ago
  BWAPI asked in the past, I think, and got pointed to the game's EULA, which forbids it. It may be a different thing if Google asks. Especially with a somewhat more remarkable result (AlphaGo) than "just" a bunch of student AI experiments.

Havoc 10y ago

This is awesome. I know there is a happy AI community on SC1 front so glad to see Blizzard anything on SC2 front.

cryptrash 10y ago

I'm pretty excited about this. I think some kids out there will really enjoy an environment like this to mess with, and maybe learn a thing or two about machine learning along the way. Starcraft is a really fun game, and I think it's enough to engage kids a little more than something like Minecraft where there's plenty of room for some cool ML hacking, but not enough stimulation from it. Instead of just seeing blocks here or there or whatever, starcraft has hard goals that will force them to use critical thinking skills, their knowledge of the game, their own personal strategic insights, and the ML skills they accrue. So exciting! Love the feature layer idea also, well done!

pdimitar 10y ago
I agree. SC2 is a very strategic game and has tons of potential for scouting, fake attacks, distractions, diversions and all sorts of deceitful play. And that's coming after having a good micro-management of units, using perfect timing for expanding your economy, upgrading your units, proxy offense, and a ton of other mechanics. DeepMind really chose well. SC2 has to be the most demanding game nowadays in terms of strategy and execution.

plg 10y ago

I understand the challenge, and the importance of the proof-of-principle. But again? Having done Atari games, then Go, at what point exactly does google deepmind start attacking some real world problems of consequence? Or maybe the answer is never, other companies are supposed to do the hard work? We only play games?

Ph0X 10y ago
Machine learning is currently being used all over the place, in almost every single of Google's products. Projects like these may have a "fun" goal, but while reaching towards that objective, a lot of new techniques and theory is developed, which then gets used in real products. Systems like this are much easier to use as a playground and test in than the real world. But as they mention, the messiness and stateful nature of games like SC2 will bring a lot of innovation which will be very relevant to the real world.
wsetchell 10y ago
They are publicly working on medicine and energy problems. Games just happen to be a convenient place to do research. https://deepmind.com/blog/putting-patients-heart-deepmind-health/ https://deepmind.com/blog/putting-patients-heart-deepmind-he... https://deepmind.com/blog/applying-machine-learning-radiotherapy-planning-head-neck-cancer/ https://deepmind.com/blog/applying-machine-learning-radiothe... https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/ https://deepmind.com/blog/deepmind-ai-reduces-google-data-ce...
empath75 10y ago
Solving toy problems in a limited domain is pretty important before you can tackle real world problems.
Houshalter 10y ago
What real world problems do you have in mind? Games are convenient because they can be easily simulated, and you can compare the performance to expert humans. The reinforcement learning they work on is extremely general and can be applied to many other types of problems. Particularly robot control, which is sort of the same sort of task as a video game.

deleted 10y ago

[deleted]

jasikpark 10y ago

Comparing AI and humans in games is not useful unless all limitations are controlled for both parties. The artificial intelligence only gets the video output of the game and output to simple controls with a human reflex - like lag on how long it takes for the controls to take effect. It just comes down to the scientific method.

rezashirazian 10y ago

If DeepMind is planning on building an AI that can beat the best human SCII player, they have their work cut out for them. I'm not sure how familiar people are with StarCraft II, but at the highest levels of the game, where player have mastered the mechanics, it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style.

cjbprime 10y ago
I agree with you -- this requires first creating the equivalent of a pro poker player, and then adding SC2 mechanics and strategy on top.
fantasticsid 10y ago
I believe the same thing could be said about Go :)
- jdoliner 10y ago
  No the same thing very much could not be said about Go. Go is a perfect information game so by definition it can't involve deception. It's a completely different problem for AIs.
seanwilson 10y ago
> I'm not sure how familiar people are with StarCraft II, but at the highest levels of the game, where player have mastered the mechanics, it's a mind game fueled by deceit, sophisticated and malleable planning, detecting subtle patterns (having a good gut feeling on what's going on) and on the pro scene knowledge of your opponent's style. I wouldn't be surprised if an AI comes along that shows this human perception of what it takes to win isn't required at all. I'm sure similar things were said about Go and Chess.
- andrepd 10y ago
  Both Go and Chess are perfect information games with a (mind bogglingly huge but) vanishingly small number of states/permutations compared to Starcraft. It's an entirely different beast, altogether.

wodenokoto 10y ago

This paper is a few years old, but it gives a good overview of the problems faced when building an AI for starcraft and the methods used. Ontanón, Santiago, et al. "A survey of real-time strategy game ai research and competition in starcraft." IEEE Transactions on Computational Intelligence and AI in games 5.4 (2013): 293-311. http://webdocs.cs.ualberta.ca/~cdavid/pdf/starcraft_survey.pdf http://webdocs.cs.ualberta.ca/~cdavid/pdf/starcraft_survey.p...

seanwilson 10y ago

Can a simulation of a complete Starcraft game be done quickly and assuming it can't doesn't that present problems to training an AI? For example, I'm guessing complete games of Go are order of magnitudes faster to simulate which makes it more practical to do things like getting AlphaGo to play against itself to train.

dsjoerg 10y ago
No, the fastest they can do right now is 8x. And yes, that will become a substantial bottleneck for AI training. I imagine they're going to have to have a large sim-farm for just running the games at 8x for training.

simonebrunozzi 10y ago

This is a wasted opportunity for other strategy games to become the most played game on the planet. I sometimes play strategy games and I always find the AI disappointing. Any game with a great AI would be my favorite for years. Heck, I would even pay a few dozen cents/hour to be able to compete against a proper AI.

kleigenfreude 10y ago

First we teach it Atari games, and now strategy and war? Why not give it lots of data to solve real problems? Training it on useless games will have no benefit.

deepnotderp 10y ago

By the ways, if anyone's interested, there was a deluge of deep learning papers today and one of them basically used deep learning to make deep learning models and it did better than humans.

beaner 10y ago
Source?
- deepnotderp 10y ago
  "Neural architecture search with reinforcement learning"
deleted 10y ago
[deleted]
daveguy 10y ago
Seriously... Source? I have looked at new papers indexed by Google scholar and I don't see anything that boasts this. It would be the feature of the abstract and would probably be a Nature/Science paper. So... Source??

ambar123 10y ago

I will consider AI to human level.. When it can fully play gta san-andreas

deleted 10y ago

[deleted]

prawn 10y ago

Makes me wonder if any game companies have seeded empty servers with bots, acting as humans, to give their games a sense of popularity.

pdimitar 10y ago
I don't see why not, unless the game eventually becomess a boss-infested cesspool where players can't have any fun of course.

oblio 10y ago

I wonder when we'll be at a point that a small, portable AI (such as the one included with games) is actually competitive with decent humans.