8 ms·

DeepMind StarCraft II Demonstration [video]

163 points by Permit 8y ago

Permit 8y ago
Here is another link (YouTube) that allows you to go back in time if you've missed anything: https://www.youtube.com/watch?v=cUTMhmVh1qs https://www.youtube.com/watch?v=cUTMhmVh1qs
codeflo 8y ago
I'm starting to get annoyed with these DeepMind publicity stunts. They don't release any code, and don't let anyone verify their results. Their chess AI beating Stockfish involved some at least a bit questionable setup. And here? I'm a big fan of TLO, but he's currently not even in the top 50 even with his main race. With Protoss, he's clearly making amateur-level mistakes. Why choose this setting to show off your supposedly superhuman AI? (Edit: This comment was made during the stream and it just looks like my point will be addressed before the stream's even finished, they'll let the AI play against a true top Protoss. Yeah!)
- samcday 8y ago
  but he's currently not even in the top 50 even with his main race Does that really matter? AFAICT he's in the top #100 [1]. For a game as complex as SC2, it's pretty significant that DeepMind is showing these kinds of results already. It's only going to improve from here right? [1]: https://www.gosugamers.net/starcraft2/players/14064-tlo https://www.gosugamers.net/starcraft2/players/14064-tlo
  - codeflo 8y ago
    With Protoss, he's not remotely near that. And that does matter if you intend to make huge a press release in the shape of "DeepMind AI beats top pro player". We'll see in an hour if that's what they're going for.
    - aurelwu 8y ago
      there is a LiquidMana coming on now ;)
      - codeflo 8y ago
        I'm excited. :)
- solaxun 8y ago
  Even so - to me this is kind of a big deal. Go is fully observable, there might be an intractable number of combinations, but the game is well defined and everything is out in the open. In Starcraft you don't have full knowledge of the game (fog of war), there are infinitely more game paths , you have to scout, plan for the long term (macro), react tactically in the short-term, micromanage, etc. I'm actually shocked at it's performance.
- Sweetness 8y ago
  Because it's still insanely impressive. I would have been blown away if the AI was merely at the platinum level. This is an AI playing starcraft. Starcraft!
  - gambler 8y ago
    >This is an AI playing starcraft. Starcraft! Yeah, it's not like computers were ever able to play StarCraft before. Oh, wait...
    - deleted 8y ago
      [deleted]
    - Zaita 8y ago
      The AI normally picks a set strategy and attempts that. On harder difficulty the AI gets to cheat (more income per mineral collected). The SC2 AI is extremely primitive and at best Bronze level. The AI never really "reacts" to your actions, it's basically always on a set path. DeepMind is orders of magnitude better than the Starcraft AI. FWIW cheating AI in RTS games is extremely common. Often they don't get fog of war, or have higher income rates. RTS AI is very very hard, so this is very impressive.
- candiodari 8y ago
  In all of the games Alphastar was slower than TLO in the build order. I agree that TLO could have been faster, but this was definitely not the deciding factor. I think the main factor was Alpha* just had incredibly accurate assessment of exactly what part of it's army it needed to defend/attack and was very careful.
  - Buttons840 8y ago
    What's AlphaStar's APM. Can it just move every single unit perfectly in a single frame?
    - aurelwu 8y ago
      it's not perfect but definitely superhuman from what I see. These are demonstration matches and afaik they want to tune the "mechanical" abilities of the agents to be somewhat in the realm of the strongest human players but it's not a trivial thing as not every action is equal and just picking a APM number would make the AI way weaker than a human in some areas (splitting marines, building/morphing lots of units ) and way stronger in other areas (multitasking)
    - samfriedman 8y ago
      Commentator just mentioned that the APM they've observed from AlphaStar is comparable to pro players.
      - Buttons840 8y ago
        Yeah, in game 4 vs Mana the commentators mentioned there are APM limitations to keep the APM within the realm of human possibility, but the decision making and multitasking are beyond even the best players. "There is not a pro in the world that can control stalkers that way" said Artosis (a commentator).
      - nightcracker 8y ago
        It really isn't. Human APM is inflated and isn't true APM, humans spam click, repeat actions, make pointless actions a lot, etc. A human can have 300 APM but only make one true action per second. DeepMind never missclicks, doesn't have to move the mouse or fingers, never spamclicks, doesn't make pointless actions, so its APM isn't inflated. I do believe that with this progress AlphaStar with more training could beat humans even with a more realistic APM limit.
- Permit 8y ago
  >I'm a big fan of TLO, but he's currently not even in the top 50 even with his main race For what it's worth, this comment reminds me a lot of[1]: > FYI, Fan Hui, the current European Go champion that Google DeepMind defeated, is ONLY a 2nd Dan Go player. The Highest Dan ranking in Go is 9th Dan! [1] https://news.ycombinator.com/item?id=10983898 https://news.ycombinator.com/item?id=10983898
- superfx 8y ago
  > Their chess AI beating Stockfish involved some at least a bit questionable setup. I believe their eventual Science paper addressed these concerns.
  - gambler 8y ago
    Which paper? How did they address it?
    - superfx 8y ago
      Science paper: http://science.sciencemag.org/content/362/6419/1140 http://science.sciencemag.org/content/362/6419/1140 NYT article summarizing some of the issues addressed: https://www.nytimes.com/2018/12/26/science/chess-artificial-intelligence.html https://www.nytimes.com/2018/12/26/science/chess-artificial-...
    - gamegoblin 8y ago
      They played a newer version of SF (newest version at the time the paper was written), let SF use opening book, stronger CPU for SF, non-fixed time control for SF.
- archgoon 8y ago
  >but he's currently not even in the top 50 even with his main race. I don't understand this part of the complaint, and it also seems to go counter to the idea that they should be releasing more data. Do we only want to see the state of these systems once they're beating everyone? I prefer to see the progress of these systems over time. Seeing an AI getting stomped on by a pro player doesn't really show us what an AI is currently capable if most people would also get stomped by a pro player.
- apetresc 8y ago
  The fact that the community has been able to use the published paper to replicate all of AlphaZero's results in both chess, go (and a number of other similar combinatorial games) and get similar performance should lay your concerns to rest.
  - gambler 8y ago
    Can you provide some links?
    - Veedrac 8y ago
      http://zero.sjeng.org/ http://zero.sjeng.org/ for AlphaZero playing Go, and http://lczero.org/ http://lczero.org/ is at least par with AlphaZero Chess based off match reproductions.
    - methodover 8y ago
      LeelaChess is one example for Chess. https://github.com/LeelaChessZero/lczero https://github.com/LeelaChessZero/lczero
      - gambler 8y ago
        This project actually proves the point of the root comment. The community spent tons of time on tuning/training this network, and it still routinely loses to Stockfish, which runs on inferior hardware. It illustrates that: 1. Deep mind kept a lot of information about their methods undisclosed. 2. Despite all the claims of generality, the algorithm requires insane amounts of fiddling to train. Just read their blog. 3. The hype around AlphaZero crushing every other algorithm in chess is overblown. It's competitive, but not clearly superior. Still, Kudos to people running Lela project for doing what DeepMind should have done - describing how things really work and testing in real-life conditions.
  - CJefferson 8y ago
    Where?
jpdus 8y ago
I am somewhat disappointed that they stream only hand-picked replays. For a big, announced presentation, they should be at a stage to stream live games (like OpenAI did with Dota). However, apart from that great work by Deepmind and I am excited to follow the progress in this area.
- kamkha 8y ago
  They're streaming a single live game right now! They explained that the research build of the game being used doesn't have an "observer" mode so it's not quite suitable for streaming live games — and, in fact, the games that they were showing were played over a month ago. (The current live game is being shown from the human player's point of view, and as such is a bit… dizzying.)
- riprock 8y ago
  They just announced there will be live streamed game(s) after Mana's series.
- soohyung 8y ago
  Apparently there is going to be a live exhibition match after showing the replays of the matches against TLO and Mana.
- pbalau 8y ago
  They said all replays will be posted online.
- Veedrac 8y ago
  > For a big, announced presentation, they should be at a stage to stream live games (like OpenAI did with Dota). This is such an inane complaint. They didn't do it all live because they have results they want to show and don't have observer mode because they've been focusing on research and don't want to delay things.
berry_sortoro 8y ago
Am I the only one actually NOT impressed by this? I mean they all claim its so great that the AI is maybe challenging some wisdom and that overproducing probes might be a way to go. I do not know, I am not that deep into SC but if someone attack you who put those resources into units instead of probs, that can decide between win and lose or not? What the AI did in this game to win is NOT the overproduction of drones, it was simply and purely and to me not impressive at all the micro. Of course a AI can micro those stalkers like crazy in a inhuman way all across the map. I think you can even bookmark map locations with shortcuts in SC right? Still the AI does not think like a human this means it can manage all this units in parallel and again, THAT is what won the game. I never played Go but that was way more impressive to me then this. A human automatically has a huge handycap again a AI in SC. Of course they picked some nice guy who is all positive and good sports about it, exited about "new strategies" when in fact he just got wrecked because of superhuman micro. Not saw the other games but my guess is that micro always plays a huge part. And I just read in another comment that the AI was actually slower in build order ...
kmnc 8y ago
Very impressive... but it seems like the AI relies entirely on abusing blink stalkers which with perfect micro is basically impossible to counter. It is no surprise it can crush pros when it has perfect timing and zero mistakes in using these units. I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed (Micro harrasing early, early expansions, a very good understanding of when to attack/retreat). It looks just like your average pro..until you see its god tier micro. Well, after seeing Mana crush it in the live game it seems the AI had zero clue as to what to do... it seems like it calculated it couldn't win a fight so let Mana destroy its entire base. So, just like with the Dota AI we see pros can exploit and win easily once they play around the micro advantages.
- samfriedman 8y ago
  As the commentators mentioned, it's no use building units to counter your enemy's army (Immortals over Stalkers) when the enemy can control their army so much more effectively. I have to wonder if future competitive games will need to take into account the abilities of reinforcement learning algorithms when releasing balance patches.
  - dllu 8y ago
    But there IS use building units to counter your enemy's army. In the last live match when Mana won, his immortal archon zealot composition was what sealed the deal in the end.
    - orbifold 8y ago
      And his far superior positioning.
    - dragontamer 8y ago
      I mean, the AI in that last match couldn't see the whole map at once. Fog of War was enabled in all cases, but the last match was them enabling the "Scrolling window" that humans are forced to look at the game with. The AI in all of the other games could see and control all of its own units on the map simultaneously. No human has this ability due to the limitation of the screen.
knicholes 8y ago
I'm having a hard time knowing if it's using a single agent of the five top agents or an ensemble of their five agents.
- soohyung 8y ago
  They select 5 agents and then put up the professional player against a different agent in each game of the series. So it's a single agent, but a different one in each of the five games.
- saulrh 8y ago
  It sounds like each match in the Bo5 is against a different agent. It's a hair dirty, but I don't think it's too bad because the same kind of random-selection strategy is just as possible "in the wild".
maerF0x0 8y ago
Some of the commentary shows that the UI is a barrier . The fact we can understand the strategies, but cannot physically make it happen shows that at least some of the advantage is just the precision of the inputs. Could be a new way to play SC2 like games where we can better communicate our intentions to the game. For example, make a type of move action where the stalkers automatically retreat and stop to fire, instead of having to do move, attack, move, attack series of inputs.
- dwaltrip 8y ago
  I was just daydreaming the other day about an RTS where the UI controls and inputs are fully customizable by the users, for this exact reason. There could be even a community repository where the best known custom UIs are available.
  - unsigner 8y ago
    The best custom UI would have one button, "Launch DeepMind AI and let it win the game". The intermediate UIs would be along the continuum between full micro and this. Maybe developing these AI/UIs would be fun (until Google/DeepMind crushes you), but the game itself?
- degenerate 8y ago
  This thought process (of the UI limiting the player) is a valid point when we are talking about Human vs. AI strength and weaknesses in RTS games. However, when pitting one human player against another, the UI limitation actually adds strategic depth to the game. Unlike pure strategy games like chess, in Starcraft when you have to move, attack, move, attack, it puts a physical burden on the player. This physical burden to work around the limited UI allows one player to outperform the other on both a strategic and physical playing field, in real time. If both players were freed of their physical shackles, it would become a much more boring display of pure strategy, which eventually would be "solved" and the game would cease to be fun. If you look at the difference between Starcraft 1 and Starcraft 2, the two most significant UI changes are the ability to select more than 12 units at a time, and the ability to "multi-cast" spells in sequence when more than 1 spellcaster is selected. These two small UI improvements greatly reduce the skill gap between an average player and a "pro" player in Starcraft 2, to such a degree that many Starcraft 2 pro players went back to Starcraft 1, where their fast reflexes give them a much greater advantage against other players due to the limited UI. So an improvement to the UI would help human players beat AI, but it takes a strategic element out of the game when considering human vs. human.
Veelox 8y ago
Wow, Alphastar was able to 5-0 vs TLO (when he was off racing) and 5-0 vs Mana in an impressive manner. I am quite impressed that DeepMind has been able to take down Go and now Starcraft. From here on out, if DeepMind makes an announcement about a demonstration of X, I expect them to beat the top humans at X. Edit: In the live game Mana is able to beat Alphastar. Alphastar was trained with a more limited camera than the previous games. Mana was able to harass Alphastar with a warp prism then push in and win.
- iamjaredwalters 8y ago
  TLO and Mana are good players but Im not sure they fulfill your criteria of top humans at X
  - deleted 8y ago
    [deleted]
  - Veelox 8y ago
    According to a site [1] Mana is 19th. While that is probably overly high, I would say he is easily in the top 200 Starcraft 2 players in the world. I would also say that to get from beating a top 200 human to beating all humans is much smaller than from scratch to beating a top 200 human. [1] https://www.gosugamers.net/starcraft2/rankings https://www.gosugamers.net/starcraft2/rankings
    - cjbprime 8y ago
      Aligulac's had Mana around 50th in the world consistently for a recent 12 month period, so I think that's a more reasonable claim, and I'm happy saying that the 50th best player is a "top player in the world".
      - kzrdude 8y ago
        Is Mana the best Protoss player in Europe?
diimdeep 8y ago
10:0 AI wins. Precision micro control of units to save them from dying.
Buttons840 8y ago
Twitch.tv chat is full of shit-posting no doubt, but I did find some of the comments amusing: - "You only have 50 GPUs in 2019, LuL" - "AlphaStar is a cyberbully" - Many were claiming the AI had "1500 APM", not sure where that idea came from - and lots more
- gamegoblin 8y ago
  There were moments during clutch blink micro where the onscreen APM number for AlphaStar exceeded 1000.
plaidfuji 8y ago
This is mad impressive, for sure. With most AI problems, I can at least comprehend the approach, and the necessary combination of models. Not so much here. My first question is what is the input to the AI? Is it the raw pixel array of the display? Or does it get API-level readouts of what’s happening? Because implementing the CV just to segment the display output in real time is crazy enough. I would assume the latter. I think this basically proves that any problem that can be exhaustively simulated is solvable now. This may mark a tipping point, as every problem for which simulations exist (essentially infinite labels) is solved - then the balance will tip back toward making faster and more accurate sims (think multi-scale first principles physics stuff).
- triangleman 8y ago
  Blizzard released a client with an output accessible to machines, but still preserving fog of war. See here: https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/ https://deepmind.com/blog/deepmind-and-blizzard-open-starcra... Here's what I want to know, were these agents developed from scratch a la AlphaZero in chess, or did they have to create a number of abstractions in order to get the AI to start learning the game? In the initial demonstration they could hardly get the AI to mine minerals or do anything. How did they make the jump to actually good play?
  - johnmoberg 8y ago
    They mentioned that they initially used imitation learning on human replays.
- pbalau 8y ago
  Open source API. Please re-watch the start of the presentation.
- deleted 8y ago
  [deleted]
- deleted 8y ago
  [deleted]
devilmoon 8y ago
I'm not proficient in Starcraft, but if they have to make Pros play against 5 different agents for a Bo5 is it because the same agent would merely repeat the same game overall and the human player would be able to see through its strategy? AFAIK at least for Go/Chess DeepMind wasn't handpicking agents to send against human opponents, but it was simply a trained agent who would try its own strategy and respond to the opponent, in this case isn't it like the single agent only ever plays a single type of meta strat? If so, I think this is a bit less impressive than what I had predicted.
- aurelwu 8y ago
  Different agents for now, and 1 of the 6 possible matchups and just on one map (starcraft usually is played on multiple different maps/boards with different layouts over the course of a series). Given that this was just a demonstration match it's reasonable restrictions and they surely will lift them one by one (just like they removed the zoomed out vision in the final, live-played match). I'm really impressed with their performance, and even if my brother was just playing offrace I didn't expect him to lose.
- codeflo 8y ago
  There's a rock-paper-scissors like aspect to StarCraft openings where you essentially have to randomize your strategy so that the other player can't blindly counter what you're doing. I assume each of their agents learns one particular strategy; in that case, they could simply create a "meta-agent" that acts like one of their trained agents at random (probably using some weighted distribution).
  - devilmoon 8y ago
    Well, why wasn't this implemented then? For an hyped and live streamed event I'm expecting to see something that blows me away, not hand picked agents still in embryonic stage
lawrenceyan 8y ago
This was incredible. The games against MaNa even more so. TLO games were like MNIST, but AlphaStar going against MaNa was like ImageNet level honestly. Hats off to DeepMind.
asar 8y ago
The live exhibition match definitely made Alphastar look like a machine making the decisions, not a super smart being. The micro was obviously impressive but that should also be the easiest part to master. I share the sentiment that DeepMind is hosting big events to paint a very one-sided picture of man vs machine, so this last win of mana feels oddly satisfying.
- dllu 8y ago
  It is a little disappointing that Mana's win was achieved in part by simply exploiting Alphastar's poor response to the immortal drops by doing it over and over again. In contrast, Lee Sedol's win versus AlphaGo involved profound strategy and a particularly inspired "divine move" that humans get to brag about.
  - larkeith 8y ago
    On the one hand, it feels a bit cheap. On the other, getting ahead due to micro but losing due to lack of pattern recognition and problem solving seems like a rather complete demonstration of both the strengths and weaknesses of current AI.
  - asar 8y ago
    I agree that this makes Lee Sedol's one win against AlphaGo even more impressive. But deepmind also felt ready to host this event and it chose European pros to compete with not Asian pros, who arguably have better micro and innovate a lot of the new meta.
    - dllu 8y ago
      TLO is to Starcraft* as Fan Hui is to Go. They are strong players but not the best. I am guessing that Deepmind knows that the strength of their bot isn't ready to take on the very top players, and wanted to show off against a well-known personality in the Starcraft II scene. Incidentally, though, the strongest Starcraft II player currently is arguably Serral [1], who is Finnish. He beat the strongest Korean players to win the WCS World Championship last year. [1] https://liquipedia.net/starcraft2/Serral https://liquipedia.net/starcraft2/Serral * at least TLO's skill level when offracing as Protoss. His main race, Zerg, is very strong, ranked around 70-80th in the world.
1258989fdg 8y ago
The AI makes some awful decisions, such as building five observers. This calls into question its "understanding" of the game. It looks like a lot of its ability comes from micro, which it's unsurprising a computer can do better than a human. This is impressive, but with a lot of caveats. DeepMind's work on chess and go was impressive with no caveats whatsoever.
- devilmoon 8y ago
  I do believe the live match was with a newer version of the agent that hadn't been tested against human yet
larkeith 8y ago
I only had a chance to see the live game, but very impressive stuff, especially in the early game! However, it does seem to have the same issue as all other AIs in that it is inflexible and fails to adapt to unusual situations - As the commentators pointed out, it kept building oracles when being constantly harassed by an immortal drop, whereas any amateur player would be able to react more effectively by building a single phoenix. It also fails to recognize patterns - MaNa was able to find and repeatedly abuse a border between two strategies: whenever he was not actively dropping, AlphaStar would attempt to move out and push, then immediately retreat to defend when being dropped, whereas a human would recognize the reoccurring theme and either keep pushing or stay in their base. While not groundbreaking, still exciting to see an AI that can hold its ground against pro players - it certainly demonstrates the potential of machine learning for constrained problem spaces.
darkmighty 8y ago
I was left unconvinced (reproducing my comment from reddit): AlphaStar had 0 cost to sense the entire map simultaneously, and when they introduced the attention/context switching cost (camera control) in the showmatch, it lost. Also very notable for me in the showmatch was it seemed completely blind and exploitable to some strategies. When AlphaGo played top players, it made a few mistakes, but there wasn't anything obvious that it just couldn't see. Here it just couldn't think of making phoenixes vs. the warp prism harass which shows strategically it isn't near human level yet. It could clearly be exploited by back and forth harassment too (probably has to do with the limited memory those networks have). Finally, DeepMind were just emphasizing average APM, when it clearly reached totally superhuman levels at times -- even a top professional can't execute 900+ flawless apm in a battle that we saw. David Silver was clearly expecting the showmatch to be totally one sided (hence his speech that 'this is another historic victory for AI), but I were left with the opposite impression: that strategically top humans are still ahead in this game. This is not the end of the game for humans yet !
- roenxi 8y ago
  The original AlphaGo showing (vs Fan Hui) had obvious problems. They weren't catastrophic, it was clearly playing at a professional level stronger than 1d, but it also wasn't obviously superhuman. Lee Sedol could have expected to win his matches, and not been arrogant about it. The gap between that AlphaGo and the AlphaGo that beat Lee Sedol was probably something like 10 years of human improvement compressed into a couple of months. 500 Elo is huge. Even if they still have further to go, comparing this to AlphaGo's progress is reasonable. I'm going to live dangerously and make an assumption - you probably understand Starcraft better than Go, so can see the flaws more easily. Any mistake in Go comes back to something the AI just didn't see.
  - andreyk 8y ago
    StarCraft is also a FAR bigger challenge than Go. So much so that even saying AlphaStar is similar to AlphaGo is quite a stretch (see https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/ https://deepmind.com/blog/alphastar-mastering-real-time-stra... - the architecture and to some extent training regime are very different). I'd be surprised if they can learn the macro without imitation, unlike in Go ; but we'll see. Would sure be nice if they also released the paper...
throwaway19472 8y ago
The singularity is near (5 yeats away max)
porpoisely 8y ago
I haven't played SC in years. Surprised that it is still going strong considering I haven't heard much SC news in a while. I remember one trick against the SC AI was to send a probe to the AI's base and lure all the AI's probes out of its base. That was the easiest way to win against AI back then. Comparing DeepMind/Alphastar now to what SC AI was a decade ago really puts AI advancement into perspective.

163 points by Permit 8y ago

Here is another link (YouTube) that allows you to go back in time if you've missed anything: https://www.youtube.com/watch?v=cUTMhmVh1qs https://www.youtube.com/watch?v=cUTMhmVh1qs

codeflo 8y ago

I'm starting to get annoyed with these DeepMind publicity stunts. They don't release any code, and don't let anyone verify their results. Their chess AI beating Stockfish involved some at least a bit questionable setup. And here? I'm a big fan of TLO, but he's currently not even in the top 50 even with his main race. With Protoss, he's clearly making amateur-level mistakes. Why choose this setting to show off your supposedly superhuman AI? (Edit: This comment was made during the stream and it just looks like my point will be addressed before the stream's even finished, they'll let the AI play against a true top Protoss. Yeah!)

samcday 8y ago
but he's currently not even in the top 50 even with his main race Does that really matter? AFAICT he's in the top #100 [1]. For a game as complex as SC2, it's pretty significant that DeepMind is showing these kinds of results already. It's only going to improve from here right? [1]: https://www.gosugamers.net/starcraft2/players/14064-tlo https://www.gosugamers.net/starcraft2/players/14064-tlo
- codeflo 8y ago
  With Protoss, he's not remotely near that. And that does matter if you intend to make huge a press release in the shape of "DeepMind AI beats top pro player". We'll see in an hour if that's what they're going for.
  - aurelwu 8y ago
    there is a LiquidMana coming on now ;)
    - codeflo 8y ago
      I'm excited. :)
solaxun 8y ago
Even so - to me this is kind of a big deal. Go is fully observable, there might be an intractable number of combinations, but the game is well defined and everything is out in the open. In Starcraft you don't have full knowledge of the game (fog of war), there are infinitely more game paths , you have to scout, plan for the long term (macro), react tactically in the short-term, micromanage, etc. I'm actually shocked at it's performance.
Sweetness 8y ago
Because it's still insanely impressive. I would have been blown away if the AI was merely at the platinum level. This is an AI playing starcraft. Starcraft!
- gambler 8y ago
  >This is an AI playing starcraft. Starcraft! Yeah, it's not like computers were ever able to play StarCraft before. Oh, wait...
  - deleted 8y ago
    [deleted]
  - Zaita 8y ago
    The AI normally picks a set strategy and attempts that. On harder difficulty the AI gets to cheat (more income per mineral collected). The SC2 AI is extremely primitive and at best Bronze level. The AI never really "reacts" to your actions, it's basically always on a set path. DeepMind is orders of magnitude better than the Starcraft AI. FWIW cheating AI in RTS games is extremely common. Often they don't get fog of war, or have higher income rates. RTS AI is very very hard, so this is very impressive.
candiodari 8y ago
In all of the games Alphastar was slower than TLO in the build order. I agree that TLO could have been faster, but this was definitely not the deciding factor. I think the main factor was Alpha* just had incredibly accurate assessment of exactly what part of it's army it needed to defend/attack and was very careful.
- Buttons840 8y ago
  What's AlphaStar's APM. Can it just move every single unit perfectly in a single frame?
  - aurelwu 8y ago
    it's not perfect but definitely superhuman from what I see. These are demonstration matches and afaik they want to tune the "mechanical" abilities of the agents to be somewhat in the realm of the strongest human players but it's not a trivial thing as not every action is equal and just picking a APM number would make the AI way weaker than a human in some areas (splitting marines, building/morphing lots of units ) and way stronger in other areas (multitasking)
  - samfriedman 8y ago
    Commentator just mentioned that the APM they've observed from AlphaStar is comparable to pro players.
    - Buttons840 8y ago
      Yeah, in game 4 vs Mana the commentators mentioned there are APM limitations to keep the APM within the realm of human possibility, but the decision making and multitasking are beyond even the best players. "There is not a pro in the world that can control stalkers that way" said Artosis (a commentator).
    - nightcracker 8y ago
      It really isn't. Human APM is inflated and isn't true APM, humans spam click, repeat actions, make pointless actions a lot, etc. A human can have 300 APM but only make one true action per second. DeepMind never missclicks, doesn't have to move the mouse or fingers, never spamclicks, doesn't make pointless actions, so its APM isn't inflated. I do believe that with this progress AlphaStar with more training could beat humans even with a more realistic APM limit.
Permit 8y ago
>I'm a big fan of TLO, but he's currently not even in the top 50 even with his main race For what it's worth, this comment reminds me a lot of[1]: > FYI, Fan Hui, the current European Go champion that Google DeepMind defeated, is ONLY a 2nd Dan Go player. The Highest Dan ranking in Go is 9th Dan! [1] https://news.ycombinator.com/item?id=10983898 https://news.ycombinator.com/item?id=10983898
superfx 8y ago
> Their chess AI beating Stockfish involved some at least a bit questionable setup. I believe their eventual Science paper addressed these concerns.
- gambler 8y ago
  Which paper? How did they address it?
  - superfx 8y ago
    Science paper: http://science.sciencemag.org/content/362/6419/1140 http://science.sciencemag.org/content/362/6419/1140 NYT article summarizing some of the issues addressed: https://www.nytimes.com/2018/12/26/science/chess-artificial-intelligence.html https://www.nytimes.com/2018/12/26/science/chess-artificial-...
  - gamegoblin 8y ago
    They played a newer version of SF (newest version at the time the paper was written), let SF use opening book, stronger CPU for SF, non-fixed time control for SF.
archgoon 8y ago
>but he's currently not even in the top 50 even with his main race. I don't understand this part of the complaint, and it also seems to go counter to the idea that they should be releasing more data. Do we only want to see the state of these systems once they're beating everyone? I prefer to see the progress of these systems over time. Seeing an AI getting stomped on by a pro player doesn't really show us what an AI is currently capable if most people would also get stomped by a pro player.
apetresc 8y ago
The fact that the community has been able to use the published paper to replicate all of AlphaZero's results in both chess, go (and a number of other similar combinatorial games) and get similar performance should lay your concerns to rest.
- gambler 8y ago
  Can you provide some links?
  - Veedrac 8y ago
    http://zero.sjeng.org/ http://zero.sjeng.org/ for AlphaZero playing Go, and http://lczero.org/ http://lczero.org/ is at least par with AlphaZero Chess based off match reproductions.
  - methodover 8y ago
    LeelaChess is one example for Chess. https://github.com/LeelaChessZero/lczero https://github.com/LeelaChessZero/lczero
    - gambler 8y ago
      This project actually proves the point of the root comment. The community spent tons of time on tuning/training this network, and it still routinely loses to Stockfish, which runs on inferior hardware. It illustrates that: 1. Deep mind kept a lot of information about their methods undisclosed. 2. Despite all the claims of generality, the algorithm requires insane amounts of fiddling to train. Just read their blog. 3. The hype around AlphaZero crushing every other algorithm in chess is overblown. It's competitive, but not clearly superior. Still, Kudos to people running Lela project for doing what DeepMind should have done - describing how things really work and testing in real-life conditions.
- CJefferson 8y ago
  Where?

jpdus 8y ago

I am somewhat disappointed that they stream only hand-picked replays. For a big, announced presentation, they should be at a stage to stream live games (like OpenAI did with Dota). However, apart from that great work by Deepmind and I am excited to follow the progress in this area.

kamkha 8y ago
They're streaming a single live game right now! They explained that the research build of the game being used doesn't have an "observer" mode so it's not quite suitable for streaming live games — and, in fact, the games that they were showing were played over a month ago. (The current live game is being shown from the human player's point of view, and as such is a bit… dizzying.)
riprock 8y ago
They just announced there will be live streamed game(s) after Mana's series.
soohyung 8y ago
Apparently there is going to be a live exhibition match after showing the replays of the matches against TLO and Mana.
pbalau 8y ago
They said all replays will be posted online.
Veedrac 8y ago
> For a big, announced presentation, they should be at a stage to stream live games (like OpenAI did with Dota). This is such an inane complaint. They didn't do it all live because they have results they want to show and don't have observer mode because they've been focusing on research and don't want to delay things.

berry_sortoro 8y ago

Am I the only one actually NOT impressed by this? I mean they all claim its so great that the AI is maybe challenging some wisdom and that overproducing probes might be a way to go. I do not know, I am not that deep into SC but if someone attack you who put those resources into units instead of probs, that can decide between win and lose or not? What the AI did in this game to win is NOT the overproduction of drones, it was simply and purely and to me not impressive at all the micro. Of course a AI can micro those stalkers like crazy in a inhuman way all across the map. I think you can even bookmark map locations with shortcuts in SC right? Still the AI does not think like a human this means it can manage all this units in parallel and again, THAT is what won the game. I never played Go but that was way more impressive to me then this. A human automatically has a huge handycap again a AI in SC. Of course they picked some nice guy who is all positive and good sports about it, exited about "new strategies" when in fact he just got wrecked because of superhuman micro. Not saw the other games but my guess is that micro always plays a huge part. And I just read in another comment that the AI was actually slower in build order ...

kmnc 8y ago

Very impressive... but it seems like the AI relies entirely on abusing blink stalkers which with perfect micro is basically impossible to counter. It is no surprise it can crush pros when it has perfect timing and zero mistakes in using these units. I think the coolest thing is how the play of AI mirrors a similar style to how pros have developed (Micro harrasing early, early expansions, a very good understanding of when to attack/retreat). It looks just like your average pro..until you see its god tier micro. Well, after seeing Mana crush it in the live game it seems the AI had zero clue as to what to do... it seems like it calculated it couldn't win a fight so let Mana destroy its entire base. So, just like with the Dota AI we see pros can exploit and win easily once they play around the micro advantages.

samfriedman 8y ago
As the commentators mentioned, it's no use building units to counter your enemy's army (Immortals over Stalkers) when the enemy can control their army so much more effectively. I have to wonder if future competitive games will need to take into account the abilities of reinforcement learning algorithms when releasing balance patches.
- dllu 8y ago
  But there IS use building units to counter your enemy's army. In the last live match when Mana won, his immortal archon zealot composition was what sealed the deal in the end.
  - orbifold 8y ago
    And his far superior positioning.
  - dragontamer 8y ago
    I mean, the AI in that last match couldn't see the whole map at once. Fog of War was enabled in all cases, but the last match was them enabling the "Scrolling window" that humans are forced to look at the game with. The AI in all of the other games could see and control all of its own units on the map simultaneously. No human has this ability due to the limitation of the screen.

knicholes 8y ago

I'm having a hard time knowing if it's using a single agent of the five top agents or an ensemble of their five agents.

soohyung 8y ago
They select 5 agents and then put up the professional player against a different agent in each game of the series. So it's a single agent, but a different one in each of the five games.
saulrh 8y ago
It sounds like each match in the Bo5 is against a different agent. It's a hair dirty, but I don't think it's too bad because the same kind of random-selection strategy is just as possible "in the wild".

maerF0x0 8y ago

Some of the commentary shows that the UI is a barrier . The fact we can understand the strategies, but cannot physically make it happen shows that at least some of the advantage is just the precision of the inputs. Could be a new way to play SC2 like games where we can better communicate our intentions to the game. For example, make a type of move action where the stalkers automatically retreat and stop to fire, instead of having to do move, attack, move, attack series of inputs.

dwaltrip 8y ago
I was just daydreaming the other day about an RTS where the UI controls and inputs are fully customizable by the users, for this exact reason. There could be even a community repository where the best known custom UIs are available.
- unsigner 8y ago
  The best custom UI would have one button, "Launch DeepMind AI and let it win the game". The intermediate UIs would be along the continuum between full micro and this. Maybe developing these AI/UIs would be fun (until Google/DeepMind crushes you), but the game itself?
degenerate 8y ago
This thought process (of the UI limiting the player) is a valid point when we are talking about Human vs. AI strength and weaknesses in RTS games. However, when pitting one human player against another, the UI limitation actually adds strategic depth to the game. Unlike pure strategy games like chess, in Starcraft when you have to move, attack, move, attack, it puts a physical burden on the player. This physical burden to work around the limited UI allows one player to outperform the other on both a strategic and physical playing field, in real time. If both players were freed of their physical shackles, it would become a much more boring display of pure strategy, which eventually would be "solved" and the game would cease to be fun. If you look at the difference between Starcraft 1 and Starcraft 2, the two most significant UI changes are the ability to select more than 12 units at a time, and the ability to "multi-cast" spells in sequence when more than 1 spellcaster is selected. These two small UI improvements greatly reduce the skill gap between an average player and a "pro" player in Starcraft 2, to such a degree that many Starcraft 2 pro players went back to Starcraft 1, where their fast reflexes give them a much greater advantage against other players due to the limited UI. So an improvement to the UI would help human players beat AI, but it takes a strategic element out of the game when considering human vs. human.

Veelox 8y ago

Wow, Alphastar was able to 5-0 vs TLO (when he was off racing) and 5-0 vs Mana in an impressive manner. I am quite impressed that DeepMind has been able to take down Go and now Starcraft. From here on out, if DeepMind makes an announcement about a demonstration of X, I expect them to beat the top humans at X. Edit: In the live game Mana is able to beat Alphastar. Alphastar was trained with a more limited camera than the previous games. Mana was able to harass Alphastar with a warp prism then push in and win.

iamjaredwalters 8y ago
TLO and Mana are good players but Im not sure they fulfill your criteria of top humans at X
- deleted 8y ago
  [deleted]
- Veelox 8y ago
  According to a site [1] Mana is 19th. While that is probably overly high, I would say he is easily in the top 200 Starcraft 2 players in the world. I would also say that to get from beating a top 200 human to beating all humans is much smaller than from scratch to beating a top 200 human. [1] https://www.gosugamers.net/starcraft2/rankings https://www.gosugamers.net/starcraft2/rankings
  - cjbprime 8y ago
    Aligulac's had Mana around 50th in the world consistently for a recent 12 month period, so I think that's a more reasonable claim, and I'm happy saying that the 50th best player is a "top player in the world".
    - kzrdude 8y ago
      Is Mana the best Protoss player in Europe?

diimdeep 8y ago

10:0 AI wins. Precision micro control of units to save them from dying.

Buttons840 8y ago

Twitch.tv chat is full of shit-posting no doubt, but I did find some of the comments amusing: - "You only have 50 GPUs in 2019, LuL" - "AlphaStar is a cyberbully" - Many were claiming the AI had "1500 APM", not sure where that idea came from - and lots more

gamegoblin 8y ago
There were moments during clutch blink micro where the onscreen APM number for AlphaStar exceeded 1000.

plaidfuji 8y ago

This is mad impressive, for sure. With most AI problems, I can at least comprehend the approach, and the necessary combination of models. Not so much here. My first question is what is the input to the AI? Is it the raw pixel array of the display? Or does it get API-level readouts of what’s happening? Because implementing the CV just to segment the display output in real time is crazy enough. I would assume the latter. I think this basically proves that any problem that can be exhaustively simulated is solvable now. This may mark a tipping point, as every problem for which simulations exist (essentially infinite labels) is solved - then the balance will tip back toward making faster and more accurate sims (think multi-scale first principles physics stuff).

triangleman 8y ago
Blizzard released a client with an output accessible to machines, but still preserving fog of war. See here: https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/ https://deepmind.com/blog/deepmind-and-blizzard-open-starcra... Here's what I want to know, were these agents developed from scratch a la AlphaZero in chess, or did they have to create a number of abstractions in order to get the AI to start learning the game? In the initial demonstration they could hardly get the AI to mine minerals or do anything. How did they make the jump to actually good play?
- johnmoberg 8y ago
  They mentioned that they initially used imitation learning on human replays.
pbalau 8y ago
Open source API. Please re-watch the start of the presentation.
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]

devilmoon 8y ago

I'm not proficient in Starcraft, but if they have to make Pros play against 5 different agents for a Bo5 is it because the same agent would merely repeat the same game overall and the human player would be able to see through its strategy? AFAIK at least for Go/Chess DeepMind wasn't handpicking agents to send against human opponents, but it was simply a trained agent who would try its own strategy and respond to the opponent, in this case isn't it like the single agent only ever plays a single type of meta strat? If so, I think this is a bit less impressive than what I had predicted.

aurelwu 8y ago
Different agents for now, and 1 of the 6 possible matchups and just on one map (starcraft usually is played on multiple different maps/boards with different layouts over the course of a series). Given that this was just a demonstration match it's reasonable restrictions and they surely will lift them one by one (just like they removed the zoomed out vision in the final, live-played match). I'm really impressed with their performance, and even if my brother was just playing offrace I didn't expect him to lose.
codeflo 8y ago
There's a rock-paper-scissors like aspect to StarCraft openings where you essentially have to randomize your strategy so that the other player can't blindly counter what you're doing. I assume each of their agents learns one particular strategy; in that case, they could simply create a "meta-agent" that acts like one of their trained agents at random (probably using some weighted distribution).
- devilmoon 8y ago
  Well, why wasn't this implemented then? For an hyped and live streamed event I'm expecting to see something that blows me away, not hand picked agents still in embryonic stage

lawrenceyan 8y ago

This was incredible. The games against MaNa even more so. TLO games were like MNIST, but AlphaStar going against MaNa was like ImageNet level honestly. Hats off to DeepMind.

asar 8y ago

The live exhibition match definitely made Alphastar look like a machine making the decisions, not a super smart being. The micro was obviously impressive but that should also be the easiest part to master. I share the sentiment that DeepMind is hosting big events to paint a very one-sided picture of man vs machine, so this last win of mana feels oddly satisfying.

dllu 8y ago
It is a little disappointing that Mana's win was achieved in part by simply exploiting Alphastar's poor response to the immortal drops by doing it over and over again. In contrast, Lee Sedol's win versus AlphaGo involved profound strategy and a particularly inspired "divine move" that humans get to brag about.
- larkeith 8y ago
  On the one hand, it feels a bit cheap. On the other, getting ahead due to micro but losing due to lack of pattern recognition and problem solving seems like a rather complete demonstration of both the strengths and weaknesses of current AI.
- asar 8y ago
  I agree that this makes Lee Sedol's one win against AlphaGo even more impressive. But deepmind also felt ready to host this event and it chose European pros to compete with not Asian pros, who arguably have better micro and innovate a lot of the new meta.
  - dllu 8y ago
    TLO is to Starcraft* as Fan Hui is to Go. They are strong players but not the best. I am guessing that Deepmind knows that the strength of their bot isn't ready to take on the very top players, and wanted to show off against a well-known personality in the Starcraft II scene. Incidentally, though, the strongest Starcraft II player currently is arguably Serral [1], who is Finnish. He beat the strongest Korean players to win the WCS World Championship last year. [1] https://liquipedia.net/starcraft2/Serral https://liquipedia.net/starcraft2/Serral * at least TLO's skill level when offracing as Protoss. His main race, Zerg, is very strong, ranked around 70-80th in the world.

1258989fdg 8y ago

The AI makes some awful decisions, such as building five observers. This calls into question its "understanding" of the game. It looks like a lot of its ability comes from micro, which it's unsurprising a computer can do better than a human. This is impressive, but with a lot of caveats. DeepMind's work on chess and go was impressive with no caveats whatsoever.

devilmoon 8y ago
I do believe the live match was with a newer version of the agent that hadn't been tested against human yet

larkeith 8y ago

I only had a chance to see the live game, but very impressive stuff, especially in the early game! However, it does seem to have the same issue as all other AIs in that it is inflexible and fails to adapt to unusual situations - As the commentators pointed out, it kept building oracles when being constantly harassed by an immortal drop, whereas any amateur player would be able to react more effectively by building a single phoenix. It also fails to recognize patterns - MaNa was able to find and repeatedly abuse a border between two strategies: whenever he was not actively dropping, AlphaStar would attempt to move out and push, then immediately retreat to defend when being dropped, whereas a human would recognize the reoccurring theme and either keep pushing or stay in their base. While not groundbreaking, still exciting to see an AI that can hold its ground against pro players - it certainly demonstrates the potential of machine learning for constrained problem spaces.

darkmighty 8y ago

I was left unconvinced (reproducing my comment from reddit): AlphaStar had 0 cost to sense the entire map simultaneously, and when they introduced the attention/context switching cost (camera control) in the showmatch, it lost. Also very notable for me in the showmatch was it seemed completely blind and exploitable to some strategies. When AlphaGo played top players, it made a few mistakes, but there wasn't anything obvious that it just couldn't see. Here it just couldn't think of making phoenixes vs. the warp prism harass which shows strategically it isn't near human level yet. It could clearly be exploited by back and forth harassment too (probably has to do with the limited memory those networks have). Finally, DeepMind were just emphasizing average APM, when it clearly reached totally superhuman levels at times -- even a top professional can't execute 900+ flawless apm in a battle that we saw. David Silver was clearly expecting the showmatch to be totally one sided (hence his speech that 'this is another historic victory for AI), but I were left with the opposite impression: that strategically top humans are still ahead in this game. This is not the end of the game for humans yet !

roenxi 8y ago
The original AlphaGo showing (vs Fan Hui) had obvious problems. They weren't catastrophic, it was clearly playing at a professional level stronger than 1d, but it also wasn't obviously superhuman. Lee Sedol could have expected to win his matches, and not been arrogant about it. The gap between that AlphaGo and the AlphaGo that beat Lee Sedol was probably something like 10 years of human improvement compressed into a couple of months. 500 Elo is huge. Even if they still have further to go, comparing this to AlphaGo's progress is reasonable. I'm going to live dangerously and make an assumption - you probably understand Starcraft better than Go, so can see the flaws more easily. Any mistake in Go comes back to something the AI just didn't see.
- andreyk 8y ago
  StarCraft is also a FAR bigger challenge than Go. So much so that even saying AlphaStar is similar to AlphaGo is quite a stretch (see https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/ https://deepmind.com/blog/alphastar-mastering-real-time-stra... - the architecture and to some extent training regime are very different). I'd be surprised if they can learn the macro without imitation, unlike in Go ; but we'll see. Would sure be nice if they also released the paper...

throwaway19472 8y ago

The singularity is near (5 yeats away max)

porpoisely 8y ago

I haven't played SC in years. Surprised that it is still going strong considering I haven't heard much SC news in a while. I remember one trick against the SC AI was to send a probe to the AI's base and lure all the AI's probes out of its base. That was the easiest way to win against AI back then. Comparing DeepMind/Alphastar now to what SC AI was a decade ago really puts AI advancement into perspective.