11 ms·

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental c

This is a really interesting one to digest. As with previous announcements about AlphaStar, much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS. These points are very valid - stepping back however, this still feels like a pretty incredible accomplishment. I'm a gold league SC2 player, so maybe in the 30th-50th percentile. Three years ago, when DeepMind started this project (and after nearly two decades of research into SC/SC2) I could probably have beaten the best non-cheating AI. Now, after 3 years, this AI is playing at a Grandmaster level, under at least a reasonable approach to fairness. By comparison, according to the AlphaGo paper [1] the best Go AIs prior to AlphaGo were playing at a 6 Dan level, which looks to be somewhere in the 90-98th percentile [2]. The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. This is like if the world's best chess AI had gone from losing high school tournaments to being competitive with Kasparov in less than 3 years. Valid criticisms aside, this feels like an incredible achievement. [1] https://www.nature.com/articles/nature16961.pdf https://www.nature.com/articles/nature16961.pdf [2] https://senseis.xmp.net/?KGSRankHistogram https://senseis.xmp.net/?KGSRankHistogram

baddox 7y ago
> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?
- minblaster 7y ago
  Starcraft is an incredibly complex game. 10^26 possible moves at any point (you can click/drag anywhere on the screen, pressing a keyboard button as you do so), imperfect information, real-time constraints, etc.
  - nuclx 7y ago
    Yes it is, but you can prune the decision space significantly by discarding bogus choices, or more concretely by modelling higher level goals and then compiling them into actual moves.
  - 6gvONxR4sf7o 7y ago
    That pales in comparison to starcraft at a higher resolution, which has even more points to click on screen! And even that is nothing compared to playing horseshoes in the physical world. There are an infinite number of moves at any given moment! And an infinite number of moments! Horseshoes is clearly the most complex game of all.
    - gpm 7y ago
      Plank would like to say something to you about lengths. Which is why the most complex game of all is actually intergalactic horseshoes.
  - 300bps 7y ago
    Starcraft is an incredibly complex game. Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really allows for basic strategies to win out.
    - tryptophan 7y ago
      And the bot also can parse the entire screen in .03 seconds and then jump to a new area of the map. No human can monitor the entire map like the bot can.
      - bkemmer 7y ago
        But when the machines start competing with humans it should be fair?
      - empath75 7y ago
        Sometimes these critiques seem to be complaining that the ai is too intelligent. That’s sort of the point isn’t it.
      - baddox 7y ago
        If they want to model the constraints of humans, they probably need to create some “attention” system where the AI needs to choose where to invest its finite attention resources. For example, the AI could choose to focus more on the minimap, which would give them higher reaction times to move around the map, but would reduce their reaction times for things on the main screen. A great example would be seeing the faint image of cloaked units. How does that work with the AI? Can the just phrase the current screen and instantly see any cloaked unit? I can imagine an attention system where more attention dedicated to part of the screen would increase the probability of noticing a cloaked unit there.
    - pps 7y ago
      264 is not that high. People (if Serral is not a cyborg) can achieve even 1000 APM; and ~300 EPM average for a game.
- gwern 7y ago
  That depends on what you consider 'very little'. SC has had annual computer tournaments and research was published on it routinely. Few computer games see nearly as much money and effort put into developing AIs for them, yet the curve of progress was not exactly impressive.
  - baddox 7y ago
    Well, I don’t have any idea of absolute amounts of money, but I’d be more interested in relative amounts of money before and after DeepMind.
- interblag 7y ago
  There's definitely some truth to that. But looking back on arXiv there are papers going back years, including from Harvard/UofT [1], Tencent [2], Facebook [3], etc. There has also been an SC1 tournament running since at least 2011 [4] I'm sure that the AlphaStar effort absolutely dwarfed everything that came before in terms of investment (as I'm sure AlphaGo did for Go as well) but based on my reading I'd also bet that Starcraft has likely seen the most continued AI research effort of any imperfect information, real-time game over the last decade - I think that's also part of what made it appealing for DeepMind over Dota 2 or any of the other options in this class of game. [1] https://arxiv.org/pdf/1909.02682.pdf https://arxiv.org/pdf/1909.02682.pdf [2] https://arxiv.org/pdf/1907.09467.pdf https://arxiv.org/pdf/1907.09467.pdf [3] https://arxiv.org/pdf/1906.12266.pdf https://arxiv.org/pdf/1906.12266.pdf [4] https://liquipedia.net/starcraft/SSCAIT https://liquipedia.net/starcraft/SSCAIT
- Mirioron 7y ago
  What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that can compete with real players.
  - skybrian 7y ago
    There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really expect to see most game developers start using the latest machine learning techniques to build their own AI's. Maybe the really successful games with competitive leagues might be interested?
    - Mirioron 7y ago
      >There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game as much, but this almost always happens due to the AI having some inhuman advantage that a person could not really replicate. I think players do want the AI to pose a challenge - that's what a lot of casual PvP games are about. Humans are (were?) the only ones that could offer a fair match against another human. I will absolutely grant you the computational power point though. However, as hardware advances, this cost will become more and more acceptable. I don't expect to see widedspread adoption of this in commercial games within a decade, but I think that in 2 or 3 decades this will be the norm. In fact, I would bet that once we can make an AI that is good at a game, we can also make it weaker. A game could estimate a player's skill rating silently and then adjust the AI's strength to give the player a difficult/fun time. Of course, it could just be that AlphaStar is able to play well against humans, because players treat it like a human. Maybe the AI still has standard game AI-like weaknesses that can be exploited if people know that they're playing against an AI. Eg some Diamond league player went mass ravens and kicked AlphaStar's ass. The AI would have to learn how to deal with stuff like this on the fly and I don't think we're there yet.
rate_ofchange 7y ago
> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research In pretty much any field, top performing humans are at the physical limitation level, you will not see any sort of breakthrough, just incremental improvement. Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest. Some say that intelligence is not like that, that it can't be scaled arbitrarily. But the burden of proof is on them, especially given the consequences if it can.
- TremendousJudge 7y ago
  >Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest. It doesn't matter how much money or interest we have, but right now it isn't technically feasible to build a 36000 km tall crane (also known as a space elevator). Humanity simply couldn't get it done even if we poured all our current resources into that project. It isn't physically impossible, but such a behemoth has requirements that current materials science cannot meet. Building tall cranes generates useful know-how for a space elevator, but it's a really different problem, not just a matter of resources. Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have. Sure, building Deep Blue and AlphaGo most likely generated part of that money, but that doesn't mean we have everything we need. You can argue that is simply also money and interest, and that's true, but it's not in the same as building a 10 meter crane and a 20 meter crane.
  - dannypgh 7y ago
    > Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have. I agree with this, but it isn't clear to me that a general AI will have a significantly different impact on society than a world where task-specific well performing AIs are easy for anyone to develop. Sure, a general AI has a set of properties that are really fascinating to discuss and debate (including what is consciousness and whether AIs should be given rights), and perhaps a general AI is required for doomsday computers-taking-over scenarios, but the impacts that AI will have on our economy and politics don't require general AI.
jrx 7y ago
> This is like if the world's best chess AI had gone from losing high school tournaments to being competitive with Kasparov in less than 3 years. I don't think it's like that at all. On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. Adding support for the new game or new "environment" to existing deep learning based backbone still requires a bit of engineering work and a few creative tricks to unlock the best possible performance, but the underlying fundamentals are already there and are getting better and better understood. There is a reason why the progress in AI is so hard to measure. Anytime a next task is solved, there is a crowd saying it's not a "real AI" and that scientists are solving "toy problems". Both statements are totally true. But the underlying substance is that each of these toy problems is of increasing complexity and brings us closer and closer to solving the "real problems", which are mostly so undeniably complex that we couldn't attack them upfront. Still, the speed of progress in the field of AI research is staggering and it's hard to keep up with it even for professional researchers who spend all their waking hours working on these things. 6 years ago we were able to solve some Atari games from pixels. Today, that feels like a trivial exercise compared to modern techniques. With billions of dollars of investment pouring in and steady supply of fresh talent, it is very hard to predict what the pace of research will be in the coming years. It is entirely possible we'll encounter a wall we won't be able to overcome for a very long time. It is also possible that we won't, and in that case we're in for a very interesting next few decades.
- deleted 7y ago
  [deleted]
- deleted 7y ago
  [deleted]
- dmreedy 7y ago
  > On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. On a practical level, this is not true. There are different algorithms, different architectures, different hyperparameters required for each of these problems, and often for each subdomain within each of these problems, and often for each specific instance of these problems. It's difficult to draw any kind of holistic picture that combines all of the individual advances in each of these problem instances; that's why progress in AI is so hard to measure, and why a statement like "each of these toy problems...brings us closer and closer to solving the 'real problems'" is probably a bit too coarse-grained to be fair as well.
gambler 7y ago
>much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS It's amazing that most people here don't understand that AI performance in any one computer game relative to humans is largely irrelevant. A system that can play many games at a mediocre level, but does it without any hand-holding, clever APIs or architecture adaptation is infinitely more impressive than a system that can beat everyone in a specific game with all those things applied. Remember, most humans are completely mediocre at Chess, Go or StarCraft.
- emerongi 7y ago
  Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another. Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all games, because whenever a new game comes out, someone will be able to build a superhuman AI on it within a week/month. The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! Compare it to a human, who would need to spend enormous amounts of time to transfer their game-brain to someone else. If you want, you can bundle all those AIs into one and pretend it can play any game. "It's amazing that most people here don't understand that human performance in all computer games relative to AIs is largely irrelevant"
  - YeGoblynQueenne 7y ago
    >> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! So far I haven't seen Google sending their AIs to a friend.
    - eru 7y ago
      They are putting ML inference on the Pixel phones.
    - simonh 7y ago

baddox 7y ago

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research. Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

minblaster 7y ago
Starcraft is an incredibly complex game. 10^26 possible moves at any point (you can click/drag anywhere on the screen, pressing a keyboard button as you do so), imperfect information, real-time constraints, etc.
- nuclx 7y ago
  Yes it is, but you can prune the decision space significantly by discarding bogus choices, or more concretely by modelling higher level goals and then compiling them into actual moves.
- 6gvONxR4sf7o 7y ago
  That pales in comparison to starcraft at a higher resolution, which has even more points to click on screen! And even that is nothing compared to playing horseshoes in the physical world. There are an infinite number of moves at any given moment! And an infinite number of moments! Horseshoes is clearly the most complex game of all.
  - gpm 7y ago
    Plank would like to say something to you about lengths. Which is why the most complex game of all is actually intergalactic horseshoes.
- 300bps 7y ago
  Starcraft is an incredibly complex game. Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really allows for basic strategies to win out.
  - tryptophan 7y ago
    And the bot also can parse the entire screen in .03 seconds and then jump to a new area of the map. No human can monitor the entire map like the bot can.
    - bkemmer 7y ago
      But when the machines start competing with humans it should be fair?
    - empath75 7y ago
      Sometimes these critiques seem to be complaining that the ai is too intelligent. That’s sort of the point isn’t it.
    - baddox 7y ago
      If they want to model the constraints of humans, they probably need to create some “attention” system where the AI needs to choose where to invest its finite attention resources. For example, the AI could choose to focus more on the minimap, which would give them higher reaction times to move around the map, but would reduce their reaction times for things on the main screen. A great example would be seeing the faint image of cloaked units. How does that work with the AI? Can the just phrase the current screen and instantly see any cloaked unit? I can imagine an attention system where more attention dedicated to part of the screen would increase the probability of noticing a cloaked unit there.
  - pps 7y ago
    264 is not that high. People (if Serral is not a cyborg) can achieve even 1000 APM; and ~300 EPM average for a game.
gwern 7y ago
That depends on what you consider 'very little'. SC has had annual computer tournaments and research was published on it routinely. Few computer games see nearly as much money and effort put into developing AIs for them, yet the curve of progress was not exactly impressive.
- baddox 7y ago
  Well, I don’t have any idea of absolute amounts of money, but I’d be more interested in relative amounts of money before and after DeepMind.
interblag 7y ago
There's definitely some truth to that. But looking back on arXiv there are papers going back years, including from Harvard/UofT [1], Tencent [2], Facebook [3], etc. There has also been an SC1 tournament running since at least 2011 [4] I'm sure that the AlphaStar effort absolutely dwarfed everything that came before in terms of investment (as I'm sure AlphaGo did for Go as well) but based on my reading I'd also bet that Starcraft has likely seen the most continued AI research effort of any imperfect information, real-time game over the last decade - I think that's also part of what made it appealing for DeepMind over Dota 2 or any of the other options in this class of game. [1] https://arxiv.org/pdf/1909.02682.pdf https://arxiv.org/pdf/1909.02682.pdf [2] https://arxiv.org/pdf/1907.09467.pdf https://arxiv.org/pdf/1907.09467.pdf [3] https://arxiv.org/pdf/1906.12266.pdf https://arxiv.org/pdf/1906.12266.pdf [4] https://liquipedia.net/starcraft/SSCAIT https://liquipedia.net/starcraft/SSCAIT
Mirioron 7y ago
What makes this amazing isn't specific to StarCraft 2. AI in strategy games has been really lackluster. I can't think of a single example of a strategy game where an AI was competitive against experienced players due to strategy and tactics, rather than inhuman speed, accuracy or cheating. So it's not just about AI in StarCraft 2, but rather AI in essentially any (strategy) game. Now we have an example of an AI that can compete with real players.
- skybrian 7y ago
  There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. Also, due to commercial pressures it's tough to put a lot of effort into the AI for a game that's still being changed to make it more fun. Even now, I wouldn't really expect to see most game developers start using the latest machine learning techniques to build their own AI's. Maybe the really successful games with competitive leagues might be interested?
  - Mirioron 7y ago
    >There is a genuine advance here, but keep in mind that when an AI is developed by the game developers, they're not necessarily playing to win, but to make the AI fun to beat, and without using too much computer power, which would make the game slower. This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game as much, but this almost always happens due to the AI having some inhuman advantage that a person could not really replicate. I think players do want the AI to pose a challenge - that's what a lot of casual PvP games are about. Humans are (were?) the only ones that could offer a fair match against another human. I will absolutely grant you the computational power point though. However, as hardware advances, this cost will become more and more acceptable. I don't expect to see widedspread adoption of this in commercial games within a decade, but I think that in 2 or 3 decades this will be the norm. In fact, I would bet that once we can make an AI that is good at a game, we can also make it weaker. A game could estimate a player's skill rating silently and then adjust the AI's strength to give the player a difficult/fun time. Of course, it could just be that AlphaStar is able to play well against humans, because players treat it like a human. Maybe the AI still has standard game AI-like weaknesses that can be exploited if people know that they're playing against an AI. Eg some Diamond league player went mass ravens and kicked AlphaStar's ass. The AI would have to learn how to deal with stuff like this on the fly and I don't think we're there yet.

rate_ofchange 7y ago

> The speed at which AlphaStar overtook previous AIs seems to me to be nearly unprecedented in AI research In pretty much any field, top performing humans are at the physical limitation level, you will not see any sort of breakthrough, just incremental improvement. Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest. Some say that intelligence is not like that, that it can't be scaled arbitrarily. But the burden of proof is on them, especially given the consequences if it can.

TremendousJudge 7y ago
>Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest. It doesn't matter how much money or interest we have, but right now it isn't technically feasible to build a 36000 km tall crane (also known as a space elevator). Humanity simply couldn't get it done even if we poured all our current resources into that project. It isn't physically impossible, but such a behemoth has requirements that current materials science cannot meet. Building tall cranes generates useful know-how for a space elevator, but it's a really different problem, not just a matter of resources. Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have. Sure, building Deep Blue and AlphaGo most likely generated part of that money, but that doesn't mean we have everything we need. You can argue that is simply also money and interest, and that's true, but it's not in the same as building a 10 meter crane and a 20 meter crane.
- dannypgh 7y ago
  > Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have. I agree with this, but it isn't clear to me that a general AI will have a significantly different impact on society than a world where task-specific well performing AIs are easy for anyone to develop. Sure, a general AI has a set of properties that are really fascinating to discuss and debate (including what is consciousness and whether AIs should be given rights), and perhaps a general AI is required for doomsday computers-taking-over scenarios, but the impacts that AI will have on our economy and politics don't require general AI.

jrx 7y ago

> This is like if the world's best chess AI had gone from losing high school tournaments to being competitive with Kasparov in less than 3 years. I don't think it's like that at all. On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. Adding support for the new game or new "environment" to existing deep learning based backbone still requires a bit of engineering work and a few creative tricks to unlock the best possible performance, but the underlying fundamentals are already there and are getting better and better understood. There is a reason why the progress in AI is so hard to measure. Anytime a next task is solved, there is a crowd saying it's not a "real AI" and that scientists are solving "toy problems". Both statements are totally true. But the underlying substance is that each of these toy problems is of increasing complexity and brings us closer and closer to solving the "real problems", which are mostly so undeniably complex that we couldn't attack them upfront. Still, the speed of progress in the field of AI research is staggering and it's hard to keep up with it even for professional researchers who spend all their waking hours working on these things. 6 years ago we were able to solve some Atari games from pixels. Today, that feels like a trivial exercise compared to modern techniques. With billions of dollars of investment pouring in and steady supply of fresh talent, it is very hard to predict what the pace of research will be in the coming years. It is entirely possible we'll encounter a wall we won't be able to overcome for a very long time. It is also possible that we won't, and in that case we're in for a very interesting next few decades.

deleted 7y ago
[deleted]
deleted 7y ago
[deleted]
dmreedy 7y ago
> On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. On a practical level, this is not true. There are different algorithms, different architectures, different hyperparameters required for each of these problems, and often for each subdomain within each of these problems, and often for each specific instance of these problems. It's difficult to draw any kind of holistic picture that combines all of the individual advances in each of these problem instances; that's why progress in AI is so hard to measure, and why a statement like "each of these toy problems...brings us closer and closer to solving the 'real problems'" is probably a bit too coarse-grained to be fair as well.

gambler 7y ago

>much of the feedback (here and elsewhere) is about the fundamental challenge of assessing human vs. machine in an RTS It's amazing that most people here don't understand that AI performance in any one computer game relative to humans is largely irrelevant. A system that can play many games at a mediocre level, but does it without any hand-holding, clever APIs or architecture adaptation is infinitely more impressive than a system that can beat everyone in a specific game with all those things applied. Remember, most humans are completely mediocre at Chess, Go or StarCraft.

emerongi 7y ago
Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another. Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all games, because whenever a new game comes out, someone will be able to build a superhuman AI on it within a week/month. The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! Compare it to a human, who would need to spend enormous amounts of time to transfer their game-brain to someone else. If you want, you can bundle all those AIs into one and pretend it can play any game. "It's amazing that most people here don't understand that human performance in all computer games relative to AIs is largely irrelevant"
- YeGoblynQueenne 7y ago
  >> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! So far I haven't seen Google sending their AIs to a friend.
  - eru 7y ago
    They are putting ML inference on the Pixel phones.
  - simonh 7y ago