5 ms·

Grandmaster level in StarCraft II using multi-agent reinforcement learning

42 points by Jyaif 7y ago

partingshots 7y ago
Just curious, not that DeepMind has to bring any return on investment anytime soon given how rich it’s Alphabet parent is and can easily continue to take out boatloads of money, but is there any foreseeable point of profitability for this company? Or is this just more of a passion project from Sergey and/or Larry that they’re personally choosing to fund?
- 29athrowaway 7y ago
  They have received worldwide news coverage for some of their work. Like Alpha Go. AI/ML aside, those achievements are huge in terms of marketing and PR. Then, some R&D expenses can be tax deductible.
  - ldiracdelta 7y ago
    Just like... IBM?
- ArtWomb 7y ago
  Looks like R & D only so far. But they are in proximity to City of London. JP Morgan and Goldman Sachs are both staffing up their AI Research divisions. They all compete in the same talent pool. So I would not be surprised to see some sort of partnership with a major bank in the coming year. Regarding multi-agent deep reinforcement learning. If anyone has caught any of the League of Legends play recently streaming on Twitch leading up to the Finals. Particularly Faker and STK. DeepMind certainly has it's work cut out for them. Even 99.9% expert level human play won't be enough ;)
- comex 7y ago
  Well, Google recently announced they were using BERT, an AI model, to improve search results. The developers of BERT are from Google but apparently not part of DeepMind; however, DeepMind has also published some things on language modeling, so... there's that, at least.
- temp20160423 7y ago
  This project doesn't seem that expensive to run and it resulted in a Nature paper. Seems like a favorable ROI already even for a for-profit company (given Google's size and possible benefits in PR and recruiting).
- forgot-my-pw 7y ago
  Some Deepmind projects got integrated into Google projects in the past: - WaveNet: https://cloud.google.com/text-to-speech/docs/wavenet https://cloud.google.com/text-to-speech/docs/wavenet - Data centre efficiency: https://deepmind.com/blog/article/deepmind-ai-reduces-google-data-centre-cooling-bill-40 https://deepmind.com/blog/article/deepmind-ai-reduces-google... But yes, these are probably very small fraction of their research projects. Since Google declared to be an AI-first company, this is probably still a good investment to keep them on the bleeding edge.
pa7ch 7y ago
Its not clear all the actions they've taken to limit themselves to human input. Some players feel that the EPM (effective actions per minute) of AlphaStar is enormously higher then a human's during a battle. This is likened to a more advanced aimbot where it teaches human's nothing and has an unfair advantage. Alternatively, alpha go has advanced the state of human go playing without feeling like it cheats. AlphaStar is impressive but I don't think AI has quite dominated the game of starcraft as convincingly as it has go.
- YeGoblynQueenne 7y ago
  It seems that the superhuman EPM was more than just a feeling, because human players were able to correctly identify DeepMind's AI from its actions: After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossible, for a human. In response, DeepMind began using a number of tricks to keep the trial blind and stop players spotting AlphaStar, such as switching accounts regularly. From the Nature article: https://www.nature.com/articles/d41586-019-03298-6 https://www.nature.com/articles/d41586-019-03298-6
  - musicale 7y ago
    Although hiding AlphaStar's identity was part of the experiment, it also seems a bit sneaky to try to hide the smoking gun of impossible actions.
    - meowface 7y ago
      The main impossible action that serves as an AlphaStar giveaway is that it doesn't use control groups. That by itself doesn't suggest an unfair advantage, though. It's just noticeable in replays. The APM still seems to be in line with high-level players.

42 points by Jyaif 7y ago

Just curious, not that DeepMind has to bring any return on investment anytime soon given how rich it’s Alphabet parent is and can easily continue to take out boatloads of money, but is there any foreseeable point of profitability for this company? Or is this just more of a passion project from Sergey and/or Larry that they’re personally choosing to fund?

29athrowaway 7y ago
They have received worldwide news coverage for some of their work. Like Alpha Go. AI/ML aside, those achievements are huge in terms of marketing and PR. Then, some R&D expenses can be tax deductible.
- ldiracdelta 7y ago
  Just like... IBM?
ArtWomb 7y ago
Looks like R & D only so far. But they are in proximity to City of London. JP Morgan and Goldman Sachs are both staffing up their AI Research divisions. They all compete in the same talent pool. So I would not be surprised to see some sort of partnership with a major bank in the coming year. Regarding multi-agent deep reinforcement learning. If anyone has caught any of the League of Legends play recently streaming on Twitch leading up to the Finals. Particularly Faker and STK. DeepMind certainly has it's work cut out for them. Even 99.9% expert level human play won't be enough ;)
comex 7y ago
Well, Google recently announced they were using BERT, an AI model, to improve search results. The developers of BERT are from Google but apparently not part of DeepMind; however, DeepMind has also published some things on language modeling, so... there's that, at least.
temp20160423 7y ago
This project doesn't seem that expensive to run and it resulted in a Nature paper. Seems like a favorable ROI already even for a for-profit company (given Google's size and possible benefits in PR and recruiting).
forgot-my-pw 7y ago
Some Deepmind projects got integrated into Google projects in the past: - WaveNet: https://cloud.google.com/text-to-speech/docs/wavenet https://cloud.google.com/text-to-speech/docs/wavenet - Data centre efficiency: https://deepmind.com/blog/article/deepmind-ai-reduces-google-data-centre-cooling-bill-40 https://deepmind.com/blog/article/deepmind-ai-reduces-google... But yes, these are probably very small fraction of their research projects. Since Google declared to be an AI-first company, this is probably still a good investment to keep them on the bleeding edge.

pa7ch 7y ago

Its not clear all the actions they've taken to limit themselves to human input. Some players feel that the EPM (effective actions per minute) of AlphaStar is enormously higher then a human's during a battle. This is likened to a more advanced aimbot where it teaches human's nothing and has an unfair advantage. Alternatively, alpha go has advanced the state of human go playing without feeling like it cheats. AlphaStar is impressive but I don't think AI has quite dominated the game of starcraft as convincingly as it has go.

YeGoblynQueenne 7y ago
It seems that the superhuman EPM was more than just a feeling, because human players were able to correctly identify DeepMind's AI from its actions: After 50 games, however, DeepMind hit a snag. Some players had noticed that three user accounts on the Battle.net gaming platform had played the exact same number of StarCraft II games over a similar time frame — the three accounts that AlphaStar was secretly using. When watching replays of these matches, players noticed that the account owner was performing actions that would be extremely difficult, if not impossible, for a human. In response, DeepMind began using a number of tricks to keep the trial blind and stop players spotting AlphaStar, such as switching accounts regularly. From the Nature article: https://www.nature.com/articles/d41586-019-03298-6 https://www.nature.com/articles/d41586-019-03298-6
- musicale 7y ago
  Although hiding AlphaStar's identity was part of the experiment, it also seems a bit sneaky to try to hide the smoking gun of impossible actions.
  - meowface 7y ago
    The main impossible action that serves as an AlphaStar giveaway is that it doesn't use control groups. That by itself doesn't suggest an unfair advantage, though. It's just noticeable in replays. The APM still seems to be in line with high-level players.