3 ms·

The point of the starcraft bot was never to be able to beat humans, it was about getting headlines and moving on. To be honest getting good at starcraft isn't r

The point of the starcraft bot was never to be able to beat humans, it was about getting headlines and moving on. To be honest getting good at starcraft isn't really an interesting problem for AI to solve, you can probably do it if you throw enough GPUs at the problem, but you learn pretty much nothing in the process.

NikolaeVarius 5y ago
I'm skeptical of your claims that Starcraft is not interesting problems for AI. For many problems, the AI is given as much information as possible of the state of the system to make decisions. The AI knows the entire state of the game in Tetris or Go or Chess. Starcraft/RTSs are designed specifically to remove information from the player and force you to balance info gathering, building, and countering new information, which is a massive difference from the currently impressive AlphaGo and such
oblak 5y ago
Strongly disagree. They were clearly pushing very hard right until their precious bot was defeated repeatedly by - again - a low tier European "pro". I still remember the principal (somewhat curly hair and glasses) engineer looking extremely pissed after the matches. Funny how they switched priorities right after that. 100% completely unrelated, as you say. Also, as NikolaeVarius pointed, SC is a game of imperfect information. If that's not interesting from an AI perspective, I don't know what is.
NavinF 5y ago
> starcraft isn't really an interesting problem for AI to solve I completely disagree. This is much more interesting than the toy problems that most academics work on. RL problems in this space are inherently difficult since you only get 1bit of “ground truth” information (win/loss) at the end of a game and have to guess which of your actions contributed to that outcome. Sure you can throw hardware at the problem, but you’ve got a huge incentive to minimize costs. It’s only recently that the default consensus for RL problems changed from “this is an inherently creative endeavor and needs human input” to “you can probably do it if you throw enough GPUs at the problem”.
- gwern 5y ago
  I'd say SC2 is still in the "inherently creative" part. Even with the very interesting AlphaStar League partially solving the self-play diversity and exploration problem, it's not at all obvious that just sinking in another 10-100x TPU-time into AlphaStar would patch up all of the errors and poor strategy AS exhibited - in fact, I'd bet heavily against it, since spending a large-but-still-feasible amount of compute doesn't appear to have fixed OA5 or AlphaGo's similar-looking problems, but AlphaZero was required. Now, applying MuZero might be powerful to solve SC2 with much more compute, but that is still not something many DRL researchers would give you large odds on, and you still probably need to come up with some ways to abstract it and condense the game tree into something tractable for planning over, especially with the partial information making modeling the future much harder. (VQ-VAE and related generative modeling approaches are promising in this regard... but still no slam dunk. If DM announced tomorrow that MuZero+VQ-VAE had defeated AS and a bunch of human pros, people would be extremely shocked, even as they scrambled to tweet about how they saw it coming all along & also are very offended by the waste of CO2 and how unreplicable it is on a grad student budget.)

NikolaeVarius 5y ago

I'm skeptical of your claims that Starcraft is not interesting problems for AI. For many problems, the AI is given as much information as possible of the state of the system to make decisions. The AI knows the entire state of the game in Tetris or Go or Chess. Starcraft/RTSs are designed specifically to remove information from the player and force you to balance info gathering, building, and countering new information, which is a massive difference from the currently impressive AlphaGo and such

oblak 5y ago

Strongly disagree. They were clearly pushing very hard right until their precious bot was defeated repeatedly by - again - a low tier European "pro". I still remember the principal (somewhat curly hair and glasses) engineer looking extremely pissed after the matches. Funny how they switched priorities right after that. 100% completely unrelated, as you say. Also, as NikolaeVarius pointed, SC is a game of imperfect information. If that's not interesting from an AI perspective, I don't know what is.

NavinF 5y ago

> starcraft isn't really an interesting problem for AI to solve I completely disagree. This is much more interesting than the toy problems that most academics work on. RL problems in this space are inherently difficult since you only get 1bit of “ground truth” information (win/loss) at the end of a game and have to guess which of your actions contributed to that outcome. Sure you can throw hardware at the problem, but you’ve got a huge incentive to minimize costs. It’s only recently that the default consensus for RL problems changed from “this is an inherently creative endeavor and needs human input” to “you can probably do it if you throw enough GPUs at the problem”.

gwern 5y ago
I'd say SC2 is still in the "inherently creative" part. Even with the very interesting AlphaStar League partially solving the self-play diversity and exploration problem, it's not at all obvious that just sinking in another 10-100x TPU-time into AlphaStar would patch up all of the errors and poor strategy AS exhibited - in fact, I'd bet heavily against it, since spending a large-but-still-feasible amount of compute doesn't appear to have fixed OA5 or AlphaGo's similar-looking problems, but AlphaZero was required. Now, applying MuZero might be powerful to solve SC2 with much more compute, but that is still not something many DRL researchers would give you large odds on, and you still probably need to come up with some ways to abstract it and condense the game tree into something tractable for planning over, especially with the partial information making modeling the future much harder. (VQ-VAE and related generative modeling approaches are promising in this regard... but still no slam dunk. If DM announced tomorrow that MuZero+VQ-VAE had defeated AS and a bunch of human pros, people would be extremely shocked, even as they scrambled to tweet about how they saw it coming all along & also are very offended by the waste of CO2 and how unreplicable it is on a grad student budget.)