8 ms·

At Blizzcon 2019, Alphastar beat Serral, undeniably a top player (although he didn't get to use his own keyboard and settings or get to prepare). Serral was abl

At Blizzcon 2019, Alphastar beat Serral, undeniably a top player (although he didn't get to use his own keyboard and settings or get to prepare). Serral was able to beat the Terran agent though. https://www.youtube.com/watch?v=nbiVbd_CEIA https://www.youtube.com/watch?v=nbiVbd_CEIA

ZephyrBlu 6y ago
People also cheesed the shit out the bot and won though. None of these AIs have proved to be robust to exploitation yet.
- CyberRage 6y ago
  StarCraft is built in such a way that you can't create a perfect, 100% winrate agent. Since there is hidden information, you could always miss a corner of the map where the enemy hidden some units and you lose the game. Is Alphastar "perfect"? no. Is it better than 99.9% of all humans? absolutely. You don't need to create a perfect agent in most cases, self driving a classic example. If you were to deploy an agent that drives 95% better than all humans the effects would be huge. It would still fail in some scenarios where professional drivers won't be it doesn't really matter because most people are not that.
  - ZephyrBlu 6y ago
    I know the bot will never have 100% winrate, but I think it shouldn't be able to be exploited (I.e. repeatedly beaten using the same strategy). Let me give you an example [0]. When AlphaStar was playing on the ladder a player in Diamond league (~70-80th percentile) beat AlphaStar easily using mass Ravens. If you're not aware of the strategy, it's a turtle strategy where the player masses air units and is generally terrible. But AlphaStar was confused by the strategy, and so it lost by a large margin. Deploying an AI which can be exploited like this is asking for trouble. [0] https://www.reddit.com/r/starcraft/comments/cgzieq/alphastar_loses_to_4000_mmr_first_loss_as_terran/ https://www.reddit.com/r/starcraft/comments/cgzieq/alphastar...
    - CyberRage 6y ago
      But that could be fixed technically. Deepmind's goal was not to create an "unexploitable" agent but to prove that ML algorithms can cope with complex, dynamic environments such as StarCraft. It seems to you weird but the same agent probably wins against GM's most of the time. humans have weaknesses too. The AI simply leans on its strengths just like humans do.
      - ZephyrBlu 6y ago
        > It seems to you weird but the same agent probably wins against GM's most of the time. humans have weaknesses too This is the whole problem though. AlphaStar beats GMs but can lose to weird strategies. On the other hand, GMs will almost never lose (Most likely >99% winrate) to a Diamond player no matter how weird their strategies are. The AI has strengths, but it also has glaring weaknesses. Imagine if you had an AI flying a plane and 99% of the time it was far better than a human pilot but 1% of the time it crashed and killed everyone. I would not fly on that plane. Maybe a bunch more training data and time would solve this type of problem, but I'm skeptical.
        CyberRage 6y ago
        You're beautifully showing the human nature which can be problematic in my opinion. First of, no human player achieves 99% winrate against diamond players. there are many cheeses, one miss-step and you lose. GM's can lose to Diamond players. Now for the main part, you're saying and I'm rephrasing here: Even if the AI is statistically better than humans because it has some weaknesses I'm going to prefer the human. But still at the end of the day, the AI does a better job on average and will be safer to use than human pilots! We already heavily rely on software\algorithms for our most important things. all modern vehicles use electronic systems that monitor\manage several key components, stock market is heavily managed by bots. If AI can do a significantly better job than human, I would choose the AI, even if it behaves strange in that 0.1% of cases. humans are not as reliable as you think.
        ZephyrBlu 6y ago
  - Dylan16807 6y ago
    > StarCraft is built in such a way that you can't create a perfect, 100% winrate agent. The worry isn't about perfect winrate, it's about finding strategies that can consistently cause the AI to lose over and over. In a cooperative environment, a high percentage is great. In a competitive environment, that .1% of scenarios where it's really weak will suddenly become the majority of games it faces.
    - deleted 6y ago
      [deleted]
- jayd16 6y ago
  Cheese is a legitimate strategy, though. I'm not sure its been proven that the most successful overall strategy is unbeatable. Besides perfect skill, you still need to worry about all in strategies. In my eyes, losing to cheese could still be possible even if you're the best overall player. I do think its fair to say these AIs should be able to grow after losing to a strategy once.
  - TulliusCicero 6y ago
    > Cheese is a legitimate strategy, though. Of course it is. But do the same cheese to a top player over and over and it will rapidly become ineffective, usually within a game or two. Each AlphaStar agent can just be exploited endlessly.
TulliusCicero 6y ago
IIRC each individual agent is essentially built around a single strategy/playstyle, not to mention the agents relied heavily on mechanical advantages to win. AlphaStar basically got to 'good enough' strategy, then won on control, which computers obviously have a massive advantage with.
rollcat 6y ago
In StarCraft 2, if you're used to playing with specific settings (esp. graphics, keybindings, mouse speed), having to revert to standard is a huge handicap. There are also OS settings (keyboard delay and repeat rate) that if left at default basically make a "standard" game unplayable, especially for Zerg.
- hntrader 6y ago
  He didn't get to use his own hotkeys? Are we sure about that? I'm sceptical, it'd make the game unplayable and it's easy to import hotkeys.
  - porphyra 6y ago
    It was at one of those public booths, not really a sanctioned showmatch. So Serral was using a public computer rather than one that he can log into his own account with.

ZephyrBlu 6y ago

People also cheesed the shit out the bot and won though. None of these AIs have proved to be robust to exploitation yet.

CyberRage 6y ago
StarCraft is built in such a way that you can't create a perfect, 100% winrate agent. Since there is hidden information, you could always miss a corner of the map where the enemy hidden some units and you lose the game. Is Alphastar "perfect"? no. Is it better than 99.9% of all humans? absolutely. You don't need to create a perfect agent in most cases, self driving a classic example. If you were to deploy an agent that drives 95% better than all humans the effects would be huge. It would still fail in some scenarios where professional drivers won't be it doesn't really matter because most people are not that.
- ZephyrBlu 6y ago
  I know the bot will never have 100% winrate, but I think it shouldn't be able to be exploited (I.e. repeatedly beaten using the same strategy). Let me give you an example [0]. When AlphaStar was playing on the ladder a player in Diamond league (~70-80th percentile) beat AlphaStar easily using mass Ravens. If you're not aware of the strategy, it's a turtle strategy where the player masses air units and is generally terrible. But AlphaStar was confused by the strategy, and so it lost by a large margin. Deploying an AI which can be exploited like this is asking for trouble. [0] https://www.reddit.com/r/starcraft/comments/cgzieq/alphastar_loses_to_4000_mmr_first_loss_as_terran/ https://www.reddit.com/r/starcraft/comments/cgzieq/alphastar...
  - CyberRage 6y ago
    But that could be fixed technically. Deepmind's goal was not to create an "unexploitable" agent but to prove that ML algorithms can cope with complex, dynamic environments such as StarCraft. It seems to you weird but the same agent probably wins against GM's most of the time. humans have weaknesses too. The AI simply leans on its strengths just like humans do.
    - ZephyrBlu 6y ago
      > It seems to you weird but the same agent probably wins against GM's most of the time. humans have weaknesses too This is the whole problem though. AlphaStar beats GMs but can lose to weird strategies. On the other hand, GMs will almost never lose (Most likely >99% winrate) to a Diamond player no matter how weird their strategies are. The AI has strengths, but it also has glaring weaknesses. Imagine if you had an AI flying a plane and 99% of the time it was far better than a human pilot but 1% of the time it crashed and killed everyone. I would not fly on that plane. Maybe a bunch more training data and time would solve this type of problem, but I'm skeptical.
      - CyberRage 6y ago
        You're beautifully showing the human nature which can be problematic in my opinion. First of, no human player achieves 99% winrate against diamond players. there are many cheeses, one miss-step and you lose. GM's can lose to Diamond players. Now for the main part, you're saying and I'm rephrasing here: Even if the AI is statistically better than humans because it has some weaknesses I'm going to prefer the human. But still at the end of the day, the AI does a better job on average and will be safer to use than human pilots! We already heavily rely on software\algorithms for our most important things. all modern vehicles use electronic systems that monitor\manage several key components, stock market is heavily managed by bots. If AI can do a significantly better job than human, I would choose the AI, even if it behaves strange in that 0.1% of cases. humans are not as reliable as you think.
        ZephyrBlu 6y ago
- Dylan16807 6y ago
  > StarCraft is built in such a way that you can't create a perfect, 100% winrate agent. The worry isn't about perfect winrate, it's about finding strategies that can consistently cause the AI to lose over and over. In a cooperative environment, a high percentage is great. In a competitive environment, that .1% of scenarios where it's really weak will suddenly become the majority of games it faces.
  - deleted 6y ago
    [deleted]
jayd16 6y ago
Cheese is a legitimate strategy, though. I'm not sure its been proven that the most successful overall strategy is unbeatable. Besides perfect skill, you still need to worry about all in strategies. In my eyes, losing to cheese could still be possible even if you're the best overall player. I do think its fair to say these AIs should be able to grow after losing to a strategy once.
- TulliusCicero 6y ago
  > Cheese is a legitimate strategy, though. Of course it is. But do the same cheese to a top player over and over and it will rapidly become ineffective, usually within a game or two. Each AlphaStar agent can just be exploited endlessly.

TulliusCicero 6y ago

IIRC each individual agent is essentially built around a single strategy/playstyle, not to mention the agents relied heavily on mechanical advantages to win. AlphaStar basically got to 'good enough' strategy, then won on control, which computers obviously have a massive advantage with.

rollcat 6y ago

In StarCraft 2, if you're used to playing with specific settings (esp. graphics, keybindings, mouse speed), having to revert to standard is a huge handicap. There are also OS settings (keyboard delay and repeat rate) that if left at default basically make a "standard" game unplayable, especially for Zerg.

hntrader 6y ago
He didn't get to use his own hotkeys? Are we sure about that? I'm sceptical, it'd make the game unplayable and it's easy to import hotkeys.
- porphyra 6y ago
  It was at one of those public booths, not really a sanctioned showmatch. So Serral was using a public computer rather than one that he can log into his own account with.