4 ms·

I think this is kind of pedantic. They built an AI agent to take pixel data as input, and provide mouse movements and clicks as output, and rather than just fla

I think this is kind of pedantic. They built an AI agent to take pixel data as input, and provide mouse movements and clicks as output, and rather than just flail around like a baby it actually played the games with a sophisticated competency. This to me is such an incredible achievement that I have no doubt that it could be enhanced to defeat top players consistently and easily. As another commenter remarks, there are holes to plug in terms of exploitable behaviours that are locked into the model, but this too I'm confident they will find a general method of preventing; on the other hand, it's not like humans aren't susceptible to similar exploits by competitors in situations where they decide to cease innovation/learning

ZephyrBlu 6y ago
> As another commenter remarks, there are holes to plug in terms of exploitable behaviours that are locked into the model, but this too I'm confident they will find a general method of preventing The problem is, I don't think there is a "general method of [prevention]" because that's not how neural networks work. It's not easy to fix things like this because you can't just say "yeah just don't do that dumb thing anymore", the network has to be re-trained to learn the exploit. The way DeepMind tried to get around this is by having a league of AIs playing against each other which try to exploit each other and expose their weaknesses. It worked pretty damn well, but people still found ways to exploit the AI.
- hntrader 6y ago
  Isn't the general method of prevention just to train a bigger model for longer, so that all these niche edge case exploits get found and addressed during policy exploration? If there's an exploit that's sufficiently rare and unpredictable, then that seems like the only way (and indeed it should be a sufficient way, if done right) to address it.
  - ZephyrBlu 6y ago
    That is the obvious answer, but I have no idea if it's true in practice.
    - hntrader 6y ago
      It worked in adversarial board games (Go, Chess) and poker. We now have unexploitable bots for these games. It hasn't worked yet in Starcraft because the strategy space is so much larger and the action space is also much larger. The networks are too small relative to this space, and humans can still put the bot into a situation it can't handle. I'm going to guess that Starcraft will end up like these other games once the hardware etc advances another 5-10 years, and we'll have an unexploitable bot. The main reason I'm thinking this is we have unlimited training data, unlike with self-driving. We can make the models arbitrarily good. The bot still won't have an ounce of common sense beyond what it's trained to do. It's just that it will have been so exhaustively exposed to every nook of the search space that a human won't be able to find any exploits.
TaupeRanger 6y ago
Not at all. It is a computer, of course it beats humans at optimization problems and speed. Not remotely surprising or interesting, any more than a calculator doing arithmetic faster than a human.
- hntrader 6y ago
  This is not accurate. AlphaStar was nerfed quite heavily to achieve near APM parity with humans. The early version that beat Liquid-TLO had superhuman spikes in APM (despite having the same mean APM) but they addressed that later. The bot's APM is now significantly less than the best humans' typical APM, which makes it roughly fair since the bot never misclicks. AlphaStar is legitimately good at strategic reasoning, strategic planning, responsive build orders, responsive scouting, long macro games, etc. It's not the best in the world at these things yet (only top 1 percent+ level) and it does still rely on cheese build orders a lot, but still, what has been achieved is incredible.
  - TulliusCicero 6y ago
    > The bot's APM is now significantly less than the best humans' typical APM, which makes it roughly fair since the bot never misclicks. This isn't really true, it still has a heavy mechanical advantage. I'd ballpark it at low or mid masters for overall strategy and tactics, top tier for 'reading' an incoming fight and deciding whether or not it should take the fight, and superhuman on mechanics/control.
    - hntrader 6y ago
      The in-game EPM ended up about 180 with spikes smaller than humans' spikes. It also had camera constraints on where it could click unlike the bot that Liquid-TLO faced. Serral has a 300+ EPM for comparison. The bot's EPM is quite a bit worse than top humans, so I'm going to conclude that its mechanics are worse too. Also I don't agree that it was low masters for strats or tactics. You can't beat GMs with worse/same mechanics unless you have good strats. Besides, low masters players are pretty bad and the replays show that this bot had super tight and highly optimized builds, it was a big fan and good at early to midgame cheeses. The main valid criticism that I remember is that it wasn't the best at long games when the utility of its sharp build wore off and it needed to think on its feet, but I don't recall the details.
FartyMcFarter 6y ago
I don't think either system was trained from pixels, but I agree it's impressive nonetheless. However, it doesn't follow that it's easy to extend to beating top players consistently. If that was the case, it probably would have been done.

ZephyrBlu 6y ago

> As another commenter remarks, there are holes to plug in terms of exploitable behaviours that are locked into the model, but this too I'm confident they will find a general method of preventing The problem is, I don't think there is a "general method of [prevention]" because that's not how neural networks work. It's not easy to fix things like this because you can't just say "yeah just don't do that dumb thing anymore", the network has to be re-trained to learn the exploit. The way DeepMind tried to get around this is by having a league of AIs playing against each other which try to exploit each other and expose their weaknesses. It worked pretty damn well, but people still found ways to exploit the AI.

hntrader 6y ago
Isn't the general method of prevention just to train a bigger model for longer, so that all these niche edge case exploits get found and addressed during policy exploration? If there's an exploit that's sufficiently rare and unpredictable, then that seems like the only way (and indeed it should be a sufficient way, if done right) to address it.
- ZephyrBlu 6y ago
  That is the obvious answer, but I have no idea if it's true in practice.
  - hntrader 6y ago
    It worked in adversarial board games (Go, Chess) and poker. We now have unexploitable bots for these games. It hasn't worked yet in Starcraft because the strategy space is so much larger and the action space is also much larger. The networks are too small relative to this space, and humans can still put the bot into a situation it can't handle. I'm going to guess that Starcraft will end up like these other games once the hardware etc advances another 5-10 years, and we'll have an unexploitable bot. The main reason I'm thinking this is we have unlimited training data, unlike with self-driving. We can make the models arbitrarily good. The bot still won't have an ounce of common sense beyond what it's trained to do. It's just that it will have been so exhaustively exposed to every nook of the search space that a human won't be able to find any exploits.

TaupeRanger 6y ago

Not at all. It is a computer, of course it beats humans at optimization problems and speed. Not remotely surprising or interesting, any more than a calculator doing arithmetic faster than a human.

hntrader 6y ago
This is not accurate. AlphaStar was nerfed quite heavily to achieve near APM parity with humans. The early version that beat Liquid-TLO had superhuman spikes in APM (despite having the same mean APM) but they addressed that later. The bot's APM is now significantly less than the best humans' typical APM, which makes it roughly fair since the bot never misclicks. AlphaStar is legitimately good at strategic reasoning, strategic planning, responsive build orders, responsive scouting, long macro games, etc. It's not the best in the world at these things yet (only top 1 percent+ level) and it does still rely on cheese build orders a lot, but still, what has been achieved is incredible.
- TulliusCicero 6y ago
  > The bot's APM is now significantly less than the best humans' typical APM, which makes it roughly fair since the bot never misclicks. This isn't really true, it still has a heavy mechanical advantage. I'd ballpark it at low or mid masters for overall strategy and tactics, top tier for 'reading' an incoming fight and deciding whether or not it should take the fight, and superhuman on mechanics/control.
  - hntrader 6y ago
    The in-game EPM ended up about 180 with spikes smaller than humans' spikes. It also had camera constraints on where it could click unlike the bot that Liquid-TLO faced. Serral has a 300+ EPM for comparison. The bot's EPM is quite a bit worse than top humans, so I'm going to conclude that its mechanics are worse too. Also I don't agree that it was low masters for strats or tactics. You can't beat GMs with worse/same mechanics unless you have good strats. Besides, low masters players are pretty bad and the replays show that this bot had super tight and highly optimized builds, it was a big fan and good at early to midgame cheeses. The main valid criticism that I remember is that it wasn't the best at long games when the utility of its sharp build wore off and it needed to think on its feet, but I don't recall the details.

FartyMcFarter 6y ago

I don't think either system was trained from pixels, but I agree it's impressive nonetheless. However, it doesn't follow that it's easy to extend to beating top players consistently. If that was the case, it probably would have been done.