10 ms·

In the mass stalker battles, the AI APM exceeded 1000 a few times, and no doubt that most of that was precisely targeted. Whereas a human doing 500 APM micro is

In the mass stalker battles, the AI APM exceeded 1000 a few times, and no doubt that most of that was precisely targeted. Whereas a human doing 500 APM micro is obviously going to be far more imprecise. I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased.

simmanian 8y ago
> artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased I like the idea of having action noise that's linearly related to APM
wnevets 8y ago
>I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased. IIRC OpenAI limits the reaction time to ~200ms when playing DoTA2. AI employing better strategies than humans will always be more interesting than AI that can out click humans.
- jgon 8y ago
  Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a superhuman level. Even with that advantage in fights, the human team still demolished the AI. Oh well, lots of things to learn still.
  - lrem 8y ago
    I've recently watched a TED talk explaining how human perception has a lag of about a third of a second. Pro players might be better, but after noticing they also need to take an action.
    - neltnerb 8y ago
      My experience is that to beat 300ms requires there to be no conscious thought in the loop. It has to be muscle memory guided by higher level intent. It's like how the gunslinger waiting to shoot hits first, it's reflex instead of decision.
    - wnevets 8y ago
      Getting sub 200ms on something like this benchmark is fairly easy [1]. While waiting for the color to change is different than processing a game like dota2 or sc2 a 200ms limit isn't too unreasonable to me. I would love to see these AIs get handicapped even more like a full second and really force them to out think humans. [1] https://www.humanbenchmark.com/tests/reactiontime https://www.humanbenchmark.com/tests/reactiontime
  - dx87 8y ago
    Another advantage was that the AI is just reading the game state through an API, it doesn't have to look on the screen. The game can be difficult to watch from a pro's perspective since they have to constantly click around the map to see what's happening, but the AI has perfect knowledge of everything it is capable of seeing, all without having to physically move a mouse to click on the screen.
    - thomasfortes 8y ago
      Yeah, put an eye tracker in a pro and you'll see that the eyes are constantly changing the focus point, if you can watch the entire scene with the same precision without the need to focus on it you're already at a nice advantage. As an aside, a few pro gamers prefer to play on windowed mode for exactly this reason.
      - bitxbitxbitcoin 8y ago
        Just supporting this. I remember the uproar when 800x600 resolution was removed as an option in SC2 around 2012[1]. [1] https://eu.battle.net/forums/en/sc2/topic/6201040181 https://eu.battle.net/forums/en/sc2/topic/6201040181
        rootlocus 8y ago
        > the uproar I'm not saying you're wrong, but 6 posts with no profanity is hardly "uproar" by blizzard forums standards.
    - theflork 8y ago
      Is the bit about reading the game through an api true? Earlier iterations of this same rl based agent that played Atari games would read just raw pixels not an api.
      - setr 8y ago
        Any game that is specifically going out of its way to support these ai’s will naturally do it through an api, though I’m only aware of dota2 and sc2 (sc:bw also does, through a community-modified client that serves the api, iirc). For adhoc games, eg atari, pixel-parsing is the natural result, but no one would intentionally set it up like that
  - asdfasgasdgasdg 8y ago
    The game is difficult to watch, but does anyone honestly believe that an AI is going to have a difficult time parsing the scene if it is trained to do so? That to me just seems like a question of resources. We're pretty good at image recognition and segmentation now, and that's without the unlimited amounts of training data one could generate when using a controlled game environment with a limited range of possible animations and effects. This is why I find the prospect of the AI agent having to parse the screen entirely uninteresting.
    - thomasfortes 8y ago
      But on the other hand, an AI that beats humans using brute force in a game where it makes a ton of difference isn't much fair too.
      - candiodari 8y ago
        The funny thing is that once we're talking about the real world, which will come, that incentive actually reverses. At that point the name of the game will be maximizing the advantage the body/infrastructure provides the AI, not minimizing it. Weird.
        gowld 8y ago
        It's the same principle as a baseball player putting extra weights on the bat in practice.
        buchanan 8y ago
        Yes, since DeepMind chose SC2 for having the right characteristics for mapping to the real world, ie imperfect information and real time response, they should have had at least one run without any speed governors. And maybe another with the CPU limited to some level we might find in an embedded system of near future.
      - red75prime 8y ago
        > using brute force "Brute force" in AI context is usually reserved for traversal of the entire search space. I think "superhuman micromanagement" is a better term. And before AlphaStar superhuman micro wasn't insurmountable obstacle for human players.
  - gsich 8y ago
    I think OpenAI would have been by lots of humans, but they decided to train it with 5 unlimited, invulnerable couriers. (until the TI showmatches, in which they were beaten easily.)
- dustycowboy 8y ago
  The only way to truly have a fair fight would be to accurately model the limits of human capacities. How fast can humans move the mouse and at what accuracy? How fast can they type keyboard commands? How fast can they move their eyes? You could study those limits in a sports lab with high speed cameras, etc. A simpler model would be to limit the bot to, say, one action per 250ms, introduce a slight delay in his reaction time, require him to move the camera to gain detailed information and take further actions, and have camera movements count as actions.
pmontra 8y ago
I understand the spirit of the proposal but that would be like limiting a computer to add at most two numbers per second. It's OK if we want an interesting contest against humans but it wouldn't be a fair estimate of a computer math capability. It's also not the point of using computers to do math instead of a room full of accountants. I'm OK with the AI going as fast as it can and play superhuman strategies because it can be that fast. After all we'll not limit AIs output rate when we'll let them manage a country's power grid.
- sl1ck731 8y ago
  The purpose of limiting speed isn't to make an interesting contest, it is to accurately compare the "math" instead of the speed the math is done at. It isn't surprising that its fast, the surprising part is that it can make human-like decisions. The only way to compare whether its thinking is human-like is to restrain it from "brute forcing" the contest through speed. The model has likely learned that the faster it does things the better the outcome. What it needs to be measured on is strategy.
  - freeflight 8y ago
    But isn't the competency of a Starcraft player is also measured on his/her speed? In that context, you can't really measure strategy without accounting for timing/speed because a lot of tactics and strategies only become viable once the player has the required speed to actually realize them aka "micro".
    - arcticfox 8y ago
      Speed is only interesting as part of fair human competition. It's trivial for the AI to win with speed and it doesn't have to be remotely smart about it. Serral (dominant world #1) was easily beat by 3 far weaker humans controlling one opponent - it wasn't even close. It's just stupid to even claim victory in those situations. Making an AI that wins by outsmarting humans, on the other hand, is what we are all interested in.
    - BigJono 8y ago
sayno3 8y ago
If you are going to go to this extent, why not limit the computer by making it control a meat-puppet that has to physically manipulate the mouse and keyboard. And also give the computer carpal tunnel syndrome and get distracted by its questionable past. For the people complaining about the information availability of 'one screen at a time': what is the difference between seeing the whole map with fog of war and quickly iterating the over the minimap at a very fast rate? Answer: a few milliseconds which would be irrelevant to a human anyway. Are they trying to make a computer that plays sc2 well or are they trying to make a computer that simulates expert human play to an indistinguishable level and which of those is more broadly applicable to other AI problems?
taneq 8y ago
How many of those 500 actions are actually useful? I haven't watched competitive StarCraft games for years but back when I did, rates were more like 300APM and even then the players basically spam clicked the background or selected random units non-stop and were probably only doing 50-100 actual effective actions.
- arcticfox 8y ago
  > How many of those 500 actions are actually useful? Exactly, a human doing 500 APM during intense moments is going to be way different than an AI bursting 1000 APM with pixel-precision during the most crucial moment in a game. TLO spent a ton of time at >1000 APM and walked his army directly into enemy shots all the time. MaNa had much better control at ~400 APM. So APM is really irrelevant to control - for humans. I suspect the AI, on the other hand, makes each action precise & count for something. This graph, which I think was supposed to show that the AI was being "human", IMO is pretty damning. We saw the APM spike to >1000 during a critical moment and we saw the APM at <30 during lulls, so we know it uses its APM at important moments, presumably with important pixel-precise actions. https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/#image-34426 https://deepmind.com/blog/alphastar-mastering-real-time-stra...
  - CydeWeys 8y ago
    I suspect that once the AI becomes good enough it will be able to beat human players using a much lower total APM than human players. We're not quite there yet, but it just needs a little bit of time. As a hopefully illustrative comparison, you could give any top player a day of play time per move against the top Chess AI being given a minute of play time per move and the AI will still win. That's how much better the AIs are than humans now. There's no reason in principle this won't be possible with StarCraft AI too.
    - setr 8y ago
      The biggest issue with allowing the ai to have high APM is that it will inevitably learn optimal strategies that depend on that high APM, eg stalkers can take on far more immortals than we normally expect, and the AI will learn it this way, because the high APM allows a new stalker strategy (or rather, empowers an old one greatly) while not affecting immortals significantly. This also naturally means the AI leagues see a different game balance than the human leagues, leading to strategy divergence. And then when you drop the APM limit, suddenly all the learned optimal ai strategies start falling apart, and the whole thing has to be relearned. More annoyingly, there’s not much for human players to learn from innovative ai strategies that are based on inhuman accuracy of play (because we couldn’t possibly execute it).
pesmhey 8y ago
There would be an entire new dimension of decision making, in addition to good macro, where you have to prioritize actions. Will be interesting to see.
- posterboy 8y ago
  I said so before, but is it really a big difference from controlling a unit that can also only do one thing at a time? The agent controls itself just like another unit, with a constraint on APM available to control other units. On the one hand, these APMs add a new parameter, if the constraint is implemented naively. On the other hand, if there are viable strategies against ultrahigh APM opponents, then the constrained is really rather limiting the dimensions of the decision space and to good effect, finding viable strategies that take less effort. Hence such things are called "hyperparameters" (I think that's something different, but you know what I mean). Likewise, the game isn't as fast as to need 100 screen switches per second, if good planning allows batching and bursting actions.
a_wild_dandan 8y ago
Here's a graph of AlphaStar's APM versus a professional player's: https://i.imgur.com/TXeLkQK.png https://i.imgur.com/TXeLkQK.png Evidently AlphaStar also has a similar Economy of Attention (where the player focuses) to a professional player, at around 30 screens per minute. Additionally, AlphaStar's reaction time is around 350ms, a significant disadvantage over a pro. The skepticism in this thread is absolutely justified but I think it's important to note the lengths to which DeepMind has gone to address and assuage the fears of superhuman mechanical skills being employed in these games.
- gowld 8y ago
  How can it do 1000 APM if if its reaction time is 350ms? (180/minute)
  - gamegoblin 8y ago
    I believe you are conflating latency and throughput. It might take AlphaStar 350ms to perceive a threat, but once perceived, it might issue many commands at high speed to respond.
  - aiiane 8y ago
    Latency != Bandwidth
- gamegoblin 8y ago
  I watched all of the event live and I feel that that graph is deceptive. If a game is 15 minutes and has 3 main battles lasting 15 seconds each, and you use 100 average APM on non-battle time and 1000 APM during battles, your average APM will be 145 but you obviously have a superhuman advantage. This is compounded by the fact that almost all of AlphaStar’s actions are “useful” whereas a significant amount of the human actions are spammy. You will typically see a human select a group of units, and fast-click a location in the general direction they want the units to move (to get them started moving that way), and then keep clicking to continuously update the destination to a more precise location. Every click counts as an action. An AI can be perfectly precise and “clicks” the right place the first time.

simmanian 8y ago

> artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased I like the idea of having action noise that's linearly related to APM

wnevets 8y ago

>I think a far more interesting limitation would be to cap APM at 150 or so, or to artificially limit action precision with some sort of virtual mouse that reduced accuracy as APM increased. IIRC OpenAI limits the reaction time to ~200ms when playing DoTA2. AI employing better strategies than humans will always be more interesting than AI that can out click humans.

jgon 8y ago
Even the 200ms reaction time seemed overly slanted towards the AI. I don't think that is the actual reaction time of top pros, in the matches the AI played the human player would teleport in from complete invisibility and try to use an instant cast spell and the AI would have already teleported out. Yes the theoretically may have been constrained to a 200ms reaction time, but in practice the AI was playing at a superhuman level. Even with that advantage in fights, the human team still demolished the AI. Oh well, lots of things to learn still.
- lrem 8y ago
  I've recently watched a TED talk explaining how human perception has a lag of about a third of a second. Pro players might be better, but after noticing they also need to take an action.
  - neltnerb 8y ago
    My experience is that to beat 300ms requires there to be no conscious thought in the loop. It has to be muscle memory guided by higher level intent. It's like how the gunslinger waiting to shoot hits first, it's reflex instead of decision.
  - wnevets 8y ago
    Getting sub 200ms on something like this benchmark is fairly easy [1]. While waiting for the color to change is different than processing a game like dota2 or sc2 a 200ms limit isn't too unreasonable to me. I would love to see these AIs get handicapped even more like a full second and really force them to out think humans. [1] https://www.humanbenchmark.com/tests/reactiontime https://www.humanbenchmark.com/tests/reactiontime
- dx87 8y ago
  Another advantage was that the AI is just reading the game state through an API, it doesn't have to look on the screen. The game can be difficult to watch from a pro's perspective since they have to constantly click around the map to see what's happening, but the AI has perfect knowledge of everything it is capable of seeing, all without having to physically move a mouse to click on the screen.
  - thomasfortes 8y ago
    Yeah, put an eye tracker in a pro and you'll see that the eyes are constantly changing the focus point, if you can watch the entire scene with the same precision without the need to focus on it you're already at a nice advantage. As an aside, a few pro gamers prefer to play on windowed mode for exactly this reason.
    - bitxbitxbitcoin 8y ago
      Just supporting this. I remember the uproar when 800x600 resolution was removed as an option in SC2 around 2012[1]. [1] https://eu.battle.net/forums/en/sc2/topic/6201040181 https://eu.battle.net/forums/en/sc2/topic/6201040181
      - rootlocus 8y ago
        > the uproar I'm not saying you're wrong, but 6 posts with no profanity is hardly "uproar" by blizzard forums standards.
  - theflork 8y ago
    Is the bit about reading the game through an api true? Earlier iterations of this same rl based agent that played Atari games would read just raw pixels not an api.
    - setr 8y ago
      Any game that is specifically going out of its way to support these ai’s will naturally do it through an api, though I’m only aware of dota2 and sc2 (sc:bw also does, through a community-modified client that serves the api, iirc). For adhoc games, eg atari, pixel-parsing is the natural result, but no one would intentionally set it up like that
- asdfasgasdgasdg 8y ago
  The game is difficult to watch, but does anyone honestly believe that an AI is going to have a difficult time parsing the scene if it is trained to do so? That to me just seems like a question of resources. We're pretty good at image recognition and segmentation now, and that's without the unlimited amounts of training data one could generate when using a controlled game environment with a limited range of possible animations and effects. This is why I find the prospect of the AI agent having to parse the screen entirely uninteresting.
  - thomasfortes 8y ago
    But on the other hand, an AI that beats humans using brute force in a game where it makes a ton of difference isn't much fair too.
    - candiodari 8y ago
      The funny thing is that once we're talking about the real world, which will come, that incentive actually reverses. At that point the name of the game will be maximizing the advantage the body/infrastructure provides the AI, not minimizing it. Weird.
      - gowld 8y ago
        It's the same principle as a baseball player putting extra weights on the bat in practice.
      - buchanan 8y ago
        Yes, since DeepMind chose SC2 for having the right characteristics for mapping to the real world, ie imperfect information and real time response, they should have had at least one run without any speed governors. And maybe another with the CPU limited to some level we might find in an embedded system of near future.
    - red75prime 8y ago
      > using brute force "Brute force" in AI context is usually reserved for traversal of the entire search space. I think "superhuman micromanagement" is a better term. And before AlphaStar superhuman micro wasn't insurmountable obstacle for human players.
- gsich 8y ago
  I think OpenAI would have been by lots of humans, but they decided to train it with 5 unlimited, invulnerable couriers. (until the TI showmatches, in which they were beaten easily.)
dustycowboy 8y ago
The only way to truly have a fair fight would be to accurately model the limits of human capacities. How fast can humans move the mouse and at what accuracy? How fast can they type keyboard commands? How fast can they move their eyes? You could study those limits in a sports lab with high speed cameras, etc. A simpler model would be to limit the bot to, say, one action per 250ms, introduce a slight delay in his reaction time, require him to move the camera to gain detailed information and take further actions, and have camera movements count as actions.

pmontra 8y ago

I understand the spirit of the proposal but that would be like limiting a computer to add at most two numbers per second. It's OK if we want an interesting contest against humans but it wouldn't be a fair estimate of a computer math capability. It's also not the point of using computers to do math instead of a room full of accountants. I'm OK with the AI going as fast as it can and play superhuman strategies because it can be that fast. After all we'll not limit AIs output rate when we'll let them manage a country's power grid.

sl1ck731 8y ago
The purpose of limiting speed isn't to make an interesting contest, it is to accurately compare the "math" instead of the speed the math is done at. It isn't surprising that its fast, the surprising part is that it can make human-like decisions. The only way to compare whether its thinking is human-like is to restrain it from "brute forcing" the contest through speed. The model has likely learned that the faster it does things the better the outcome. What it needs to be measured on is strategy.
- freeflight 8y ago
  But isn't the competency of a Starcraft player is also measured on his/her speed? In that context, you can't really measure strategy without accounting for timing/speed because a lot of tactics and strategies only become viable once the player has the required speed to actually realize them aka "micro".
  - arcticfox 8y ago
    Speed is only interesting as part of fair human competition. It's trivial for the AI to win with speed and it doesn't have to be remotely smart about it. Serral (dominant world #1) was easily beat by 3 far weaker humans controlling one opponent - it wasn't even close. It's just stupid to even claim victory in those situations. Making an AI that wins by outsmarting humans, on the other hand, is what we are all interested in.
  - BigJono 8y ago

sayno3 8y ago

If you are going to go to this extent, why not limit the computer by making it control a meat-puppet that has to physically manipulate the mouse and keyboard. And also give the computer carpal tunnel syndrome and get distracted by its questionable past. For the people complaining about the information availability of 'one screen at a time': what is the difference between seeing the whole map with fog of war and quickly iterating the over the minimap at a very fast rate? Answer: a few milliseconds which would be irrelevant to a human anyway. Are they trying to make a computer that plays sc2 well or are they trying to make a computer that simulates expert human play to an indistinguishable level and which of those is more broadly applicable to other AI problems?

taneq 8y ago

How many of those 500 actions are actually useful? I haven't watched competitive StarCraft games for years but back when I did, rates were more like 300APM and even then the players basically spam clicked the background or selected random units non-stop and were probably only doing 50-100 actual effective actions.

arcticfox 8y ago
> How many of those 500 actions are actually useful? Exactly, a human doing 500 APM during intense moments is going to be way different than an AI bursting 1000 APM with pixel-precision during the most crucial moment in a game. TLO spent a ton of time at >1000 APM and walked his army directly into enemy shots all the time. MaNa had much better control at ~400 APM. So APM is really irrelevant to control - for humans. I suspect the AI, on the other hand, makes each action precise & count for something. This graph, which I think was supposed to show that the AI was being "human", IMO is pretty damning. We saw the APM spike to >1000 during a critical moment and we saw the APM at <30 during lulls, so we know it uses its APM at important moments, presumably with important pixel-precise actions. https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/#image-34426 https://deepmind.com/blog/alphastar-mastering-real-time-stra...
- CydeWeys 8y ago
  I suspect that once the AI becomes good enough it will be able to beat human players using a much lower total APM than human players. We're not quite there yet, but it just needs a little bit of time. As a hopefully illustrative comparison, you could give any top player a day of play time per move against the top Chess AI being given a minute of play time per move and the AI will still win. That's how much better the AIs are than humans now. There's no reason in principle this won't be possible with StarCraft AI too.
  - setr 8y ago
    The biggest issue with allowing the ai to have high APM is that it will inevitably learn optimal strategies that depend on that high APM, eg stalkers can take on far more immortals than we normally expect, and the AI will learn it this way, because the high APM allows a new stalker strategy (or rather, empowers an old one greatly) while not affecting immortals significantly. This also naturally means the AI leagues see a different game balance than the human leagues, leading to strategy divergence. And then when you drop the APM limit, suddenly all the learned optimal ai strategies start falling apart, and the whole thing has to be relearned. More annoyingly, there’s not much for human players to learn from innovative ai strategies that are based on inhuman accuracy of play (because we couldn’t possibly execute it).

pesmhey 8y ago

There would be an entire new dimension of decision making, in addition to good macro, where you have to prioritize actions. Will be interesting to see.

posterboy 8y ago
I said so before, but is it really a big difference from controlling a unit that can also only do one thing at a time? The agent controls itself just like another unit, with a constraint on APM available to control other units. On the one hand, these APMs add a new parameter, if the constraint is implemented naively. On the other hand, if there are viable strategies against ultrahigh APM opponents, then the constrained is really rather limiting the dimensions of the decision space and to good effect, finding viable strategies that take less effort. Hence such things are called "hyperparameters" (I think that's something different, but you know what I mean). Likewise, the game isn't as fast as to need 100 screen switches per second, if good planning allows batching and bursting actions.

a_wild_dandan 8y ago

Here's a graph of AlphaStar's APM versus a professional player's: https://i.imgur.com/TXeLkQK.png https://i.imgur.com/TXeLkQK.png Evidently AlphaStar also has a similar Economy of Attention (where the player focuses) to a professional player, at around 30 screens per minute. Additionally, AlphaStar's reaction time is around 350ms, a significant disadvantage over a pro. The skepticism in this thread is absolutely justified but I think it's important to note the lengths to which DeepMind has gone to address and assuage the fears of superhuman mechanical skills being employed in these games.

gowld 8y ago
How can it do 1000 APM if if its reaction time is 350ms? (180/minute)
- gamegoblin 8y ago
  I believe you are conflating latency and throughput. It might take AlphaStar 350ms to perceive a threat, but once perceived, it might issue many commands at high speed to respond.
- aiiane 8y ago
  Latency != Bandwidth
gamegoblin 8y ago
I watched all of the event live and I feel that that graph is deceptive. If a game is 15 minutes and has 3 main battles lasting 15 seconds each, and you use 100 average APM on non-battle time and 1000 APM during battles, your average APM will be 145 but you obviously have a superhuman advantage. This is compounded by the fact that almost all of AlphaStar’s actions are “useful” whereas a significant amount of the human actions are spammy. You will typically see a human select a group of units, and fast-click a location in the general direction they want the units to move (to get them started moving that way), and then keep clicking to continuously update the destination to a more precise location. Every click counts as an action. An AI can be perfectly precise and “clicks” the right place the first time.