5 ms·
An army of Stalkers should NEVER beat an army of Immortals. Stalkers have 14.4 DPS (vs Armored units) while Immortals have 48.1 DPS (vs Armored units). Both Im
by dragontamer 8y ago
An army of Stalkers should NEVER beat an army of Immortals.
Stalkers have 14.4 DPS (vs Armored units) while Immortals have 48.1 DPS (vs Armored units). Both Immortals and Stalkers are armored.
Immortals provide more than double DPS, with 200+100 HP / Shields. While Stalkers have only 80+80 HP / shields. Stalkers cost roughly 1/2 of an Immortal, but any "even match" would have roughly 2x Stalkers for every Immortal.
Immortals do roughly 3.5x more damage while having 2x the HP (Immortals have a bonus ability to gain +100 Shields as well). Even if outnumbered 2-to-1, Immortals shouldn't lose the fight.
The problem is that AlphaStar can watch all of its Stalkers's HP simultaneously, and blink them away from danger instantly. AlphaStar is playing a game that is fundamentally different than what Humans play. In AlphaStar's version of Starcraft, Stalkers win vs Immortals. All you gotta do is Blink away low-HP Stalkers to the back line, and then have the Stalkers regenerate their shields before bringing them back into battle.
Its inhuman. Maybe its intelligence, but its an exceptionally poor demonstration of intelligence.
---------------
The game that the AI played was not Starcraft, certainly not the game that humans play. Humans cannot physically click quickly enough to move stalkers in the way that AlphaStar did. The screen humans look at cannot show all of the HP of your individual units. The screen humans look at can only see some of the battlefield, not the entire battlefield.
That's the main issue with the demonstration. Sure, the APM rules were designed to prevent this situation from happening, but the demonstration proved that the APM-caps were insufficient at creating human-like playing conditions.
- T-hawk 8y agoWe should establish and recognize as a concept OPM, observations-per-minute. That's what the AI is doing in excess of what the human can, observing the game state faster, even if acting at the same APM.
- dragontamer 8y agoAPM Accuracy as a concept also needs to be revisited. A human who misclicks a "move" action, will quickly issue a 2nd or 3rd move to move the unit to the correct location. All together, it will count as 5 actions (1-action to click on a unit. 4-actions to move the unit around, because 3 of them were a mistake). This is seen with the 800 APM that TLO played with. He has a lot of spurious clicks and button clicks. The units are controlled well, but a lot of his actions are redundant. (There's no penalty for redundant actions, so why not??) A computer will not misclick. In fact, the AI doesn't even have a mouse: it can accurately click any location on the screen it wants without moving a cursor. Therefore, a computer will have 300-"perfect" APM (no misclicks involved), while a human will have more and more misclicks as their APM goes higher. --------- I dunno, I think going for Starcraft was a fundamental mistake. A huge amount of Starcraft practice is just moving the mouse around and developing finger-dexterity to support 100+ APM (at my level of Starcraft anyway. I never really got over 150 APM even at my best). The dexterity of your wrists and fingers play a huge part in the game. Any game where dexterity, reaction speed, and so forth are an element, computers will likely dominate.
- ALittleLight 8y agoThere is the concert of effective actions per minute or eapm. Presumably AlphaStar has a high eapm to apm ratio. https://liquipedia.net/starcraft/Actions_per_Minute#APM_vs._eAPM https://liquipedia.net/starcraft/Actions_per_Minute#APM_vs._...
- cwkoss 8y agoIt would be interesting if they made the AI harness add noise to the click position signal based on how long it has been since the last click. Ex. up to 5 perfectly targeted ClkPS, or up to 10 with 10px accuracy, or up to 15 with 30px accuracy
- jnwatson 8y agoThe main “cheat” that occurred is AlphaStar’s definition of APM. They capped APM averaged over an entire game. Essentially it saved its actions it didn’t need at the beginning of the game and unleashed a torrent of superhuman >1000 APM for a solid 30 seconds in that stalker battle. That should be accounted for in the next demo.
- RoboTeddy 8y agoThe averaging period doesn't last the whole game: "we set a maximum of 600 APMs over 5 second periods, 400 over 15 second periods, 320 over 30 second periods, and 300 over 60 second period. If the agent issues more actions in such periods, we drop / ignore the actions. These were values taken from human statistics." [1] I'd be curious about the approach they used to draw from human statistics. It's possible that peaks in the human APM distribution are caused by e.g. holding down keys, resulting in ~2000 APM [2]. [1] https://www.reddit.com/r/MachineLearning/comments/ajgzoc/we_are_oriol_vinyals_and_david_silver_from/eexs0pd/ https://www.reddit.com/r/MachineLearning/comments/ajgzoc/we_... [2] https://blog.usejournal.com/an-analysis-on-how-deepminds-starcraft-2-ai-s-superhuman-speed-could-be-a-band-aid-fix-for-the-1702fb8344d6 https://blog.usejournal.com/an-analysis-on-how-deepminds-sta...
- setr 8y agoIts also very unlikely that an AI click is equivalent to a human click — humans will spamclick to ensure the action goes through, while the machine can do it once with full confidence
- Supermancho 8y ago> An army of Stalkers should NEVER beat an army of Immortals. That happens. Not often, but it happens. The size of the armies is the issue. They happened to both be large and the better your ability to track life and micro, the better your chances at winning with stalkers. There are replays with 5v2 immortal fights that they lose, right? I've seen blinkstalkers blink out between the immortal Rate of Fire recovery plenty...ok. It just happens the AI showcased the issue in a larger fight. The game is still starcraft, it's just there's an obvious issue that needs to be worked on within the game (immortal blasts would disable special abilities from being toggled/used in a tiny aoe, is the obvious fix). The hyperbole about it not being Starcraft is just wrong. Micro is part of starcraft.
- dragontamer 8y ago> The game is still starcraft, it's just there's an obvious issue that needs to be worked on within the game (immortal blasts would disable special abilities from being toggled/used in a tiny aoe, is the obvious fix). Starcraft II wasn't designed to play at this level. The 4 vs 2 trick is cute, but no human can perform blink-stalker micro on a mass scale like Match4 AlphaStar. https://youtu.be/L0FvTQ3W-wQ?t=633 https://youtu.be/L0FvTQ3W-wQ?t=633 No human would be able to perform the Blink-stalker strategy between 3 groups of Stalkers across the map, like what goes on over there. Mana's Immortal + Zealot army base is clearly superior from a composition perspective. Against any human, Mana's Immortals+Zealots would have won. Period. Its not just the "Blink-Stalkers", but notice how AlphaStar's Stalkers explicitly avoid attacking Zealots and focus fire on Immortals instead. That entire section is clearly super-human levels of micro. Yeah, humans can perform these feats at like 4 vs 2 situations, or small scale experiments. But no pro can do it at the scale or accuracy that AlphaStar did. Seriously, watch the match. Its super-human micro to a degree I've never seen before. ---------- Look, I'm not saying it was a complete failure. I recognize that the AlphaStar team went to great lengths to implement APM-caps and reaction-speed caps (iirc: the entire game was being played at 350ms of latency from AlphaStar's perspective, so it wouldn't have super-human reaction speeds). But these caps were insufficient. The AlphaStar team needs to develop new metrics to make the AI play more "strategic" rather than brute-forcing micro like it does here. I dunno, maybe AlphaStar needs to be capped at 100apm and have 500ms of latency instead. Or maybe new metrics ("effective APM") or APM-burst metrics need to be developed. Its an issue the AlphaStar team needs to work around for sure. As it is, AlphaStar is playing with superhuman micro, even with the APM and Latency caps.
- Rallerbabs 8y agoThe goal was to have an AI defeat humans at SC2. Mission accomplished. Was it fair when AlphaStar tried getting at an air unit with all its land units and pissed away a lot of time doing so, while Mana proceeded to obliterate its base? What if a future AI solves cancer from scratch in a day? Are we going to find reasons to call this 'AI vs Mother Nature' match unfair? Silly hoomans don't need to look for reasons why AlphaStar was playing unfair. It's just a game. And machines will be better at it. They'll be better at everything. Make your peace with that. DeepMind has only just gotten started. Like with Go. AlphaGo was trained for years. AlphaZero was trained for days. The latter obliterated the former. DeepMind is reading this criticism, don't you know. What do you think AlphaStarZero will be doing in a few years time? Crushing AlphaStar 100 out of 100 times, with low APM. That's what.
- dragontamer 8y ago> The goal was to have an AI defeat humans at SC2. Hardly. They put APM caps in there for a reason. We all know that a real research team going all out with 100,000+ APM would have zerglings dodging tanks and other super-human shennanigans going on. But that's simply not interesting. Nominally, AlphaStar set a 300 APM cap on the match. Apparently its a 300 APM average across the match however, which means the cap wasn't really effective at preventing "superhuman" conditions. Hopefully the next demo will better address that shortcoming and lead to more interesting matches. I think the main lesson of the demo is that "APM" is a poor metric for what they're going for. A new metric needs to be developed. Other posters downthread were talking about "eAPM" (Effective APM). A formalized version of the concept may help.
- Rallerbabs 8y ago>>The goal was to have an AI defeat humans at SC2. >Hardly. After cracking Go, DeepMind stated that it was going after SC2. This is recorded history, so don't even think about deluding yourself into thinking otherwise.
- muh_gradle 8y agoStop it. No one is saying anything that you're saying. People are just disappointed because they wanted superior strategy and relatively equal game mechanics. We saw strategy to a certain degree, but it was spoiled by the lack of the latter.
- justfor1comment 8y agoI also agree with these observations. The AI essentially uncovered a bug is Starcraft. Mass rushing Stalkers with carefully calculated blinks can beat any opponent strategy up to a certain time in the game. I think a patch that removes Stalker blink would fix this bug and force the AI to try something else.