3 ms·

< OpenAI's DoTA 2 system wasn't playing the full game. I think the final version could play 17 of the 117 heroes, and the opposing human players were also restr

< OpenAI's DoTA 2 system wasn't playing the full game. I think the final version could play 17 of the 117 heroes, and the opposing human players were also restricted to playing this subset of the game. The bigger issue in my eyes was that while OpenAI 5 defeated the world champion team OG, when they let anyone in the world fight it, some ingenious players figured out a pretty robust method to consistently exploit and defeat the bot. As I haven't heard any buzz about OpenAI 5 since then, I think it was more or less unsuccessful unless they can show that their training method produces unexploitable bots (instead of bots that are really good against certain strategies)

Gunax 6y ago
They can train the bots based on on those games though, right? Seems more like a flaw in the training data than the principle. I am not sure if the training is done live or not--that is does the algorithm learn based on each game against a real, live player? Or do they just train the model offline, then allow players to play against the static model?
- ZephyrBlu 6y ago
  Assuming the OpenAI model is similar to DeepMind's AlphaStar model, the model is static. And a few games of being exploited is nowhere near enough data for the AI to be re-trained.
- CyberRage 6y ago
  Training requires millions of games. playing against humans is only for evaluation purposes, not for training. In both cases, it was indeed a static model but more recent work which is called MuZero is not static and achieves great results in board games and atari.
- kevinwang 6y ago
  > They can train the bots based on on those games though, right? Seems more like a flaw in the training data than the principle. I guess you could phrase it that way, but that's essentially the problem statement for developing a strategy for an imperfect-information game. So I would say it is a flaw in the principle if their final output is exploitable.
- jrumbut 6y ago
  Sibling posters have pointed out technical issues but I'd like to point out that while chasing after each successive exploit might let them stay on top of the Starcraft rankings, it changes the accomplishment from "we made a model that understands Starcraft better than almost any human" to "we made a model that memorized a selection of very good strategies and tactics." When it was first demonstrated, it really looked like it was doing very smart things (while also taking advantage of the fact that it doesn't have attention lapses and hand fatigue) and reacting well to different strategies on a level that was freaky to me.
  - _carbyau_ 6y ago
    This is why I stick to the definition some other HN poster provided: Cognitive Automation. But still really damn handy!
  - codyb 6y ago
    Learning through encounter seems to parallel a lot of human learning though. The question is, can it then apply partial applications of learned techniques to create unique offenses and defenses in novel situations I think.
  - pottertheotter 6y ago
    This doesn't seem any different than what humans do. Whenever I've played games everyone is usually along some progression of memorizing a selection of very good strategies and tactics. Then someone figures out a new tactic that exploits a weakness. Then people start to learn from that and figure out ways to protect against it. Rinse and repeat.
    - ghostbrainalpha 6y ago
      I always wanted to see if the AI could get to the point where it developed a novel strategy, that then changed the meta. Something humans hadn't thought of yet, but were also psychically capable of doing. Too me that will be the true moment that the AI has really surpassed us.
- Aerroon 6y ago
  The problem is how quickly the AI learns. When a player gets caught off guard by a strategy in one match, they will already react to it in the very next match. The AI we have does not. Also, did the StarCraft AI have to move the camera around? I remember watching the show matches. In those the AI lost the match that it had to handle the camera and couldn't just give impossible orders that a player's interface would not allow.

Gunax 6y ago

They can train the bots based on on those games though, right? Seems more like a flaw in the training data than the principle. I am not sure if the training is done live or not--that is does the algorithm learn based on each game against a real, live player? Or do they just train the model offline, then allow players to play against the static model?

ZephyrBlu 6y ago
Assuming the OpenAI model is similar to DeepMind's AlphaStar model, the model is static. And a few games of being exploited is nowhere near enough data for the AI to be re-trained.
CyberRage 6y ago
Training requires millions of games. playing against humans is only for evaluation purposes, not for training. In both cases, it was indeed a static model but more recent work which is called MuZero is not static and achieves great results in board games and atari.
kevinwang 6y ago
> They can train the bots based on on those games though, right? Seems more like a flaw in the training data than the principle. I guess you could phrase it that way, but that's essentially the problem statement for developing a strategy for an imperfect-information game. So I would say it is a flaw in the principle if their final output is exploitable.
jrumbut 6y ago
Sibling posters have pointed out technical issues but I'd like to point out that while chasing after each successive exploit might let them stay on top of the Starcraft rankings, it changes the accomplishment from "we made a model that understands Starcraft better than almost any human" to "we made a model that memorized a selection of very good strategies and tactics." When it was first demonstrated, it really looked like it was doing very smart things (while also taking advantage of the fact that it doesn't have attention lapses and hand fatigue) and reacting well to different strategies on a level that was freaky to me.
- _carbyau_ 6y ago
  This is why I stick to the definition some other HN poster provided: Cognitive Automation. But still really damn handy!
- codyb 6y ago
  Learning through encounter seems to parallel a lot of human learning though. The question is, can it then apply partial applications of learned techniques to create unique offenses and defenses in novel situations I think.
- pottertheotter 6y ago
  This doesn't seem any different than what humans do. Whenever I've played games everyone is usually along some progression of memorizing a selection of very good strategies and tactics. Then someone figures out a new tactic that exploits a weakness. Then people start to learn from that and figure out ways to protect against it. Rinse and repeat.
  - ghostbrainalpha 6y ago
    I always wanted to see if the AI could get to the point where it developed a novel strategy, that then changed the meta. Something humans hadn't thought of yet, but were also psychically capable of doing. Too me that will be the true moment that the AI has really surpassed us.
Aerroon 6y ago
The problem is how quickly the AI learns. When a player gets caught off guard by a strategy in one match, they will already react to it in the very next match. The AI we have does not. Also, did the StarCraft AI have to move the camera around? I remember watching the show matches. In those the AI lost the match that it had to handle the camera and couldn't just give impossible orders that a player's interface would not allow.