4 ms·

>it is obvious that it would do just as well at the full game excepting that OpenAI didn't have the stomach to try and tackle the engineering challenge for no r

>it is obvious that it would do just as well at the full game excepting that OpenAI didn't have the stomach to try and tackle the engineering challenge for no reward. That's an interesting interpretation. My interpretation is that they simply weren't able to do it yet (not for a lack of trying though), and so they gave up. It's not obvious at all to me that there is a way to simply scale their approach to the real game. I mean, certainly there is a way with infinite computing power, but it's not clear to me that even with all the computing power on earth currently this approach would work. 18 out of 117 heroes does reduce complexity quite a lot, especially when you pick ones with less complex mechanics. >They weren't even trying to abuse the reaction time of the computer They introduced a constant reaction time, but they still "abused" the interface they are provided. If this was a task in the real world, the AI would have the same interface as humans: The screen. They would have to move the screen and run object recognition algorithms on the rendered values, and they would have to "click" on units (not physically) instead of being able to internally select them. This would introduce completely different (and more realistic) reaction time parameters, because if a hero starts an animation it might not be immediately clear which animation it is in the first 5 frames, but the API will provide the value immediately. AlphaStar's strength dropped considerably for example when they realized that the AI being able to perceive the entire map at once was an unfair advantage and restricted it so that it had to move the camera around. But still even AlphaStar is using an API instead of just the rendered output, which enables things like selecting units that are hidden behind other units, which for a human player is literally impossible without moving it out of the way first. I think for AI to be able to claim that it is better at a game than humans, the AI ought to be playing the actual game, and use the same interface. (Not necessarily physically, you don't need a camera and a robot, but the AI's input should be the rendered output of the game, and the AI's output should be mouse movement / key strokes.) You might think that would be a trivial and useless exercise (like it would be in Chess/Go), but I'm really not convinced of that for video games. There are a lot of examples where even playing a frame-by-frame recording of a fight you just can't tell what exactly is going on in a fight until a couple frames later, but the API will always provide floating-point accurate data of everything. This is a huge advantage that just doesn't translate into real world tasks when the AI has to use cameras and sensors to make sense of the world (in automated driving for example) just like we have to.

roenxi 7y ago
> 18 out of 117 heroes does reduce complexity quite a lot, especially when you pick ones with less complex mechanics. Sure; but you can train one agent/hero so the training scales in a somewhat linear fashion once they get beyond ~15-20 heros. The computer won't be able to nail the full complexity of the game - but neither can a human, and the evidence from Go is the amount of the game a computer can grasp is much greater than a human. Humans can't fully understand full DotA either; it is common for pros to be surprised by mechanical interactions or quirks on specific heros. It might be an interesting theoretical challenge, but there would need to be some solid evidence of that beyond OpenAI calling it done and going home. Especially given the breakneck speed that the first 17 heros got trained. The AI was improving faster than a human does in MMR/day rates for specific heros. It looks like they got bored once the outcome became clear but needed a lot o work to achieve. > If this was a task in the real world, the AI would have the same interface as humans: The screen. This part of your argument is weak; you are relying on a computer having human limitations to show that it is superior. Computers don't have human limitations; that is a significant part of why humans can't compete with computers. > I think for AI to be able to claim that it is better at a game than humans, the AI ought to be playing the actual game, and use the same interface. What could you do if the bot people don't agree with you? When TeamDotaBot2000 is winning every game on the DotA ladder you can write them an angry all-chat message saying they didn't really win while your hero dead and their destroying your tree. That'd blend right in with the usual DotA crowd. > You might think that would be a trivial and useless exercise (like it would be in Chess/Go), but I'm really not convinced of that for video games... It isn't trivial; but it also isn't hard. It is time consuming and there are a lot of practical things that need to be solved. But if the resources are spent, it is expected that computers beat humans in any field with formally defined rules and objectives. That is a big deal.
- pingyong 7y ago
  >but you can train one agent/hero so the training scales in a somewhat linear fashion once they get beyond ~15-20 heros. No, you can't. The problem isn't training the AI to actually play all the heroes, that's the easy part (and also it's not even required to play the full game, no human can play every single hero), the problem is training the AI to play against all the heroes. That's the challenge - and that's the part that doesn't scale linearly at all. The more heroes the opponent can pick, the more complex your "behavior tree" has to become to think about every possible combination of what they could do. >you are relying on a computer having human limitations to show that it is superior. No, I rely on a computer playing the same actual game and not cheat. Because that's pretty much exactly what they're doing right now. It's like modding the game so that every animation is color coded by if it would currently hit you, every minion perfectly marked if it is in execution range, every champion running around with circles that show their ability ranges, with your champion being color coded depending on which ranges, etc. (Except actually even worse than that.) Again, in the actual real world an AI might have a 360 degree view of the environment, but it still has cameras and it still needs to do naturally imperfect object recognition algorithms on everything - it doesn't just get floating point values for every object's position and trajectory from God. >When TeamDotaBot2000 is winning every game on the DotA ladder If TeamDotaBot2000 wasn't directly affiliated with Valve, they would literally get banned for cheating. Because they're not interacting with the game like a normal player would, and they are doing things even a perfect, god-given AI playing the game through the same interface couldn't. >it is expected that computers beat humans in any field with formally defined rules and objectives. I expect this to be eventually true in any field. By so much that humans will be fundamentally useless in terms of advancing knowledge, culture, or anything in between. But if that's going to happen in 20, 50, 100 or 10,000 years really isn't clear to me, and neither is that all they'd have to do is put a realistic amount of hardware into their current approach to learn the entire game. The complexity of this task is definitely scaling exponentially, and just like you can't just throw twice the compute at a 128-bit cryptographic key because it worked so well for 64-bits, you can't just throw more compute at a game to learn to play against more champions.

roenxi 7y ago

> 18 out of 117 heroes does reduce complexity quite a lot, especially when you pick ones with less complex mechanics. Sure; but you can train one agent/hero so the training scales in a somewhat linear fashion once they get beyond ~15-20 heros. The computer won't be able to nail the full complexity of the game - but neither can a human, and the evidence from Go is the amount of the game a computer can grasp is much greater than a human. Humans can't fully understand full DotA either; it is common for pros to be surprised by mechanical interactions or quirks on specific heros. It might be an interesting theoretical challenge, but there would need to be some solid evidence of that beyond OpenAI calling it done and going home. Especially given the breakneck speed that the first 17 heros got trained. The AI was improving faster than a human does in MMR/day rates for specific heros. It looks like they got bored once the outcome became clear but needed a lot o work to achieve. > If this was a task in the real world, the AI would have the same interface as humans: The screen. This part of your argument is weak; you are relying on a computer having human limitations to show that it is superior. Computers don't have human limitations; that is a significant part of why humans can't compete with computers. > I think for AI to be able to claim that it is better at a game than humans, the AI ought to be playing the actual game, and use the same interface. What could you do if the bot people don't agree with you? When TeamDotaBot2000 is winning every game on the DotA ladder you can write them an angry all-chat message saying they didn't really win while your hero dead and their destroying your tree. That'd blend right in with the usual DotA crowd. > You might think that would be a trivial and useless exercise (like it would be in Chess/Go), but I'm really not convinced of that for video games... It isn't trivial; but it also isn't hard. It is time consuming and there are a lot of practical things that need to be solved. But if the resources are spent, it is expected that computers beat humans in any field with formally defined rules and objectives. That is a big deal.

pingyong 7y ago
>but you can train one agent/hero so the training scales in a somewhat linear fashion once they get beyond ~15-20 heros. No, you can't. The problem isn't training the AI to actually play all the heroes, that's the easy part (and also it's not even required to play the full game, no human can play every single hero), the problem is training the AI to play against all the heroes. That's the challenge - and that's the part that doesn't scale linearly at all. The more heroes the opponent can pick, the more complex your "behavior tree" has to become to think about every possible combination of what they could do. >you are relying on a computer having human limitations to show that it is superior. No, I rely on a computer playing the same actual game and not cheat. Because that's pretty much exactly what they're doing right now. It's like modding the game so that every animation is color coded by if it would currently hit you, every minion perfectly marked if it is in execution range, every champion running around with circles that show their ability ranges, with your champion being color coded depending on which ranges, etc. (Except actually even worse than that.) Again, in the actual real world an AI might have a 360 degree view of the environment, but it still has cameras and it still needs to do naturally imperfect object recognition algorithms on everything - it doesn't just get floating point values for every object's position and trajectory from God. >When TeamDotaBot2000 is winning every game on the DotA ladder If TeamDotaBot2000 wasn't directly affiliated with Valve, they would literally get banned for cheating. Because they're not interacting with the game like a normal player would, and they are doing things even a perfect, god-given AI playing the game through the same interface couldn't. >it is expected that computers beat humans in any field with formally defined rules and objectives. I expect this to be eventually true in any field. By so much that humans will be fundamentally useless in terms of advancing knowledge, culture, or anything in between. But if that's going to happen in 20, 50, 100 or 10,000 years really isn't clear to me, and neither is that all they'd have to do is put a realistic amount of hardware into their current approach to learn the entire game. The complexity of this task is definitely scaling exponentially, and just like you can't just throw twice the compute at a 128-bit cryptographic key because it worked so well for 64-bits, you can't just throw more compute at a game to learn to play against more champions.