5 ms·

The point isn't to see if the model can process images. OpenAI's goal is to see if they can recreate the ability to plan and strategize over a partial informati

The point isn't to see if the model can process images. OpenAI's goal is to see if they can recreate the ability to plan and strategize over a partial information continuous long time horizon environment. You wouldn't want AlphaGo to have to input it's commands using robotic hands right? It's the same thing in that, sure it might be interesting, but that isn't what we care about. Image processing and robotics controls are largely solved. Showcasing that a model can gain the ability to plan and think is the novel stuff here, and is the path to where "artificial intelligence" if any appears. That's the ultimate goal in playing any of these games.

sgillen 8y ago
>> Image processing and robotics controls are largely solved Image processing and robotic control are very far from being solved problems. I guess you are saying that in the case of alpha go it would not be a super difficult step to have a camera and robotic hand physically move pieces around, and that's probably true. But I think in the DOTA case are new image processing challenges that interact with the AI in interesting ways. I'm mostly talking about the need to move the game's camera around to gain more information. If you don't see your ally on your screen and need to see how they are handling a gank or something (full disclosure I don't play DOTA at all this could be a silly scenario). Then the AI would have to recognize this and move the camera to the allies location in order to gain that information. So really the novelty here would be in the network to somehow realize what information it needs and then further to learn how to gather that information. I honestly think that sounds like an extremely difficult next step.
- nsomaru 8y ago
  This is something i noticed; a human initiator would get counter initiated almost instantly, every single time by OpenAI. The blink dagger is much less effective. Pro humans do this too, but not every single time with perfect timing. Humans dont concentrate on the whole screen, attention is directed...
  - jacoblambda 8y ago
    It would be interesting to take this project up a few levels later on and see how it compares to direct API interaction. I would love to see camera/mechanical interface like mentioned by others. Similarly, like you said humans don't focus on the whole screen. I would love to see how well the AI could perform if it was given something like blinders where only a small portion of the screen is in focus at any one time much like how human eyes work.
  - acchow 8y ago
    I believe human counter-initiation is only frame-perfect when the human is anticipating the initiation to happen (baiting). Otherwise, you still have to add time to react plus mouse travel time.
- adamrezich 8y ago
  Human beings are restricted to what's on the screen, which includes not only the camera's perspective of the playfield but also, crucially, the minimap in the corner of the screen. Plus there's weird stuff like how you don't have perfect information about the health/mana of your teammates unless you hold down Alt... so yeah OpenAI is "cheating" somewhat and it would be really cool to see, once it evolves further, restrictions that allow it to better mimic human player capabilities. That said, everything they've done so far is absolutely incredible (especially now that the AI can draft!!)
- zaarn 8y ago
  There is actually an interesting paper on that, you can find it on the youtube channel two minute papers. Basically, they let an AI loose on a simplified version of Quake's Capture the Flag. The AI processes game video output only and has learned several key strategies. The latest update has the AI with a winrate of 71% against top humans. Unlike the DOTA match the AI has no restricted reaction time. The AI seems to be jittering the camera left and right to reconstruct a 3D image reliable from the screen which is aquite interesting way to compensate for lack of 3D vision (and the compensation our brain is capable of naturally to get a 3D intuition from a 2D image)
TulliusCicero 8y ago
> OpenAI's goal is to see if they can recreate the ability to plan and strategize over a partial information continuous long time horizon environment. If this was actually the goal, they would add further mechanical restrictions beyond the 200ms delay to simulate the way humans players play. That way, human and AI would be on a roughly even mechanical playing field, leaving the differentiating factor only strategy/tactics. As it stands, it looks like their victory is as much based on raw mechanical superiority as it is strategic/tactical superiority. Computers being able to be pixel-perfect accurate at all times, issue commands at ludicrous speeds, etc. is kind of an uninteresting advantage in the context of building strategic AI.
- lawrenceyan 8y ago
  What in your opinion would be a sufficient delay? 200 milliseconds is well within the average reaction time of a human, especially where at the pro level, I'm betting it's much lower than that.
  - yourMadness 8y ago
    Reaction time is usually measured as time from stimulus to a button press. For many things in Dota you also need to move the mouse cursor to a specific point on the screen which obviously takes longer than just pressing a button.
  - CrystalLangUser 8y ago
    That doesn't account for focusing on other things. There are a multitude of things to take into account while playing dota that can pull your attention; you can't always directly focus on your character in expectation of a blink initiation.
    - chii 8y ago
      but that's moving the goal post: why is attention a factor here at all?
      - 8y ago

sgillen 8y ago

>> Image processing and robotics controls are largely solved Image processing and robotic control are very far from being solved problems. I guess you are saying that in the case of alpha go it would not be a super difficult step to have a camera and robotic hand physically move pieces around, and that's probably true. But I think in the DOTA case are new image processing challenges that interact with the AI in interesting ways. I'm mostly talking about the need to move the game's camera around to gain more information. If you don't see your ally on your screen and need to see how they are handling a gank or something (full disclosure I don't play DOTA at all this could be a silly scenario). Then the AI would have to recognize this and move the camera to the allies location in order to gain that information. So really the novelty here would be in the network to somehow realize what information it needs and then further to learn how to gather that information. I honestly think that sounds like an extremely difficult next step.

nsomaru 8y ago
This is something i noticed; a human initiator would get counter initiated almost instantly, every single time by OpenAI. The blink dagger is much less effective. Pro humans do this too, but not every single time with perfect timing. Humans dont concentrate on the whole screen, attention is directed...
- jacoblambda 8y ago
  It would be interesting to take this project up a few levels later on and see how it compares to direct API interaction. I would love to see camera/mechanical interface like mentioned by others. Similarly, like you said humans don't focus on the whole screen. I would love to see how well the AI could perform if it was given something like blinders where only a small portion of the screen is in focus at any one time much like how human eyes work.
- acchow 8y ago
  I believe human counter-initiation is only frame-perfect when the human is anticipating the initiation to happen (baiting). Otherwise, you still have to add time to react plus mouse travel time.
adamrezich 8y ago
Human beings are restricted to what's on the screen, which includes not only the camera's perspective of the playfield but also, crucially, the minimap in the corner of the screen. Plus there's weird stuff like how you don't have perfect information about the health/mana of your teammates unless you hold down Alt... so yeah OpenAI is "cheating" somewhat and it would be really cool to see, once it evolves further, restrictions that allow it to better mimic human player capabilities. That said, everything they've done so far is absolutely incredible (especially now that the AI can draft!!)
zaarn 8y ago
There is actually an interesting paper on that, you can find it on the youtube channel two minute papers. Basically, they let an AI loose on a simplified version of Quake's Capture the Flag. The AI processes game video output only and has learned several key strategies. The latest update has the AI with a winrate of 71% against top humans. Unlike the DOTA match the AI has no restricted reaction time. The AI seems to be jittering the camera left and right to reconstruct a 3D image reliable from the screen which is aquite interesting way to compensate for lack of 3D vision (and the compensation our brain is capable of naturally to get a 3D intuition from a 2D image)

TulliusCicero 8y ago

> OpenAI's goal is to see if they can recreate the ability to plan and strategize over a partial information continuous long time horizon environment. If this was actually the goal, they would add further mechanical restrictions beyond the 200ms delay to simulate the way humans players play. That way, human and AI would be on a roughly even mechanical playing field, leaving the differentiating factor only strategy/tactics. As it stands, it looks like their victory is as much based on raw mechanical superiority as it is strategic/tactical superiority. Computers being able to be pixel-perfect accurate at all times, issue commands at ludicrous speeds, etc. is kind of an uninteresting advantage in the context of building strategic AI.

lawrenceyan 8y ago
What in your opinion would be a sufficient delay? 200 milliseconds is well within the average reaction time of a human, especially where at the pro level, I'm betting it's much lower than that.
- yourMadness 8y ago
  Reaction time is usually measured as time from stimulus to a button press. For many things in Dota you also need to move the mouse cursor to a specific point on the screen which obviously takes longer than just pressing a button.
- CrystalLangUser 8y ago
  That doesn't account for focusing on other things. There are a multitude of things to take into account while playing dota that can pull your attention; you can't always directly focus on your character in expectation of a blink initiation.
  - chii 8y ago
    but that's moving the goal post: why is attention a factor here at all?
    - 8y ago