3 ms·

Is dealing with imperfect information a research goal? Does the OpenAI team think there's a way to adapt the UX of DOTA 2 "Perfect Information Edition" to comm

Is dealing with imperfect information a research goal? Does the OpenAI team think there's a way to adapt the UX of DOTA 2 "Perfect Information Edition" to communicate the game better to human players?

modeless 8y ago
AFAIK the bot's "vision" is subject to fog of war, so it's not a perfect information game in the usual sense. Yes, it gets precise numerical values for hit points etc from the API, but only for visible units. Honestly I think that it would not be much more difficult to train a bot that looks at screen pixels and outputs keyboard and mouse events instead of using the bot API. In fact it might be easier to code, but the problem is it would require several orders of magnitude more processing power to train, which is impractical. I am confident it would work if the processing power was available, given the success of these techniques on other problems.
- drexlspivey 8y ago
  This would require the bot to learn to point the screen to the right place
  - modeless 8y ago
    It would require the bot to learn a lot of things. Perhaps a curriculum learning method would be appropriate. I don't see any reason why it wouldn't be possible though, given several orders of magnitude more compute power.
  - Benjammer 8y ago
    Fog of war on the map doesn't depend on where a single player's viewport is pointing...
- a_humean 8y ago
  Probably bigger than you think. The AI has perfect information about everything in vision with few exceptions. If they have to use the standard UI they lose a significant amount of information as they only view a very small percentage of the map per frame + the minimap. Just think about the implications for team fights. If all the agents have different information about a fight, then you aren't necessarily going to see the uncanny stun stacking and perfect long range nukes. The AI cannot assume that all the other agents share the same information, and so the other agents might not be as predictable. I imagine fragmented information would dramatically change behavior, for the worse. They will probably act more like human players - more cautiously and with more mistakes due to "miscommunication".
  - itsaporter 8y ago
    One thing to note is that the AI does not orchestrate all 5 players on the team at once like 1 superhuman player would. Instead it runs 5 instances of itself on each of the heroes and they play together based on how selfish they need to be in any given moment. This earlier blog post[1] has some additional information about how the bot works, including an interactive view into the bot's eyes. >"OpenAI Five does not contain an explicit communication channel between the heroes’ neural networks. Teamwork is controlled by a hyperparameter we dubbed “team spirit”. Team spirit ranges from 0 to 1, putting a weight on how much each of OpenAI Five’s heroes should care about its individual reward function versus the average of the team’s reward functions. We anneal its value from 0 to 1 over training." [1] https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/

modeless 8y ago

AFAIK the bot's "vision" is subject to fog of war, so it's not a perfect information game in the usual sense. Yes, it gets precise numerical values for hit points etc from the API, but only for visible units. Honestly I think that it would not be much more difficult to train a bot that looks at screen pixels and outputs keyboard and mouse events instead of using the bot API. In fact it might be easier to code, but the problem is it would require several orders of magnitude more processing power to train, which is impractical. I am confident it would work if the processing power was available, given the success of these techniques on other problems.

drexlspivey 8y ago
This would require the bot to learn to point the screen to the right place
- modeless 8y ago
  It would require the bot to learn a lot of things. Perhaps a curriculum learning method would be appropriate. I don't see any reason why it wouldn't be possible though, given several orders of magnitude more compute power.
- Benjammer 8y ago
  Fog of war on the map doesn't depend on where a single player's viewport is pointing...
a_humean 8y ago
Probably bigger than you think. The AI has perfect information about everything in vision with few exceptions. If they have to use the standard UI they lose a significant amount of information as they only view a very small percentage of the map per frame + the minimap. Just think about the implications for team fights. If all the agents have different information about a fight, then you aren't necessarily going to see the uncanny stun stacking and perfect long range nukes. The AI cannot assume that all the other agents share the same information, and so the other agents might not be as predictable. I imagine fragmented information would dramatically change behavior, for the worse. They will probably act more like human players - more cautiously and with more mistakes due to "miscommunication".
- itsaporter 8y ago
  One thing to note is that the AI does not orchestrate all 5 players on the team at once like 1 superhuman player would. Instead it runs 5 instances of itself on each of the heroes and they play together based on how selfish they need to be in any given moment. This earlier blog post[1] has some additional information about how the bot works, including an interactive view into the bot's eyes. >"OpenAI Five does not contain an explicit communication channel between the heroes’ neural networks. Teamwork is controlled by a hyperparameter we dubbed “team spirit”. Team spirit ranges from 0 to 1, putting a weight on how much each of OpenAI Five’s heroes should care about its individual reward function versus the average of the team’s reward functions. We anneal its value from 0 to 1 over training." [1] https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/