4 ms·

I was making slides to discuss this model in my deep learning class tomorrow. From what I can find on the model, it isn't that complex -- https://d4mucfpksywv.c

I was making slides to discuss this model in my deep learning class tomorrow. From what I can find on the model, it isn't that complex -- https://d4mucfpksywv.cloudfront.net/research-covers/openai-five/network-architecture.pdf https://d4mucfpksywv.cloudfront.net/research-covers/openai-f.... The real trick seems to be just massive amounts of self-play. This model had the human equivalent of 45,000 years of experience playing DOTA 2, with 250 years of experience per day.

qlk1123 7y ago
Outsider here. How do you measure the equivalent experiences of a model? Tens of thousands of years ...... how is that possible? Does any timing trick involve in your training/evaluating phase?
- ALittleLight 7y ago
  Speculating: the game has a clock. I assume they are running the game sped up, at a faster tick rate than humans play at, and also many instances, but if you could sum how many game-seconds elapsed in all the training games you could come up with a number that would make sense.
- hug 7y ago
  The game has an in-game clock, and there are regular "timed" events that take place throughout the game that are basically a hard cap on when you can realistically win the game. (i.e.: To earn gold & experience, you must kill 'creeps', and waves of creeps spawn every 1 minute in game. You'll need a certain amount of gold & experience to have the in-game strength to take down defensive towers.) An average human game lasts 30 to 45 minutes. If OpenAI simulated 1,000,000 games, that'd be over 28 years of non-stop gameplay to a human.
  - qlk1123 7y ago
    It makes me wonder, what to do then if an game of interest does not have such timing mechanism? To put it another way, how to train the agents as fast as AlphaGO (no timing issues at all) or OpenAI Five (in-game overclocking) if the environment can only be simulated in the same time scale? Any thoughts?
CGamesPlay 7y ago
Please help me by correcting any misunderstanding I have: So the large section is run for each unit that is known to the agent. For heroes, each modifier / ability / item that hero possesses is processed. Since it’s max-pooled, only one of the modifiers / abilities / items would actually be considered by the agent. Later, the known units are grouped by their relation to the agent and max-pooled again, so the agent would only consider a single enemy non-hero and single enemy hero at a time. Also, what activation function is used for the FC (fully-connected) layers that aren’t relu?

qlk1123 7y ago

Outsider here. How do you measure the equivalent experiences of a model? Tens of thousands of years ...... how is that possible? Does any timing trick involve in your training/evaluating phase?

ALittleLight 7y ago
Speculating: the game has a clock. I assume they are running the game sped up, at a faster tick rate than humans play at, and also many instances, but if you could sum how many game-seconds elapsed in all the training games you could come up with a number that would make sense.
hug 7y ago
The game has an in-game clock, and there are regular "timed" events that take place throughout the game that are basically a hard cap on when you can realistically win the game. (i.e.: To earn gold & experience, you must kill 'creeps', and waves of creeps spawn every 1 minute in game. You'll need a certain amount of gold & experience to have the in-game strength to take down defensive towers.) An average human game lasts 30 to 45 minutes. If OpenAI simulated 1,000,000 games, that'd be over 28 years of non-stop gameplay to a human.
- qlk1123 7y ago
  It makes me wonder, what to do then if an game of interest does not have such timing mechanism? To put it another way, how to train the agents as fast as AlphaGO (no timing issues at all) or OpenAI Five (in-game overclocking) if the environment can only be simulated in the same time scale? Any thoughts?

CGamesPlay 7y ago

Please help me by correcting any misunderstanding I have: So the large section is run for each unit that is known to the agent. For heroes, each modifier / ability / item that hero possesses is processed. Since it’s max-pooled, only one of the modifiers / abilities / items would actually be considered by the agent. Later, the known units are grouped by their relation to the agent and max-pooled again, so the agent would only consider a single enemy non-hero and single enemy hero at a time. Also, what activation function is used for the FC (fully-connected) layers that aren’t relu?