4 ms·

This seems pretty obvious for practicality. The AI can play thousands, or millions of games in different VMs 24/7 and be exposed to a radically higher number o

This seems pretty obvious for practicality. The AI can play thousands, or millions of games in different VMs 24/7 and be exposed to a radically higher number of simulated circumstances versus the comparatively plodding rate of genuine interaction.

aeleos 9y ago
The main issue with self-play is that unless done very methodically it can lead to behavior that does not learn what we want it to learn, but games the simulation and basically cheats. Its not a perfect solution and is just another tool being used to improve models. It can produce really interesting results especially in complex games but its also not perfect.
- vorotato 9y ago
  One could argue that "cheating" is a valid solution.
  - tlb 9y ago
    Only when the learning environment is the same environment it will use when deployed. Agents for robots are often trained in simulation, where it's a common problem for the agent to exploit a physics bug in the simulator.
- recursive 9y ago
  You only get what you measure. If you can figure out how to measure the right thing, then that's what you will get.
- gooseus 9y ago
  I agree... I feel like advanced games will always need guidance to alter the game such that they AI don't reach some local maxima through an exploit. This happens all the time when making games for humans and is evident by how many balancing patches are made to new, highly competitive games (such as any Blizzard title). The next logical step would be for another, impartial AI to observe the games and changing the rules and parameters intelligently as they evolve to guide the player AI toward the actual goal. So, I'll just dive sideways right into a religious/philosophical thought based on the simulation discussion I've been having all over the place: A universe-sized simulation built for a purpose, which requires simulated intelligence to carry out that purpose, would almost certainly include a God intelligence to alter parameters and induce suffering/hardship to direct the simulated intelligence toward that purpose.
  - Danihan 9y ago
    >would almost certainly include a God intelligence to alter parameters and induce suffering/hardship to direct the simulated intelligence toward that purpose. Wow that's pretty deep, actually.
  - danohu 9y ago
    Earlier iterations are buggier and have poorer dev tools. So the God intelligence has more need to smite and command the AIs within the game. After a while the bugs are ironed out, so God can settle back and gently tweak parameters at a distance.
    - visarga 9y ago
      > After a while the bugs are ironed out, so God can settle back and gently tweak parameters at a distance. That explains the hands off approach God has lately with the human society.
skgoa 9y ago
It is obvious. This is pure PR for OpenAI. They are not doing anything that hasn't been done to death a decade ago.
- dmix 9y ago
  It's some excellent PR regardless, no need to be cynical, not to mention highly educational and usually comes with high quality open source code demonstrations with white papers.

aeleos 9y ago

The main issue with self-play is that unless done very methodically it can lead to behavior that does not learn what we want it to learn, but games the simulation and basically cheats. Its not a perfect solution and is just another tool being used to improve models. It can produce really interesting results especially in complex games but its also not perfect.

vorotato 9y ago
One could argue that "cheating" is a valid solution.
- tlb 9y ago
  Only when the learning environment is the same environment it will use when deployed. Agents for robots are often trained in simulation, where it's a common problem for the agent to exploit a physics bug in the simulator.
recursive 9y ago
You only get what you measure. If you can figure out how to measure the right thing, then that's what you will get.
gooseus 9y ago
I agree... I feel like advanced games will always need guidance to alter the game such that they AI don't reach some local maxima through an exploit. This happens all the time when making games for humans and is evident by how many balancing patches are made to new, highly competitive games (such as any Blizzard title). The next logical step would be for another, impartial AI to observe the games and changing the rules and parameters intelligently as they evolve to guide the player AI toward the actual goal. So, I'll just dive sideways right into a religious/philosophical thought based on the simulation discussion I've been having all over the place: A universe-sized simulation built for a purpose, which requires simulated intelligence to carry out that purpose, would almost certainly include a God intelligence to alter parameters and induce suffering/hardship to direct the simulated intelligence toward that purpose.
- Danihan 9y ago
  >would almost certainly include a God intelligence to alter parameters and induce suffering/hardship to direct the simulated intelligence toward that purpose. Wow that's pretty deep, actually.
- danohu 9y ago
  Earlier iterations are buggier and have poorer dev tools. So the God intelligence has more need to smite and command the AIs within the game. After a while the bugs are ironed out, so God can settle back and gently tweak parameters at a distance.
  - visarga 9y ago
    > After a while the bugs are ironed out, so God can settle back and gently tweak parameters at a distance. That explains the hands off approach God has lately with the human society.

skgoa 9y ago

It is obvious. This is pure PR for OpenAI. They are not doing anything that hasn't been done to death a decade ago.

dmix 9y ago
It's some excellent PR regardless, no need to be cynical, not to mention highly educational and usually comes with high quality open source code demonstrations with white papers.