4 ms·

(I work at OpenAI.) Great project. We've found that the VNC Universe environments are hard for today's RL algorithms primarily due to the their async nature. W

by gdb 9y ago

(I work at OpenAI.) Great project. We've found that the VNC Universe environments are hard for today's RL algorithms primarily due to the their async nature. We're currently working on a new set of Universe environments without VNC; I'm very happy to see others inspired by the core ideas of Universe as well.

Aqueous 9y ago
It seems like you might be duplicating work? At the end he mentions he's dropping VNC in favor of headless Chrome.
evc123 9y ago
recruit Alex Nichol (unixpickle).
du_bing 9y ago
Oh, Greg, nice to see you here, I am eager to see some solid Universe environments without VNC, that will be interesting.
unixpickle 9y ago
(Author here). Hi Greg! I am excited to hear about the new Universe environments. I want as many RL environments as possible for my upcoming project, so I will probably draw from Universe and ALE as well as µniverse. I took a lot of inspiration from Universe and am grateful for OpenAI's work on RL in general :). I probably wouldn't have started on this project if a company like OpenAI hadn't already decided it was a worthy goal.
pixelHD 9y ago
honest question, how interested is the academia/industry in deep learning libraries & game engines integrations? I've worked on unreal and tensorflow the last semester, and I found out that there aren't any existing integrations. I will probably work on a plugin, but I wanted to know if there is any interest? The way I see it, having hooks into the engines themselves helps with what the article talks about - not needing to go through VNCs or other _glue_ to get realtime data. It could potentially send the framebuffers themselves directly from the game/simulation and tie in the actions back to the game/simulation. And using framebuffers is just one direction, we could instead stream the co-ords/the current payoff/etc. Also, having such plugins would help with the adoption in both directions - games now have an always updating/learning AI (might need a network connection + cloud backend), and researchers can have training/testing environments.
- evc123 9y ago
  Arthur Juliani is about to open source his interface for connecting ML agents to Unity3D game engine: https://twitter.com/awjuliani/status/879142906178785281 https://twitter.com/awjuliani/status/879142906178785281
  - pixelHD 9y ago
    Wow, that looks great! I guess that's what i'm intending to do with unreal.

by gdb 9y ago

Aqueous 9y ago

It seems like you might be duplicating work? At the end he mentions he's dropping VNC in favor of headless Chrome.

evc123 9y ago

recruit Alex Nichol (unixpickle).

du_bing 9y ago

Oh, Greg, nice to see you here, I am eager to see some solid Universe environments without VNC, that will be interesting.

unixpickle 9y ago

(Author here). Hi Greg! I am excited to hear about the new Universe environments. I want as many RL environments as possible for my upcoming project, so I will probably draw from Universe and ALE as well as µniverse. I took a lot of inspiration from Universe and am grateful for OpenAI's work on RL in general :). I probably wouldn't have started on this project if a company like OpenAI hadn't already decided it was a worthy goal.

pixelHD 9y ago

honest question, how interested is the academia/industry in deep learning libraries & game engines integrations? I've worked on unreal and tensorflow the last semester, and I found out that there aren't any existing integrations. I will probably work on a plugin, but I wanted to know if there is any interest? The way I see it, having hooks into the engines themselves helps with what the article talks about - not needing to go through VNCs or other _glue_ to get realtime data. It could potentially send the framebuffers themselves directly from the game/simulation and tie in the actions back to the game/simulation. And using framebuffers is just one direction, we could instead stream the co-ords/the current payoff/etc. Also, having such plugins would help with the adoption in both directions - games now have an always updating/learning AI (might need a network connection + cloud backend), and researchers can have training/testing environments.

evc123 9y ago
Arthur Juliani is about to open source his interface for connecting ML agents to Unity3D game engine: https://twitter.com/awjuliani/status/879142906178785281 https://twitter.com/awjuliani/status/879142906178785281
- pixelHD 9y ago
  Wow, that looks great! I guess that's what i'm intending to do with unreal.