3 ms·

Thanks for your response, I had actually heard about Yuri before and was hoping you would respond to this post! I have a lot of questions for you if you don't

Thanks for your response, I had actually heard about Yuri before and was hoping you would respond to this post! I have a lot of questions for you if you don't mind, maybe you don't want to answer any/all of them that's fine. 1. How do you handle constraining the AI such that it's enjoyable? If you read through the comments here you'll notice just about everyone is saying the hard part is not making a good AI but in making a fun AI. Do you know of any research in this area? most examples I see are all about making the best AI possible. It seems like it's really hard to find a good reward signal to use for "fun". Have you seen this [0,1] paper on learning from human preferences? does Yuri attempt to do something similar? It seems like it would be labor intensive for the play testers. 2. Does Yuri make use of imitation learning at all? I.E learning from lots of human data when it's available to bootstrap the learning process? 3. Do you let the game designers impose any structure on the AI? I.E. "I want three phases for the boss where he progressively gets more aggressive" or use RL just for certain small parts of the behavior? A well defined path finding algorithm with the RL deciding where the agent tries to go? stuff like that? I have a ton of questions about the technical details side of Yuri too, what frameworks are you using, which RL algorithms etc etc, not sure if you are willing to share those or not. [0] https://arxiv.org/abs/1706.03741 https://arxiv.org/abs/1706.03741 [1] https://blog.openai.com/deep-reinforcement-learning-from-human-preferences/ https://blog.openai.com/deep-reinforcement-learning-from-hum...

formalsystem 8y ago
1. We use a couple of tricks for this, there's ideas from human preferences. There's modeling a reward function to maximize fun instead of pure performance, there's handicaps and a host of other tricks. Different tricks work better for different games but game developers can easily configure them 2. It can, imitation learning turns RL into a supervised learning problem though from my experience it's not really needed. It's more of a if you already have it then you can make the training a bit faster 3. Yes all the RL part becomes tuning a config file where the game developer needs to describe the task it wants to be accomplished Regarding technical details, the infra is agnostic to the specific deep learning library you'd like to use and the algorithms are chosen based on the types of inputs (continuous vs discrete), size of the space (the end all of strategy games or a casual puzzle game) and type of game (full information vs partial information).

formalsystem 8y ago

1. We use a couple of tricks for this, there's ideas from human preferences. There's modeling a reward function to maximize fun instead of pure performance, there's handicaps and a host of other tricks. Different tricks work better for different games but game developers can easily configure them 2. It can, imitation learning turns RL into a supervised learning problem though from my experience it's not really needed. It's more of a if you already have it then you can make the training a bit faster 3. Yes all the RL part becomes tuning a config file where the game developer needs to describe the task it wants to be accomplished Regarding technical details, the infra is agnostic to the specific deep learning library you'd like to use and the algorithms are chosen based on the types of inputs (continuous vs discrete), size of the space (the end all of strategy games or a casual puzzle game) and type of game (full information vs partial information).