4 ms·

> There are two parts to this research program: (1) designing ever-more intelligent agents capable of more-and-more sophisticated cognitive skills, and (2) bui

> There are two parts to this research program: (1) designing ever-more intelligent agents capable of more-and-more sophisticated cognitive skills, and (2) building increasingly complex environments where agents can be trained and evaluated. I find this puzzling. If your goal were to create an human-like AI (which I always assume is at least partly implicit in these ambitious projects), it seems to me that the trickiest part is to determine what rewards make an optimization algorithm "human". How rewards weight and interact amongst themselves is where the mistery is, isn't it? So why isn't this part of the research program? Any deepminder wants to weight in on this?

nl 10y ago
which I always assume is at least partly implicit in these ambitious projects No serious researcher is even contemplating that problem yet, except as a thought experiment. These projects are more about working out how to work out what questions to ask to direct research which might lead to more generalised AI.
- rahrahrah 10y ago
  Alright thanks. Do you want to recommend reading regarding the state of the art? Technical papers Ok.
  - nl 10y ago
    SOTA on what specifically? DeepMind is probably the world leader on reinforcement learning, so https://deepmind.com/research/publications/ https://deepmind.com/research/publications/ isn't a bad place to start.
davmre 10y ago
There are plenty of domains where the objective is quite well defined. Video games are excellent examples. An agent that achieved superhuman performance on, say, Starcraft, would have a very impressive suite of capabilities involving some significant advances from the current state of the art. As such these tasks are great drivers of research. But you're absolutely right that humans don't optimize any 'simple' reward function, and achieving human-like behavior in real-world domains will likely require learning reward functions. There are a few people starting to think about this, e.g. this paper https://arxiv.org/abs/1606.03137 https://arxiv.org/abs/1606.03137 (disclaimer: work from my research group though I'm not involved) proposing a framework by which a robot can learn a human's (implicit, complex) values through interaction. This is also related to concerns over AI safety, since naive reward-optimizing agents are (like drug addicts) willing to do arbitrarily bad things in service of maximizing their prescribed "reward", while agents that maintain uncertainty over reward functions are willing to ask for guidance when confused or unsure. However this line of research is much more preliminary and academic -- there are probably people at DeepMind thinking along these lines, but certainly their focus is more on directions that will produce new breakthroughs and practical capabilities within forseeable timeframes.

nl 10y ago

which I always assume is at least partly implicit in these ambitious projects No serious researcher is even contemplating that problem yet, except as a thought experiment. These projects are more about working out how to work out what questions to ask to direct research which might lead to more generalised AI.

rahrahrah 10y ago
Alright thanks. Do you want to recommend reading regarding the state of the art? Technical papers Ok.
- nl 10y ago
  SOTA on what specifically? DeepMind is probably the world leader on reinforcement learning, so https://deepmind.com/research/publications/ https://deepmind.com/research/publications/ isn't a bad place to start.

davmre 10y ago

There are plenty of domains where the objective is quite well defined. Video games are excellent examples. An agent that achieved superhuman performance on, say, Starcraft, would have a very impressive suite of capabilities involving some significant advances from the current state of the art. As such these tasks are great drivers of research. But you're absolutely right that humans don't optimize any 'simple' reward function, and achieving human-like behavior in real-world domains will likely require learning reward functions. There are a few people starting to think about this, e.g. this paper https://arxiv.org/abs/1606.03137 https://arxiv.org/abs/1606.03137 (disclaimer: work from my research group though I'm not involved) proposing a framework by which a robot can learn a human's (implicit, complex) values through interaction. This is also related to concerns over AI safety, since naive reward-optimizing agents are (like drug addicts) willing to do arbitrarily bad things in service of maximizing their prescribed "reward", while agents that maintain uncertainty over reward functions are willing to ask for guidance when confused or unsure. However this line of research is much more preliminary and academic -- there are probably people at DeepMind thinking along these lines, but certainly their focus is more on directions that will produce new breakthroughs and practical capabilities within forseeable timeframes.