10 ms·

OpenAI technical goals

220 points by runesoerensen 10y ago

jimfleming 10y ago
This is great! The goals seem reasonably ambitious and mostly doable over a few years. I am surprised by #2: "Build a household robot". It's my understanding that efficient actuation and power are largely unsolved problems outside of the software realm. What's the plan for tackling stairs, variable height targets, manipulator dexterity, power supply, etc. in a general purpose robot with off-the-shelf parts? (Answering these questions may be part of that goal but maybe someone knows more on the subject.)
- jhuckestein 10y ago
  I wonder if OpenAI could define and build against a robot API that hardware robots would implement in different ways. That way OpenAI could focus on the software and various other teams could compete to solve all the mechanical challenges you mention.
- nightski 10y ago
  When it comes to AI/ML most problems seem "doable over a few years". Yet in practice this rarely is the case. These are some extremely lofty goals. The team behind OpenAI is quite capable but to say they'll have any of these done in a few years is quite a stretch. I'm guessing they may achieve one of the goals in a decade. But I'd love to be wrong.
  - jimfleming 10y ago
    It was an off-hand remark. I'm aware of the landscape, though perhaps slightly more optimistic. The first goal is simple enough and largely underway with the Gym. Significant progress has been made on #3 and #4 just in the last year but I agree that "a few years" is a bit brief. I remain doubtful about #2.
- ylem 10y ago
  At least on stairs, I think this is a solvable problem: https://www.technologyreview.com/s/601240/an-impressive-walking-google-robot-tries-to-vacuum-the-stairs/ https://www.technologyreview.com/s/601240/an-impressive-walk... In terms of off-the-shelf: http://makezine.com/2015/05/01/meet-stair-bear-adorable-climbing-robot/ http://makezine.com/2015/05/01/meet-stair-bear-adorable-clim... I also vaguely recall seeing some demos of first responder robots and stairs. I don't know about the other issues you've raised...
- tlb 10y ago
  Off-the-shelf robots are convenient for developing learning algorithms. When the learning algorithms are good enough to take advantage of more capable hardware, it'll make sense to build it.
- davmre 10y ago
  Pieter Abbeel has been working towards household robots for quite a while; given his involvement and several of his students at OpenAI it's not surprising they'd be thinking along similar lines. The issues you mention are real, but current hardware is already capable of useful tasks if we had software to control it properly. For example, here's a demo of a tele-operated PR2 performing household chores: https://www.youtube.com/watch?v=S2GAz5F03Ls#t=60s https://www.youtube.com/watch?v=S2GAz5F03Ls#t=60s
JoshTriplett 10y ago
#4, solving multiple games with one agent, seems like a reasonably interesting step towards more general intelligence. An agent that can play a new game without any game-specific information other than "what can I do" and "what should I value" starts to sound a lot more like an agent that can solve non-game problems. Especially if that agent can play games that model real-world problems.
fitzwatermellow 10y ago
Goal 5: Inventing the Next Paradigm Any impetus into actually dreaming up what may come next after GPUs and Async DRL? Non-neural models, quantum computing based AI, optogenetic hacking ;) Otherwise, excellent list!
nxzero 10y ago
Is OpenAI willing to support true AI having basic rights to given to humans? If so, why is this not one of the fundamental technical goals? If not, why?
- jimfleming 10y ago
  You should have a look a PETRL[0]: People for the Ethical Treatment of Reinforcement Learners :) [0] http://petrl.org/ http://petrl.org/
  - castis 10y ago
    That was way more of an interesting read that I thought it was going to be. Thanks for posting that.
- JoshTriplett 10y ago
  I would argue that we should avoid building sentient AI in the first place. Human-level or better problem-solving ability does not require the inclusion of sentience or independent agency.
  - nxzero 10y ago
    Then I assume you're against any tech that would allow brains to be digitized, right?
    - JoshTriplett 10y ago
      Not at all; I'm incredibly in favor of doing so, and I would argue that a human mind running in a computer is still unambiguously human. The hard part comes in defining when a computer has reached that point. I'm arguing that we shouldn't develop computer AI that comes anywhere close to needing to answer that question; we don't need that to solve the problems that AI can solve.
      - nxzero 10y ago
ktta 10y ago
I find it really interesting that Goal 4 is a game playing agent. Deepmind has been focussing on this since the beginning[0] and actually has great progress as far as Atari games go[1]. And DeepMind is able to use a single agent rather than a different specific one for each game. I wonder if OpenAI wants to go in a different direction although RL has considerable success. Whereas the other goals definitely need a lot of work before they are "real-world" functional, especially Goal 3. But of course that depends on their definition of 'useful'. [0]:https://www.youtube.com/watch?v=rbsqaJwpu6A https://www.youtube.com/watch?v=rbsqaJwpu6A [1]:http://arxiv.org/pdf/1312.5602v1.pdf http://arxiv.org/pdf/1312.5602v1.pdf
- deleted 10y ago
  [deleted]
- ratsimihah 10y ago
  I feel like both companies are trying to "solve intelligence" by aiming for true AI, in which case there has to be some overlap in the work they're doing to get there.
- argonaut 10y ago
  You're mistaken, DeepMind uses a different agent for each game. What's the same across games is the learning method. But the model is trained separately for each game.
  - ktta 10y ago
    I feel like I'm a bit confused at what constitutes a model. You basically have the same 'agent' to start off on the RL process. You can just group the learning agents together and make it (easier to tell it) automatically detect the game that has to be played. I guess I wanted to say that there isn't a lot different to be done when an agent is being trained for different games. And that makes it a general game playing agent, doesn't it?
    - argonaut 10y ago
      If you group the agents together and call it a single "general" agent, the research community (and pretty much everyone else) will call you out on your BS. That's the difference.
Houshalter 10y ago
I'm concerned that none of these goals involve AI safety. Instead all of their goals are nearly the exact opposite, of accelerating AI technology as much as possible. Safety was one of the main goals they promoted when it was founded. 2 of the 4 authors listed have publicly spoken about their belief of AI as an existential risk. I'm not saying that a game playing AI is going to take over the world. But it does demonstrate the risk - we still have no idea how to control such an AI. We can train it to get high scores. But it won't want to do anything other than get high scores. And it will do whatever it takes to get the highest score possible, even if it means exploiting the game or hurting other players, or disobeying it's masters. Now imagine they succeed in making smarter AIs. And their research spawns new research, which inspires new research, etc. Perhaps over several decades we could have AIs that are a lot more formidable than being able to play Pac Man. But we may still not have made any progress on the ability to control them.
- astanway 10y ago
  Almost no one who actually works or has done serious research in ML is genuinely concerned about "malevolent AI". We are so, so, so far away from anything remotely close to that. Please stop trying to gin up fear and listen to the experts, who uniformly agree that this is not something to be concerned about.
  - ikeboy 10y ago
    Obligatory http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/ http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri...
  - crpatino 10y ago
    The issue is not "malevolent AI", the probability of producing an AI with "malevolent goals" by random chance is so much more unlikely than getting any AI at all that it's not even worth it to worry about that. The problem is "any AI" whose goals are not positively aligned-with/respectul-of human society's values. In terms with Asimov's Lay of Robotics meme, the problem is not robots harming humans, but robots allowing humans to come to harm as a side effect of their actions. This is an important ethical issue that, IMHO, technologist in general are failing to address. Unethical AIs are only a special case in the sense that they are believed to be able to cause much more chaos and destruction than spammers, patent trolls or high frequency trading firms.
feral 10y ago
Question: "Build a household robot" is high up the list. That doesn't seem inherently 'general'. Certainly, people have been working on that for years; there are all sorts of subproblems like vision, contextual reasoning etc. It could be treated as a general problem, requiring a lot of 'common sense'. But a team which sets out to optimize that particular goal, could spend years on relatively narrow tasks that get good performance returns on household chores (e.g. developing version 10 of the floor cleaning algorithm), but don't really make progress towards the problem of general intelligence. For me, what was really interesting about the benchmarks that Deepmind chose (the choice of a selection of Atari games) was that they were inherently somewhat general. Are you not worried that by putting a narrow domain fairly high up, you'll get distracted by narrow tasks, rather than making progress towards what's really interesting - generality? Won't it introduce tension to try and keep the general focus in the presence of a narrow goal, where you can get good returns by overfitting?
- argonaut 10y ago
  This isn't a particularly meaningful issue. The problem you describe of falling into the trap of brute-force optimizing a narrow task also applies to the Atari games. In fact it applies even more-so: it would be trivial for a lot of HN programmers to brute-force code an AI for challenging Atari games that deep learning still struggles against (like Montezuma's Revenge). But it would be nontrivial (difficult but still possible) to brute-force code a program for many narrow household robot tasks. You avoid this problem by... not hard-coding brute force solutions/heuristics! The research community can smell BS very easily (HN, not so much). A household robot is substantially (probably an order of magnitude) more general than Atari games, even for narrow tasks (obviously it is nowhere near the vicinity of the generality of AGI). The perception problem is tremendously more complex. The control/planning problem is similarly tremendously more complex.
  - ktta 10y ago
    >But it would be nontrivial (difficult but still possible) to brute-force code a program for many narrow household robot tasks. How difficult are you talking about here? Similar to training game agents? I really doubt it. I feel like the training problem is VERY hard in case of real world random object handling (factory like fixed, mechanical situation can be purely hard coded and be much better than a human). In case of virtual games you can just use a bunch of GPUs and accelerate the process. But it is a much more difficult problem in reality. The grasping ability that we have with everyday objects is a marvel once you try to make a computer to do it. This might be of interest: http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/google-large-scale-robotic-grasping-project http://spectrum.ieee.org/automaton/robotics/artificial-intel...
lowglow 10y ago
Are the objectives of "OpenAI" conflicting with the interests of the startups applying to YC? OpenAI is building their own products/platforms and insights acquired from AI/Robotics startups applying and sharing information about what they're working on might be used as a competitive advantage. Is there an information firewall between what startups are sharing in the hope of investment and what is shared to advance OpenAI? If there is a firewall, how is it enforced?
mrfusion 10y ago
I wish they had mentioned asimovs three laws.

220 points by runesoerensen 10y ago

This is great! The goals seem reasonably ambitious and mostly doable over a few years. I am surprised by #2: "Build a household robot". It's my understanding that efficient actuation and power are largely unsolved problems outside of the software realm. What's the plan for tackling stairs, variable height targets, manipulator dexterity, power supply, etc. in a general purpose robot with off-the-shelf parts? (Answering these questions may be part of that goal but maybe someone knows more on the subject.)

jhuckestein 10y ago
I wonder if OpenAI could define and build against a robot API that hardware robots would implement in different ways. That way OpenAI could focus on the software and various other teams could compete to solve all the mechanical challenges you mention.
nightski 10y ago
When it comes to AI/ML most problems seem "doable over a few years". Yet in practice this rarely is the case. These are some extremely lofty goals. The team behind OpenAI is quite capable but to say they'll have any of these done in a few years is quite a stretch. I'm guessing they may achieve one of the goals in a decade. But I'd love to be wrong.
- jimfleming 10y ago
  It was an off-hand remark. I'm aware of the landscape, though perhaps slightly more optimistic. The first goal is simple enough and largely underway with the Gym. Significant progress has been made on #3 and #4 just in the last year but I agree that "a few years" is a bit brief. I remain doubtful about #2.
ylem 10y ago
At least on stairs, I think this is a solvable problem: https://www.technologyreview.com/s/601240/an-impressive-walking-google-robot-tries-to-vacuum-the-stairs/ https://www.technologyreview.com/s/601240/an-impressive-walk... In terms of off-the-shelf: http://makezine.com/2015/05/01/meet-stair-bear-adorable-climbing-robot/ http://makezine.com/2015/05/01/meet-stair-bear-adorable-clim... I also vaguely recall seeing some demos of first responder robots and stairs. I don't know about the other issues you've raised...
tlb 10y ago
Off-the-shelf robots are convenient for developing learning algorithms. When the learning algorithms are good enough to take advantage of more capable hardware, it'll make sense to build it.
davmre 10y ago
Pieter Abbeel has been working towards household robots for quite a while; given his involvement and several of his students at OpenAI it's not surprising they'd be thinking along similar lines. The issues you mention are real, but current hardware is already capable of useful tasks if we had software to control it properly. For example, here's a demo of a tele-operated PR2 performing household chores: https://www.youtube.com/watch?v=S2GAz5F03Ls#t=60s https://www.youtube.com/watch?v=S2GAz5F03Ls#t=60s

JoshTriplett 10y ago

#4, solving multiple games with one agent, seems like a reasonably interesting step towards more general intelligence. An agent that can play a new game without any game-specific information other than "what can I do" and "what should I value" starts to sound a lot more like an agent that can solve non-game problems. Especially if that agent can play games that model real-world problems.

fitzwatermellow 10y ago

Goal 5: Inventing the Next Paradigm Any impetus into actually dreaming up what may come next after GPUs and Async DRL? Non-neural models, quantum computing based AI, optogenetic hacking ;) Otherwise, excellent list!

nxzero 10y ago

Is OpenAI willing to support true AI having basic rights to given to humans? If so, why is this not one of the fundamental technical goals? If not, why?

jimfleming 10y ago
You should have a look a PETRL[0]: People for the Ethical Treatment of Reinforcement Learners :) [0] http://petrl.org/ http://petrl.org/
- castis 10y ago
  That was way more of an interesting read that I thought it was going to be. Thanks for posting that.
JoshTriplett 10y ago
I would argue that we should avoid building sentient AI in the first place. Human-level or better problem-solving ability does not require the inclusion of sentience or independent agency.
- nxzero 10y ago
  Then I assume you're against any tech that would allow brains to be digitized, right?
  - JoshTriplett 10y ago
    Not at all; I'm incredibly in favor of doing so, and I would argue that a human mind running in a computer is still unambiguously human. The hard part comes in defining when a computer has reached that point. I'm arguing that we shouldn't develop computer AI that comes anywhere close to needing to answer that question; we don't need that to solve the problems that AI can solve.
    - nxzero 10y ago

ktta 10y ago

I find it really interesting that Goal 4 is a game playing agent. Deepmind has been focussing on this since the beginning[0] and actually has great progress as far as Atari games go[1]. And DeepMind is able to use a single agent rather than a different specific one for each game. I wonder if OpenAI wants to go in a different direction although RL has considerable success. Whereas the other goals definitely need a lot of work before they are "real-world" functional, especially Goal 3. But of course that depends on their definition of 'useful'. [0]:https://www.youtube.com/watch?v=rbsqaJwpu6A https://www.youtube.com/watch?v=rbsqaJwpu6A [1]:http://arxiv.org/pdf/1312.5602v1.pdf http://arxiv.org/pdf/1312.5602v1.pdf

deleted 10y ago
[deleted]
ratsimihah 10y ago
I feel like both companies are trying to "solve intelligence" by aiming for true AI, in which case there has to be some overlap in the work they're doing to get there.
argonaut 10y ago
You're mistaken, DeepMind uses a different agent for each game. What's the same across games is the learning method. But the model is trained separately for each game.
- ktta 10y ago
  I feel like I'm a bit confused at what constitutes a model. You basically have the same 'agent' to start off on the RL process. You can just group the learning agents together and make it (easier to tell it) automatically detect the game that has to be played. I guess I wanted to say that there isn't a lot different to be done when an agent is being trained for different games. And that makes it a general game playing agent, doesn't it?
  - argonaut 10y ago
    If you group the agents together and call it a single "general" agent, the research community (and pretty much everyone else) will call you out on your BS. That's the difference.

Houshalter 10y ago

I'm concerned that none of these goals involve AI safety. Instead all of their goals are nearly the exact opposite, of accelerating AI technology as much as possible. Safety was one of the main goals they promoted when it was founded. 2 of the 4 authors listed have publicly spoken about their belief of AI as an existential risk. I'm not saying that a game playing AI is going to take over the world. But it does demonstrate the risk - we still have no idea how to control such an AI. We can train it to get high scores. But it won't want to do anything other than get high scores. And it will do whatever it takes to get the highest score possible, even if it means exploiting the game or hurting other players, or disobeying it's masters. Now imagine they succeed in making smarter AIs. And their research spawns new research, which inspires new research, etc. Perhaps over several decades we could have AIs that are a lot more formidable than being able to play Pac Man. But we may still not have made any progress on the ability to control them.

astanway 10y ago
Almost no one who actually works or has done serious research in ML is genuinely concerned about "malevolent AI". We are so, so, so far away from anything remotely close to that. Please stop trying to gin up fear and listen to the experts, who uniformly agree that this is not something to be concerned about.
- ikeboy 10y ago
  Obligatory http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-risk/ http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri...
- crpatino 10y ago
  The issue is not "malevolent AI", the probability of producing an AI with "malevolent goals" by random chance is so much more unlikely than getting any AI at all that it's not even worth it to worry about that. The problem is "any AI" whose goals are not positively aligned-with/respectul-of human society's values. In terms with Asimov's Lay of Robotics meme, the problem is not robots harming humans, but robots allowing humans to come to harm as a side effect of their actions. This is an important ethical issue that, IMHO, technologist in general are failing to address. Unethical AIs are only a special case in the sense that they are believed to be able to cause much more chaos and destruction than spammers, patent trolls or high frequency trading firms.

feral 10y ago

Question: "Build a household robot" is high up the list. That doesn't seem inherently 'general'. Certainly, people have been working on that for years; there are all sorts of subproblems like vision, contextual reasoning etc. It could be treated as a general problem, requiring a lot of 'common sense'. But a team which sets out to optimize that particular goal, could spend years on relatively narrow tasks that get good performance returns on household chores (e.g. developing version 10 of the floor cleaning algorithm), but don't really make progress towards the problem of general intelligence. For me, what was really interesting about the benchmarks that Deepmind chose (the choice of a selection of Atari games) was that they were inherently somewhat general. Are you not worried that by putting a narrow domain fairly high up, you'll get distracted by narrow tasks, rather than making progress towards what's really interesting - generality? Won't it introduce tension to try and keep the general focus in the presence of a narrow goal, where you can get good returns by overfitting?

argonaut 10y ago
This isn't a particularly meaningful issue. The problem you describe of falling into the trap of brute-force optimizing a narrow task also applies to the Atari games. In fact it applies even more-so: it would be trivial for a lot of HN programmers to brute-force code an AI for challenging Atari games that deep learning still struggles against (like Montezuma's Revenge). But it would be nontrivial (difficult but still possible) to brute-force code a program for many narrow household robot tasks. You avoid this problem by... not hard-coding brute force solutions/heuristics! The research community can smell BS very easily (HN, not so much). A household robot is substantially (probably an order of magnitude) more general than Atari games, even for narrow tasks (obviously it is nowhere near the vicinity of the generality of AGI). The perception problem is tremendously more complex. The control/planning problem is similarly tremendously more complex.
- ktta 10y ago
  >But it would be nontrivial (difficult but still possible) to brute-force code a program for many narrow household robot tasks. How difficult are you talking about here? Similar to training game agents? I really doubt it. I feel like the training problem is VERY hard in case of real world random object handling (factory like fixed, mechanical situation can be purely hard coded and be much better than a human). In case of virtual games you can just use a bunch of GPUs and accelerate the process. But it is a much more difficult problem in reality. The grasping ability that we have with everyday objects is a marvel once you try to make a computer to do it. This might be of interest: http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/google-large-scale-robotic-grasping-project http://spectrum.ieee.org/automaton/robotics/artificial-intel...

lowglow 10y ago

Are the objectives of "OpenAI" conflicting with the interests of the startups applying to YC? OpenAI is building their own products/platforms and insights acquired from AI/Robotics startups applying and sharing information about what they're working on might be used as a competitive advantage. Is there an information firewall between what startups are sharing in the hope of investment and what is shared to advance OpenAI? If there is a firewall, how is it enforced?

mrfusion 10y ago

I wish they had mentioned asimovs three laws.