4 ms·

Google Brain releases two large datasets for robotics research

230 points by hurrycane 10y ago

misiti3780 10y ago
Link to datasets: https://sites.google.com/site/brainrobotdata/home https://sites.google.com/site/brainrobotdata/home
epberry 10y ago
Awesome - I think this could be really useful for action recognition. Collecting large video datasets is really challenging and google's robot array is a great way to repeatedly create these kinds of custom datasets.
- irascible 10y ago
  This is the kind of stuff companies should get tax breaks for.
IshKebab 10y ago
Very cool, but isn't this a reinforcement learning task? Don't you need access to the machines to get them to learn?
- gwern 10y ago
  Only for some algorithms. Many RL algorithms can learn off-policy. Or you can treat it as a supervised problem: "given this image, predict the taken action". (Think of AlphaGo initially being trained to predict human players' next moves based on a large KGS corpus of games; no interaction or access to Go games required.) Admittedly, I'm not entirely sure what you would do with any of that after all the learning is done, if you don't eventually have one of those very expensive robot arms to use.
  - iofj 10y ago
    The paper talks about this. Answer: no it isn't, but it's a bit of a matter of definition. A reinforcement learning task will discount future actions, this won't. Also the training is what many people would call a supervised learning task : It works by training 2 functions, and those functions are very similar to Deep-Q learning functions: f(s, c, a) s = start image (before grasp attempt, robot in zero position), c = current image, a = possible robot command, predicts the odds that a will lead to a successful grasp. And while f could work by itself (ie. 10k random commands, evaluate f, pick the one with the highest odds of success), it doesn't. There's also a g(c) which tries to predict which commands f will give the best odds. You might wonder how it decides it's grasped something ? When doing nothing is >90% likely to result in a successful grasp it considers the grasp attempt finished (I assume there's also a time limit involved). There's also some method to trigger an abort. What I find particularly weird (although very much like in the Deep-Q learning papers) is that g essentially tries to predict f, and is mostly an optimization. You couldn't easily train g directly without ridiculous amounts of supervision (you'd need example successful grasps, potentially a lot of them). But to train f, you only need a few successful grasps, which you can just make happen by having a dumb action that will work 1/1000 times. Evaluating f is a huge problem (to many inputs need to be tried), so you couldn't use that function without g. And training g is something that can happen quickly offline. The brilliance is that this method takes a success signal, and generates supervised learning samples. Those are the easiest thing you could possibly train on, and so very useful. Combining f and g (on presumably a decent GPU) they can get a 90% grasp success rate on what looks like a motley collection of cutlery, pencils, a stapler, a nightlight, a few pens and (?) some lipstick, a few small boxes. But this is an "online" method, it should get better at it over time by itself. (God help the rest of us if Google decided to release their data before training fully converged because it takes too long)
- protomikron 10y ago
  As far as I know most robotics research is done offline, i.e. in a simulation (e.g. via gazebo, ROS, etc.). Obviously one has and does make the step to reality eventually (which is always messy), but you can go miles using simulation only. I would even say the learning curve from nothing (however being proficient in basic CS and programming) to successful demonstrations and experiments in a simulator is steeper, than from simulation to real robot. The first step is time and the second step money consuming.
  - kctess5 10y ago
    This is accurate. I've seen people use multiple levels of simulation complexity. That way they can learn the basics in a simple environment and use that as a prior for the more complex (and therefore slower/more costly to run) simulators. It's like bootstrapping up to the real thing. In general being able to run things in the simulator is critical to software testing. If your only method of doing integration testing and parameter optimization is the real robot, it will be a bad time, quite possibly leading to a crashed robot.
    - protomikron 10y ago
      I totally agree. Personal rant incoming: I think the ideas and intentions of ROS, gazebo and co. are great, but in my humble opinion things are a lot more complicated than they should be (as always in the world of software). First I think that the robotics world has a non-free vibe going on, in particular from industries who are big in the robotics building business. They mostly ship their own broken (sometimes Windows-only) API for their robot and it is mostly a mess. In particular they often do too much stuff, like providing proprietary shitty cartesian path planning or developing strange in-house communication protocols. I know that this is hard realtime territory and stuff is hard, but I feel definitely a tension between the mostly robotics researcher software crowd and the providers of the "real" thing, the actual robot, e.g. a manipulator, which is still in the 100.000 euros. Secondly I am afraid that ROS is going down the Corba path. The very fact that they ship their own build system (an ongoing debatte on the ros mailing list) rings alarm bells for me and I hope that they can move forward. However, kudos to the developers! Robotics is a hard and interdisciplinary topic, so one inherits all the problems of the software AND hardware world.
Apofis 10y ago
Damn, 800,000 grasp attempts? Pretty nuts that they need this many... but I guess without tactile feedback, only relying on visual feedback you need this many attempts to get it right.
- TulliusCicero 10y ago
  Do cutting-edge robots still not incorporate tactile feedback?
  - adrianratnapala 10y ago
    I am sure that many do have tactile feedback, but every touch switch or pressure sensor is a subsytem that you must decide is worth the added complexity. I doubt any simple approach works as well has human skin. As always, the thing that gets better fastest is the computing. The mechanics and the sensors improve too -- but according to a slower set of laws.
- samscully 10y ago
  Babies take a while to learn how to grasp things, it's believable that the number of attempts they make is in the 10,000s. 1 grasp attempt a minute, 8 hours a day for 3 months is 43,000 attempts. Of course they are learning a lot of other stuff at the same time so it's not really comparable.
  - united893 10y ago
    Well, it's a hard problem. Looking at the data, could you come up with a way to learn control more quickly? Ie, given a series of pixel intensities, could you send commands to each motor to get them to grasp something?
  - taneq 10y ago
    This is something that I think people often miss when they compare machine learning to human performance. Humans spend a LOT of time in their early learning and calibrating phase. Like, it's our full time job, 365 days a year, for several years. One interaction every ~5sec, for 12 hours a day, seems a modest estimate. That's over 3 million training examples per year.
AndrewKemendo 10y ago
I'm actually really excited about this because this is the kind of data that a tiny startup couldn't just get and helps people compete. I am still worried that nobody will be able to compete with the big guys long term in RL, simply because they have the money and access to build these huge training sets, but at least this gives someone somewhat of a chance. Kudos to Vincent, Sergey, Chelsea and Laura for promoting openness in ML research!!
- ericjang 10y ago
  The resources one has at his or her disposal dictate their solution space, and not necessarily the quality of the solution. Google is uniquely good at doing stuff at scale. Some day a grad student who doesn't have the money to buy a GPU might instead invent a form of sample-efficient RL that the big guys never even thought of :)

230 points by hurrycane 10y ago

Link to datasets: https://sites.google.com/site/brainrobotdata/home https://sites.google.com/site/brainrobotdata/home

Awesome - I think this could be really useful for action recognition. Collecting large video datasets is really challenging and google's robot array is a great way to repeatedly create these kinds of custom datasets.

irascible 10y ago
This is the kind of stuff companies should get tax breaks for.

IshKebab 10y ago

Very cool, but isn't this a reinforcement learning task? Don't you need access to the machines to get them to learn?

gwern 10y ago
Only for some algorithms. Many RL algorithms can learn off-policy. Or you can treat it as a supervised problem: "given this image, predict the taken action". (Think of AlphaGo initially being trained to predict human players' next moves based on a large KGS corpus of games; no interaction or access to Go games required.) Admittedly, I'm not entirely sure what you would do with any of that after all the learning is done, if you don't eventually have one of those very expensive robot arms to use.
- iofj 10y ago
  The paper talks about this. Answer: no it isn't, but it's a bit of a matter of definition. A reinforcement learning task will discount future actions, this won't. Also the training is what many people would call a supervised learning task : It works by training 2 functions, and those functions are very similar to Deep-Q learning functions: f(s, c, a) s = start image (before grasp attempt, robot in zero position), c = current image, a = possible robot command, predicts the odds that a will lead to a successful grasp. And while f could work by itself (ie. 10k random commands, evaluate f, pick the one with the highest odds of success), it doesn't. There's also a g(c) which tries to predict which commands f will give the best odds. You might wonder how it decides it's grasped something ? When doing nothing is >90% likely to result in a successful grasp it considers the grasp attempt finished (I assume there's also a time limit involved). There's also some method to trigger an abort. What I find particularly weird (although very much like in the Deep-Q learning papers) is that g essentially tries to predict f, and is mostly an optimization. You couldn't easily train g directly without ridiculous amounts of supervision (you'd need example successful grasps, potentially a lot of them). But to train f, you only need a few successful grasps, which you can just make happen by having a dumb action that will work 1/1000 times. Evaluating f is a huge problem (to many inputs need to be tried), so you couldn't use that function without g. And training g is something that can happen quickly offline. The brilliance is that this method takes a success signal, and generates supervised learning samples. Those are the easiest thing you could possibly train on, and so very useful. Combining f and g (on presumably a decent GPU) they can get a 90% grasp success rate on what looks like a motley collection of cutlery, pencils, a stapler, a nightlight, a few pens and (?) some lipstick, a few small boxes. But this is an "online" method, it should get better at it over time by itself. (God help the rest of us if Google decided to release their data before training fully converged because it takes too long)
protomikron 10y ago
As far as I know most robotics research is done offline, i.e. in a simulation (e.g. via gazebo, ROS, etc.). Obviously one has and does make the step to reality eventually (which is always messy), but you can go miles using simulation only. I would even say the learning curve from nothing (however being proficient in basic CS and programming) to successful demonstrations and experiments in a simulator is steeper, than from simulation to real robot. The first step is time and the second step money consuming.
- kctess5 10y ago
  This is accurate. I've seen people use multiple levels of simulation complexity. That way they can learn the basics in a simple environment and use that as a prior for the more complex (and therefore slower/more costly to run) simulators. It's like bootstrapping up to the real thing. In general being able to run things in the simulator is critical to software testing. If your only method of doing integration testing and parameter optimization is the real robot, it will be a bad time, quite possibly leading to a crashed robot.
  - protomikron 10y ago
    I totally agree. Personal rant incoming: I think the ideas and intentions of ROS, gazebo and co. are great, but in my humble opinion things are a lot more complicated than they should be (as always in the world of software). First I think that the robotics world has a non-free vibe going on, in particular from industries who are big in the robotics building business. They mostly ship their own broken (sometimes Windows-only) API for their robot and it is mostly a mess. In particular they often do too much stuff, like providing proprietary shitty cartesian path planning or developing strange in-house communication protocols. I know that this is hard realtime territory and stuff is hard, but I feel definitely a tension between the mostly robotics researcher software crowd and the providers of the "real" thing, the actual robot, e.g. a manipulator, which is still in the 100.000 euros. Secondly I am afraid that ROS is going down the Corba path. The very fact that they ship their own build system (an ongoing debatte on the ros mailing list) rings alarm bells for me and I hope that they can move forward. However, kudos to the developers! Robotics is a hard and interdisciplinary topic, so one inherits all the problems of the software AND hardware world.

Apofis 10y ago

Damn, 800,000 grasp attempts? Pretty nuts that they need this many... but I guess without tactile feedback, only relying on visual feedback you need this many attempts to get it right.

TulliusCicero 10y ago
Do cutting-edge robots still not incorporate tactile feedback?
- adrianratnapala 10y ago
  I am sure that many do have tactile feedback, but every touch switch or pressure sensor is a subsytem that you must decide is worth the added complexity. I doubt any simple approach works as well has human skin. As always, the thing that gets better fastest is the computing. The mechanics and the sensors improve too -- but according to a slower set of laws.
samscully 10y ago
Babies take a while to learn how to grasp things, it's believable that the number of attempts they make is in the 10,000s. 1 grasp attempt a minute, 8 hours a day for 3 months is 43,000 attempts. Of course they are learning a lot of other stuff at the same time so it's not really comparable.
- united893 10y ago
  Well, it's a hard problem. Looking at the data, could you come up with a way to learn control more quickly? Ie, given a series of pixel intensities, could you send commands to each motor to get them to grasp something?
- taneq 10y ago
  This is something that I think people often miss when they compare machine learning to human performance. Humans spend a LOT of time in their early learning and calibrating phase. Like, it's our full time job, 365 days a year, for several years. One interaction every ~5sec, for 12 hours a day, seems a modest estimate. That's over 3 million training examples per year.

AndrewKemendo 10y ago

I'm actually really excited about this because this is the kind of data that a tiny startup couldn't just get and helps people compete. I am still worried that nobody will be able to compete with the big guys long term in RL, simply because they have the money and access to build these huge training sets, but at least this gives someone somewhat of a chance. Kudos to Vincent, Sergey, Chelsea and Laura for promoting openness in ML research!!

ericjang 10y ago
The resources one has at his or her disposal dictate their solution space, and not necessarily the quality of the solution. Google is uniquely good at doing stuff at scale. Some day a grad student who doesn't have the money to buy a GPU might instead invent a form of sample-efficient RL that the big guys never even thought of :)