5 ms·

I love how bad robots are at picking stuff up. It's amazing how far science has advanced but the act of picking something up that hasn't been seen before is alm

I love how bad robots are at picking stuff up. It's amazing how far science has advanced but the act of picking something up that hasn't been seen before is almost impossible to do reliably. It really helps illustrate how AI is really not as powerful as people think it is.

krapp 9y ago
https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox It's not even just picking something up, it's picking one specific item out of an arbitrarily messy or overstuffed bin without dislodging or dropping anything else. Items in a bin can be wedged anywhere, in any position, in nondescript boxes only identifiable by the ASIN, or a vague natural language description. It's a hard visual logic, inference and manual dexterity problem made harder by unpredictability and harder still by the need to meet quotas, and every job has its own mess of them. And on top of that, employees in Amazon warehouses are often cross-trained to save costs, which means you either need one robot that can unload, decant, move juice carts, pick, stow, pallete-stow, etc, at a moment's notice or else multiple dedicated robots for each task. It's something humans do so easily that Amazon can justify paying almost nothing for the work, but still way beyond what AI and robotics can achieve. The Kiva pods seem to be the current state of the art for Amazon warehouses, and all they seem to do is move bins around (and can't even get that right sometimes.)
- joshvm 9y ago
  This is an important point. Grasping itself isn't that difficult if you know the object and its orientation. It's a pre-configured action (grip here and here, move, release). Ocado, a large online-only grocery store in the UK have a demo they show trade shows that uses a suction cup. They don't bother with a grasper, they just locate a big enough surface to latch onto and use that. That's considerably easier than picking things up using fingers and you can use fairly simple algorithms to detect planar surfaces. The problems start when things are tightly packed. One object might be in the way of your gripper so you can't take the thing you want. What happens if you drop the object? Does your robot know how to move things aside to grab the item you want? And on top of this you have the whole issue of detecting objects in a cluttered environment: this requires state of the art semantic segmentation, robust 3D vision that will work on non-cooperative targets and a classifier that can work with occlusions and huge numbers of objects reliably. I think it's reasonable that some items can be scanned upon entry into the warehouse, e.g. on a laser triangulation turntable that also grabs a video as it rotates. With this sort of task you want to exploit as many priors as you can find. If you look at papers (and factories), you see a lot of tricks: objects on flat surfaces that make segmentation easier, small numbers of known objects, etc.
  - wiz21c 9y ago
    >> The problems start when things are tightly packed. One object might be in the way of your gripper so you can't take the thing you want. Simple algorithm for that : Annoy (pun intended) towers :-)
  - krapp 9y ago
    >I think it's reasonable that some items can be scanned upon entry into the warehouse, e.g. on a laser triangulation turntable that also grabs a video as it rotates If it could be done quickly enough with enough volume, maybe, but the constraint is that items have to be loaded off the truck, scanned and stowed into a bin (which makes them available for purchase on the site) as quickly as possible.
    - joshvm 9y ago
      You could have a conveyor system which would cover 5/6 directions without too much trouble, and it'd be fast enough for real-time capture. Or you could automate the most common purchases e.g. Amazon basics items, Kindles, electronics. They could even ask sellers to provide models or dimensions.
  - amelius 9y ago
    > uses a suction cup I wonder how they pick up books using this method? Books are difficult because they appear to have hard surfaces on the side, which turn out to be not so hard and not suitable for gripping.
    - joshvm 9y ago
      Probably not an issue for them, Ocado sell pretty much only food so it'd be for boxes/bags mostly. It's not been deployed yet though. I think they're also testing gripper robots for picking up larger items like bottle packs. Also books, unless they're shrink wrapped, have a tendency to open which I imagine would be a pain.
    - sjg007 9y ago
      Well just sell Ebooks.
vedant 9y ago
Alternatively, it helps illustrate the complexity of the problems solved by natural selection over long periods of iteration.
- dibujante 9y ago
  Yeah. You (I assume, unless you are a dog on the internet) and I (definitely not a dog) are the result of something on the order of billions of iterations of the "grasping hand" problem.
jacquesm 9y ago
It's incredible how versatile our hands are. Simple pressure, two jawed chuck, three jawed chuck, four jawed chuck, roundgrip, tweezer grip, scoop and all that in a tiny, totally quiet package with force feedback across the whole surface. Mimicking that mechanically is a major challenge, it has nothing to do with AI per se.
- QAPereo 9y ago
  It starts to click when you think about how many years it takes us to form those basic skills, refine them, and then the years spent practicing specific applications. All of this still requires dedicated trainers in the form of parents, driven by love and commitment. It's not a trivial challenge in any form really.
  - davidreiss 9y ago
    Not even just on an individual basis. If you think about how long it took for dexterous hands to develop on an evolutionary basis, it took tens/hundreds of millions of years or even billions of years depending on where you start from. Also, there is an interesting theory that humans hands not only evolved to grab things, human hands evolved to form "efficient" fists for punching. http://www.latimes.com/science/sciencenow/la-sci-sn-human-fist-punching-evolution-males--20151021-story.html http://www.latimes.com/science/sciencenow/la-sci-sn-human-fi...
- deleted 9y ago
  [deleted]
- Govindae 9y ago
  I've seen AI learn to walk in simulation, I've never seen one learn to tie a shoe.
  - digi_owl 9y ago
    Because the developers have never seen the need...
Symmetry 9y ago
If you think you've got a really good solution for robotics grasping you shouldn't be competing in the Amazon Picking Challenge given the IP terms. We (Righthand Robotics) think we've got a handle on the problem but didn't want to get involved for that reason. There are some other companies out there with interesting solutions who weren't involved either. EDIT: Pun retroactively intended.
- amelius 9y ago
  > you shouldn't be competing in the Amazon Picking Challenge given the IP terms. This makes me wonder. Why did people compete for the Netflix prize, [1]? Were those IP terms better? [1] https://en.wikipedia.org/wiki/Netflix_Prize https://en.wikipedia.org/wiki/Netflix_Prize
  - mattnewton 9y ago
    Probably has to do with the different nature of the industries: robots versus software
  - Symmetry 9y ago
    People are competing in the Amazon Picking Challenge too! But the picking prize is just $50,000. If it were $1,000,000 like the Netflix prize I'm sure there would be more competition.
  - robotresearcher 9y ago
    The cost of participation is totally different. Physical robot vs. pure software.
deepnotderp 9y ago
Actually there are a couple of companies and labs who are close to (if not already solved) the problem, but no sane person is going to compete in the Amazon Picking Competition if they've actually solved it.
gozur88 9y ago
The difference with robots being it only has to be figured out once and the entire next generation of robots will be able to do it.

krapp 9y ago

https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox It's not even just picking something up, it's picking one specific item out of an arbitrarily messy or overstuffed bin without dislodging or dropping anything else. Items in a bin can be wedged anywhere, in any position, in nondescript boxes only identifiable by the ASIN, or a vague natural language description. It's a hard visual logic, inference and manual dexterity problem made harder by unpredictability and harder still by the need to meet quotas, and every job has its own mess of them. And on top of that, employees in Amazon warehouses are often cross-trained to save costs, which means you either need one robot that can unload, decant, move juice carts, pick, stow, pallete-stow, etc, at a moment's notice or else multiple dedicated robots for each task. It's something humans do so easily that Amazon can justify paying almost nothing for the work, but still way beyond what AI and robotics can achieve. The Kiva pods seem to be the current state of the art for Amazon warehouses, and all they seem to do is move bins around (and can't even get that right sometimes.)

joshvm 9y ago
This is an important point. Grasping itself isn't that difficult if you know the object and its orientation. It's a pre-configured action (grip here and here, move, release). Ocado, a large online-only grocery store in the UK have a demo they show trade shows that uses a suction cup. They don't bother with a grasper, they just locate a big enough surface to latch onto and use that. That's considerably easier than picking things up using fingers and you can use fairly simple algorithms to detect planar surfaces. The problems start when things are tightly packed. One object might be in the way of your gripper so you can't take the thing you want. What happens if you drop the object? Does your robot know how to move things aside to grab the item you want? And on top of this you have the whole issue of detecting objects in a cluttered environment: this requires state of the art semantic segmentation, robust 3D vision that will work on non-cooperative targets and a classifier that can work with occlusions and huge numbers of objects reliably. I think it's reasonable that some items can be scanned upon entry into the warehouse, e.g. on a laser triangulation turntable that also grabs a video as it rotates. With this sort of task you want to exploit as many priors as you can find. If you look at papers (and factories), you see a lot of tricks: objects on flat surfaces that make segmentation easier, small numbers of known objects, etc.
- wiz21c 9y ago
  >> The problems start when things are tightly packed. One object might be in the way of your gripper so you can't take the thing you want. Simple algorithm for that : Annoy (pun intended) towers :-)
- krapp 9y ago
  >I think it's reasonable that some items can be scanned upon entry into the warehouse, e.g. on a laser triangulation turntable that also grabs a video as it rotates If it could be done quickly enough with enough volume, maybe, but the constraint is that items have to be loaded off the truck, scanned and stowed into a bin (which makes them available for purchase on the site) as quickly as possible.
  - joshvm 9y ago
    You could have a conveyor system which would cover 5/6 directions without too much trouble, and it'd be fast enough for real-time capture. Or you could automate the most common purchases e.g. Amazon basics items, Kindles, electronics. They could even ask sellers to provide models or dimensions.
- amelius 9y ago
  > uses a suction cup I wonder how they pick up books using this method? Books are difficult because they appear to have hard surfaces on the side, which turn out to be not so hard and not suitable for gripping.
  - joshvm 9y ago
    Probably not an issue for them, Ocado sell pretty much only food so it'd be for boxes/bags mostly. It's not been deployed yet though. I think they're also testing gripper robots for picking up larger items like bottle packs. Also books, unless they're shrink wrapped, have a tendency to open which I imagine would be a pain.
  - sjg007 9y ago
    Well just sell Ebooks.

vedant 9y ago

Alternatively, it helps illustrate the complexity of the problems solved by natural selection over long periods of iteration.

dibujante 9y ago
Yeah. You (I assume, unless you are a dog on the internet) and I (definitely not a dog) are the result of something on the order of billions of iterations of the "grasping hand" problem.

jacquesm 9y ago

It's incredible how versatile our hands are. Simple pressure, two jawed chuck, three jawed chuck, four jawed chuck, roundgrip, tweezer grip, scoop and all that in a tiny, totally quiet package with force feedback across the whole surface. Mimicking that mechanically is a major challenge, it has nothing to do with AI per se.

QAPereo 9y ago
It starts to click when you think about how many years it takes us to form those basic skills, refine them, and then the years spent practicing specific applications. All of this still requires dedicated trainers in the form of parents, driven by love and commitment. It's not a trivial challenge in any form really.
- davidreiss 9y ago
  Not even just on an individual basis. If you think about how long it took for dexterous hands to develop on an evolutionary basis, it took tens/hundreds of millions of years or even billions of years depending on where you start from. Also, there is an interesting theory that humans hands not only evolved to grab things, human hands evolved to form "efficient" fists for punching. http://www.latimes.com/science/sciencenow/la-sci-sn-human-fist-punching-evolution-males--20151021-story.html http://www.latimes.com/science/sciencenow/la-sci-sn-human-fi...
deleted 9y ago
[deleted]
Govindae 9y ago
I've seen AI learn to walk in simulation, I've never seen one learn to tie a shoe.
- digi_owl 9y ago
  Because the developers have never seen the need...

Symmetry 9y ago

If you think you've got a really good solution for robotics grasping you shouldn't be competing in the Amazon Picking Challenge given the IP terms. We (Righthand Robotics) think we've got a handle on the problem but didn't want to get involved for that reason. There are some other companies out there with interesting solutions who weren't involved either. EDIT: Pun retroactively intended.

amelius 9y ago
> you shouldn't be competing in the Amazon Picking Challenge given the IP terms. This makes me wonder. Why did people compete for the Netflix prize, [1]? Were those IP terms better? [1] https://en.wikipedia.org/wiki/Netflix_Prize https://en.wikipedia.org/wiki/Netflix_Prize
- mattnewton 9y ago
  Probably has to do with the different nature of the industries: robots versus software
- Symmetry 9y ago
  People are competing in the Amazon Picking Challenge too! But the picking prize is just $50,000. If it were $1,000,000 like the Netflix prize I'm sure there would be more competition.
- robotresearcher 9y ago
  The cost of participation is totally different. Physical robot vs. pure software.

deepnotderp 9y ago

Actually there are a couple of companies and labs who are close to (if not already solved) the problem, but no sane person is going to compete in the Amazon Picking Competition if they've actually solved it.

gozur88 9y ago

The difference with robots being it only has to be figured out once and the entire next generation of robots will be able to do it.