Interesting project! I've thought about this one before, mostly due to Jimmy Neutron. What arm are you using? Also what depth sensor? A few suggestions: It w

Interesting project! I've thought about this one before, mostly due to Jimmy Neutron. What arm are you using? Also what depth sensor? A few suggestions: It would appear that you're taking a pretty common approach to robot manipulation, sense, plan, "close your eyes" and then act. This works for stationary objects when you have high accuracy control and high accuracy sensing. Unless you're envisioning strapping people's heads down, your target is going to move. This will necessitate a closed-loop approach. A closed-loop approach here would benefit from force control at the end effector. ATI[1] is a good source for 6-axis load cells and they've recently come out with smaller form factor load cells. A second useful sensor would be a Time-of-Flight depth sensor. The old PMD Camboard Nano used to get up to 90 FPS and a close effective range (if I recall correctly ~3-4"). The time of flight sensors can be a bit noisy, but I've written up some of the smoothing approaches I've taken [2]. Good luck! [1] https://www.ati-ia.com/ https://www.ati-ia.com/ [2] http://www.cbames.com/publications http://www.cbames.com/publications