3 ms·

Instead of having only a single physical robot, what if you had a virtual (unity3d?) robot. I imagine you could speed up the learning process immensely if you d

Instead of having only a single physical robot, what if you had a virtual (unity3d?) robot. I imagine you could speed up the learning process immensely if you didn't have to have a ~200k robot actually manually performing the tasks. Just do the physics simulation and fake the sensors and other inputs. I'm sure that's a pain to set up, but probably way more effective and cheaper in the long run than trying to train a single PR2 This was done a long time ago on a smaller scale: http://www.demo.cs.brandeis.edu/pr/neural_controllers/evo_control.html http://www.demo.cs.brandeis.edu/pr/neural_controllers/evo_co... https://youtu.be/0fg1Jj8ehHs https://youtu.be/0fg1Jj8ehHs http://youtu.be/n6uMBveqbyE http://youtu.be/n6uMBveqbyE I'd love to see something like that tried with more modern computers / neural algorithms. Does anyone know places that are actually doing this?

PeterisP 11y ago
I'm not sure if we can do good enough physics simulations in real time so that the behavior of manipulator grip on e.g. a bottle cap and the stumbling of the bottle cap against the bottle as you push it around matches reality. Perhaps we can, but not with the common physics engines used in unity3d games and physics demos. In any case, dealing with the actual physical actuators and sensors, their limitations and feedback is the hard part of the problems - if you have a solution that works perfectly in a virtual environment, then IMHO you have achieved something like 10% of the progress needed to do the same in real life.
- swsieber 11y ago
  So what if you trained the robot to know when it needed real life training? Give it the option to train in a simulated environment or the real environment, but grade it on performance in the real environment. I bet it would try to do most things in the virtual and then later in the physical, since virtual training would (could.../should?) be faster. And it would be able to learn what sort of things in the virtual machine don't map well to real life. Heck, it might even learn to generally compensate for flaws in the 3d simulation.
- natosaichek 11y ago
  My guess is that you don't need something perfect to acheive 90% success. Maybe you switch to real-world physical training for the last 10%, but the first 90% (or whatever) of the neural network training - learning which 'mental' control corresponds with which 'muscle', getting the grip close to the right place with close to the right force, getting an approximation of sensor feedback into the network (with some noise added, perhaps) should get a long way there. I mean - you can see the terrible simulation they used for that asimo robot neural net training - not anywhere close to a unity simulator. I did just happen across this: http://gazebosim.org/ http://gazebosim.org/ Which seems like it's basically designed for these sorts of things.
- yongelee 11y ago
  https://www.youtube.com/watch?v=x8Fo2slT2WA https://www.youtube.com/watch?v=x8Fo2slT2WA Any 3d software is more than capable of creating perfect physics simulations in real time. I'm not sure what you mean by matching reality but in a physics simulation the only reality that matters is the virtual simulation, which can be done perfectly inside a program.
  - PeterisP 11y ago
    If the mind learns from the simulation that sending output X to the robotic hand will successfully screw a bottle cap on the bottle, but in real life it turns out that the same outputs result in the bottle cap slipping and falling out of the hand, then the simulation was not appropriate. You can do physics simulations that match some properties of reality that you are modeling. However, for robotics, you would need very detailed simulations of the exact hardware you'd be using, and it is very difficult to simulate the relevant aspects of it, a CAD drawing of the hand isn't enough - when some joint will wiggle slightly or the friction of a 'finger' surface will be different than you'd expect, then it will cause different behavior and the simulation will not match reality.

PeterisP 11y ago

I'm not sure if we can do good enough physics simulations in real time so that the behavior of manipulator grip on e.g. a bottle cap and the stumbling of the bottle cap against the bottle as you push it around matches reality. Perhaps we can, but not with the common physics engines used in unity3d games and physics demos. In any case, dealing with the actual physical actuators and sensors, their limitations and feedback is the hard part of the problems - if you have a solution that works perfectly in a virtual environment, then IMHO you have achieved something like 10% of the progress needed to do the same in real life.

swsieber 11y ago
So what if you trained the robot to know when it needed real life training? Give it the option to train in a simulated environment or the real environment, but grade it on performance in the real environment. I bet it would try to do most things in the virtual and then later in the physical, since virtual training would (could.../should?) be faster. And it would be able to learn what sort of things in the virtual machine don't map well to real life. Heck, it might even learn to generally compensate for flaws in the 3d simulation.
natosaichek 11y ago
My guess is that you don't need something perfect to acheive 90% success. Maybe you switch to real-world physical training for the last 10%, but the first 90% (or whatever) of the neural network training - learning which 'mental' control corresponds with which 'muscle', getting the grip close to the right place with close to the right force, getting an approximation of sensor feedback into the network (with some noise added, perhaps) should get a long way there. I mean - you can see the terrible simulation they used for that asimo robot neural net training - not anywhere close to a unity simulator. I did just happen across this: http://gazebosim.org/ http://gazebosim.org/ Which seems like it's basically designed for these sorts of things.
yongelee 11y ago
https://www.youtube.com/watch?v=x8Fo2slT2WA https://www.youtube.com/watch?v=x8Fo2slT2WA Any 3d software is more than capable of creating perfect physics simulations in real time. I'm not sure what you mean by matching reality but in a physics simulation the only reality that matters is the virtual simulation, which can be done perfectly inside a program.
- PeterisP 11y ago
  If the mind learns from the simulation that sending output X to the robotic hand will successfully screw a bottle cap on the bottle, but in real life it turns out that the same outputs result in the bottle cap slipping and falling out of the hand, then the simulation was not appropriate. You can do physics simulations that match some properties of reality that you are modeling. However, for robotics, you would need very detailed simulations of the exact hardware you'd be using, and it is very difficult to simulate the relevant aspects of it, a CAD drawing of the hand isn't enough - when some joint will wiggle slightly or the friction of a 'finger' surface will be different than you'd expect, then it will cause different behavior and the simulation will not match reality.