3 ms·

Thank you! It's so good to see hype free discussion of this stuff. I would also add that using simulations to test (or train) robots is also not new, and has s

by 60654 9y ago

Thank you! It's so good to see hype free discussion of this stuff. I would also add that using simulations to test (or train) robots is also not new, and has significant limitations. Robotics has been using simulation since before the Grand Challenge, but what we find is that simulations make simplifying assumptions that don't bear out in real life: simulated sensors are more idealized than in real life, object recognition is idealized, actuators are idealized. But that makes it hard to apply any learnings to the real world, where sensors hallucinate and actuators lie all the time. And yeah, we _could_ be fuzzing the simulation and adding failure modes, but not many people do. It's hard to simulate these kinds of defects in a usefully realistic way.

theamk 9y ago
I work in robotics, and I meet people with this attitude all the time. This kind of thinking is OK when your robot is a small desktop toy, or a humanoid robot moving at 1 cm/second, but this approach will not work if your robots can damage expensive equipment or kill people. The key thing to understand is that your simulator is not a fixed product provided by a third-party. It grows organically with your codebase. So any time you encounter a bug in the real word, you have a three step process: - Create a test case to reproduce your bug in the simulation. This may involve updating your simulator to simulate the new physical properties we care about (such as new failure modes or new types of the sensor noise) - Fix your code, and make sure your test case passes - Add the test case to automatic testing suite and keep it there *forever*. this is the most important part. For an SDC company, the test suite is almost as important as the code itself. The time spent on writing testing code and test cases could be actually more than time spent writing the driving code itself. Unfortunately, many academic courses use an opposite approach: they use pre-made simulators, and do not modify them at all. This gives students the wrong expectation that you can find a simulator which can just simulate everything perfectly out-of-the-box; when this (predictably) fails, they become disappointed in the whole simulation idea.

by 60654 9y ago

theamk 9y ago

I work in robotics, and I meet people with this attitude all the time. This kind of thinking is OK when your robot is a small desktop toy, or a humanoid robot moving at 1 cm/second, but this approach will not work if your robots can damage expensive equipment or kill people. The key thing to understand is that your simulator is not a fixed product provided by a third-party. It grows organically with your codebase. So any time you encounter a bug in the real word, you have a three step process: - Create a test case to reproduce your bug in the simulation. This may involve updating your simulator to simulate the new physical properties we care about (such as new failure modes or new types of the sensor noise) - Fix your code, and make sure your test case passes - Add the test case to automatic testing suite and keep it there *forever*. this is the most important part. For an SDC company, the test suite is almost as important as the code itself. The time spent on writing testing code and test cases could be actually more than time spent writing the driving code itself. Unfortunately, many academic courses use an opposite approach: they use pre-made simulators, and do not modify them at all. This gives students the wrong expectation that you can find a simulator which can just simulate everything perfectly out-of-the-box; when this (predictably) fails, they become disappointed in the whole simulation idea.