4 ms·

This depends almost entirely on the accuracy of the simulated environment relative to the real world. If we have very accurate simulators then sim2real isn't to

This depends almost entirely on the accuracy of the simulated environment relative to the real world. If we have very accurate simulators then sim2real isn't too bad. If we have poor simulators (or simulators which don't fully capture that <0.1% probability action that happens to have REALLY bad consequences...) then sim2real doesn't work. Flying a drone through a static forrest in no-wind (no moving branches, etc.) seems like a reasonable sim2real environment, in part because... well... video games have had forrests and trees don't tend to move very fast unless there's wind. But sim2real for self driving cars is a lot harder given the huge diversity in streets and freak events that happen (people, bicycles, shopping carts, piano-delivery-trucks) - Twitter: @iamtrask

adenverd 5y ago
I'll add that real world sensors (including cameras) are noisy in ways that are difficult to model 100% accurately in simulation. Same thing with aerodynamic effects, especially when the drone gets close to the ground or other objects, or y'know...it's windy outside. Dealing with these types of noise and unpredictably and building in resilience to them is a big part of testing and validating systems like these, and can only happen in the real world. I'm pretty confident the drone linked in this article doesn't have high reliability. I'd be shocked if they managed to get >70% critical without significant real-world iteration.
- apendleton 5y ago
  It looks from the video someone linked to above that these things don't use cameras, just depth sensors -- maybe they're easier to simulate? It looks like they do introduce noise into their simulated depth-sensor data, but maybe that's more straightforward than creating realistic visual noise.
- meheleventyone 5y ago
  I think it basically helps that quadcopters are already computer stabilized and highly manoeuvrable so the problem is more identifying spaces to move into and motion planning through them. What’s impressive since the last time this kind of thing was posted is the increase in speed and at least in the provided video the ability to avoid really quite small branches.

adenverd 5y ago

I'll add that real world sensors (including cameras) are noisy in ways that are difficult to model 100% accurately in simulation. Same thing with aerodynamic effects, especially when the drone gets close to the ground or other objects, or y'know...it's windy outside. Dealing with these types of noise and unpredictably and building in resilience to them is a big part of testing and validating systems like these, and can only happen in the real world. I'm pretty confident the drone linked in this article doesn't have high reliability. I'd be shocked if they managed to get >70% critical without significant real-world iteration.

apendleton 5y ago
It looks from the video someone linked to above that these things don't use cameras, just depth sensors -- maybe they're easier to simulate? It looks like they do introduce noise into their simulated depth-sensor data, but maybe that's more straightforward than creating realistic visual noise.
meheleventyone 5y ago
I think it basically helps that quadcopters are already computer stabilized and highly manoeuvrable so the problem is more identifying spaces to move into and motion planning through them. What’s impressive since the last time this kind of thing was posted is the increase in speed and at least in the provided video the ability to avoid really quite small branches.