> What could be the main cause for things being so hard to cross over from simulation to reality? When you are simulating you are leaving out of details. Some

> What could be the main cause for things being so hard to cross over from simulation to reality? When you are simulating you are leaving out of details. Some knowingly, some without you even being aware of them. Let’s imagine you are working on some perception algorithm for a lidar. You have a cool simulation using geometry to calculate how far each laser goes before they bounce back. Did you remember to implement that the lidar takes some amount of time to sweep around? It is not making instantenous global captures. (Which is a lot easier to code down.) This causes a kind of smearing effect particularly as things move fast relative to the sensor. Do you know if your sensor sweeps with one plane of lasers or multiple? What is the exact geometry? Are you simulating it as the sensor is designed or as it is built? Did you remember that the sun blinds the sensor? And if so, do you simulate it correctly? Including the algorithms it has internally to try to compensate for it? Did you know that retroreflective surfaces interact with the laser differently? In particular if there is moisture on the lidar. The light straying through water droplets cause false readings. Did you know that lidars can, under certain meteorological conditions see the exhaust of diesel vehicles? I hope you planed a fluid dynamics engine into your similation to account for the exhaust plume spreading upwards and expanding. And these are just the niggles of a particular sensor modality from the top of my head. I know similarly long lists for other sensor types. And then we didn’t even got into thermal issues, or the sensor aging or anything. And I bet reality has an even longer list than I do. Is any of this important in your domain? If yes you ignore them at your peril. If no, you will waste your time implementing them in your simulation. But the only way you will know for sure is when you stumble uppon them and they surprise you out of nowhere by breaking your assumptions and your code. > Should we invest some more in stochastic programming to handle all the little quirks that cannot be modeled perfectly? That is robotics. This is what people writing software for robots do. This is what researchers working on robotics problems are working on. If you want a nice and small example of this maybe read up onnKalman filters or the RANSAC algorithm. If you fancy a more academic treatment of the topic I recommend Probabilistic Robotics from Sebastian Thrun. > is the major issue that robots must work together with humans That certainly does not make it easier. Have you seen the video of the russian chess robot breaking its oponent’s finger?[1] That is just the least gruesome of the recent incidents that I can recall. How much is this your “main problem” depens on your problem domain though. 1: https://youtu.be/Mt-1smlom_M?si=19sOPkQf3s6JHoFo https://youtu.be/Mt-1smlom_M?si=19sOPkQf3s6JHoFo