3 ms·

The hard part of robotics are the edge cases. The things that happen 0.1% of the time that will interfere with the automation promised by the robot. Simulation,

The hard part of robotics are the edge cases. The things that happen 0.1% of the time that will interfere with the automation promised by the robot. Simulation, especially when driven by RL, i.e., based on models learned from data, are extremely bad at predicting these edge cases. In fact, I would argue that if you already knew the edge cases and could simulate them, you wouldn't need to simulate them in first place. Besides that, I doubt that the fidelity of your simulation will even cover that. Anyone who's worked in robotics for a couple of years knows this and can share anecdotes. My favorite ones are sun reflections on the floor in a specific corridor at a specific customer at a specific time of day and season and direction of travel, that resulted in washing out the pattern projected on the ground by the 3D camera. This is not the kind of stuff you model upfront (unless you already know to worry about reflections, in which there is no need to model and simulate, just account for it in your design). This is only one of many examples.

jakubsuchanek 2y ago
Great point! Edge cases really are the bane of getting robotics deployed & profitable. Reflections caused us issues just as you describe :) ― we build & sold these robots https://www.youtube.com/watch?v=trrwJnDXX1k https://www.youtube.com/watch?v=trrwJnDXX1k now operating daily in Singapore. Still, even if we put aside edge cases, we believe we can save companies 1-2 years of development on the concept level. What do I mean? I see robotics development as mostly consecutive phases: ideate approach, develop physical POC, make it profitable. Failing at any point can send you to the start of the previous phase, or even to the complete beginning. We've spoken to multiple companies and think people spend too little time on the ideation and pay the price after developing the first POC ― or worse ― when unable to make it profitable. Currently, our aim is to help improve the ideation phase, before you settle on the specific approach. For example, Canvas, Conbotics and Nova Spray Tec. all do wall painting with plans to go further to all of wall finishing. Yet each of them has chosen a completely different mechanism of moving the end effector in 3D space. Canvas went mostly with off-the-shelf lifting platform + robotic arm. Conbotics combined 2 linear actuators with a rotary joint. Nova Spray went for an even simpler system with only a linear actuator and tilting sprayers. Each approach has concept-level trade-offs on the types of rooms they can geometrically cover, the component price-range and performance metrics like area covered per hour. From these you can calculate rough ROI. The differences between the approaches on this conceptual level are greater than differences due to edge cases, as they make 2-100x difference in the profitability. We had in-depth discussions with tens of robotics companies, both startups and corporate, and heard again and again how companies pick an approach A just to discover 1-3 years later they are 10x off from it being usable. That's what we're currently addressing. In our experience, all edge-cases are addressable either with more engineering, or small changes to the product application. But if the approach itself is wrong, you have to redesign the robot and throw away most of your solved edge-cases, just to get new ones :/ Back to solving edge cases: in the future, we expect to handle even the phases of developing physical POC and making it profitable. The line of research from ETH Zurich, where applying random forces, noise to sensors or turning sensors off altogether led to zero-shot sim2real transfer. Even better, it worked beyond 2 min demo, but for an hour long hike in reality nothing like the simulation: high grass blocking lidar, walking in sand, rocks breaking below the feet. This seems to me like a promising approach to handle edge-cases in simulation without directly modelling them. Do you have any thoughts on this line of research?

jakubsuchanek 2y ago

Great point! Edge cases really are the bane of getting robotics deployed & profitable. Reflections caused us issues just as you describe :) ― we build & sold these robots https://www.youtube.com/watch?v=trrwJnDXX1k https://www.youtube.com/watch?v=trrwJnDXX1k now operating daily in Singapore. Still, even if we put aside edge cases, we believe we can save companies 1-2 years of development on the concept level. What do I mean? I see robotics development as mostly consecutive phases: ideate approach, develop physical POC, make it profitable. Failing at any point can send you to the start of the previous phase, or even to the complete beginning. We've spoken to multiple companies and think people spend too little time on the ideation and pay the price after developing the first POC ― or worse ― when unable to make it profitable. Currently, our aim is to help improve the ideation phase, before you settle on the specific approach. For example, Canvas, Conbotics and Nova Spray Tec. all do wall painting with plans to go further to all of wall finishing. Yet each of them has chosen a completely different mechanism of moving the end effector in 3D space. Canvas went mostly with off-the-shelf lifting platform + robotic arm. Conbotics combined 2 linear actuators with a rotary joint. Nova Spray went for an even simpler system with only a linear actuator and tilting sprayers. Each approach has concept-level trade-offs on the types of rooms they can geometrically cover, the component price-range and performance metrics like area covered per hour. From these you can calculate rough ROI. The differences between the approaches on this conceptual level are greater than differences due to edge cases, as they make 2-100x difference in the profitability. We had in-depth discussions with tens of robotics companies, both startups and corporate, and heard again and again how companies pick an approach A just to discover 1-3 years later they are 10x off from it being usable. That's what we're currently addressing. In our experience, all edge-cases are addressable either with more engineering, or small changes to the product application. But if the approach itself is wrong, you have to redesign the robot and throw away most of your solved edge-cases, just to get new ones :/ Back to solving edge cases: in the future, we expect to handle even the phases of developing physical POC and making it profitable. The line of research from ETH Zurich, where applying random forces, noise to sensors or turning sensors off altogether led to zero-shot sim2real transfer. Even better, it worked beyond 2 min demo, but for an hour long hike in reality nothing like the simulation: high grass blocking lidar, walking in sand, rocks breaking below the feet. This seems to me like a promising approach to handle edge-cases in simulation without directly modelling them. Do you have any thoughts on this line of research?