3 ms·

> I've said over and over that there are only two really hard problems in robotics: Perception and funding. A perfectly perceived system and world can be trivia

> I've said over and over that there are only two really hard problems in robotics: Perception and funding. A perfectly perceived system and world can be trivially planned for and (at least proprio-)controlled. Funding for sure. :) But as for perception, the inverse is also true. If I have an perfect planning/prediction system, I can throw the grungiest, worst perception data into it and it will still plan successfully despite tons of uncertainty. And therein lies the real challenge of robotics: It's fundamentally a systems engineering problem. You will never have perfect perception or a perfect planner. So, can you make a perception system that is good enough that, when coupled with your planning system which is good enough, you are able to solve enough problems with enough 9s to make it successful. The most commercially successful robots I've seen have had some of the smartest systems engineering behind them, such that entire classes of failures were eliminated by being smarter about what you actually need to do to solve the problem and aggressively avoid solving subproblems that aren't absolutely necessary. Only then do you really have a hope of getting good enough at that focused domain to ship something before the money runs out. :)

portaouflop 2y ago
> being smarter about what you actually need to do to solve the problem and aggressively avoid solving subproblems that aren't absolutely necessary I feel like this is true for every engineering discipline or maybe even every field that needs to operate in the real world
- vrighter 2y ago
  except software, of course. Nowadays it seems that software is all about creating problems to create solutions for.
krisoft 2y ago
> If I have an perfect planning/prediction system, I can throw the grungiest, worst perception data into it and it will still plan successfully despite tons of uncertainty. Not really. Even the perfect planning system will appear eratic in the presence of perception noise. It must be because it can’t create information out of nowhere. I have seen robots eratically stop because they thought that the traffic in the oncomming lane is enroaching on theirs. You can’t make the planning system ignore that because then sometimes it will collide with people playing chicken with you. Likewise I have seen robots eratically stop because they thought that a lamp post was slowly reversing out in front of them. All due to perception noise (in this case both location noise, and misclassification.) And do note that these are just the false positives. If you have a bad perception system you can also suffer from false negatives. Just experiment biases hide those. So your “perfect planning/prediction” will appear overly cautious while at the same time will be sometimes reckless. Because it doesn’t have the information to not to. You can’t magic plan your way out of that. (Unless you pipe the raw sensor data into the planner, in which case you created a second perception system you are just not calling it perception.)
- YeGoblynQueenne 2y ago
  >> (Unless you pipe the raw sensor data into the planner, in which case you created a second perception system you are just not calling it perception.) Like with model-free RL learning a model from pixels?
jvanderbot 2y ago
A "perfect" planning system which can handle arbitrarily bad perception is indistinguishable from a perception system. I've not seen a system that claimed to be robust to sensor noise that didn't do some filtering, estimation, or state representation internally. Those are just sensor systems inside the box.

portaouflop 2y ago

> being smarter about what you actually need to do to solve the problem and aggressively avoid solving subproblems that aren't absolutely necessary I feel like this is true for every engineering discipline or maybe even every field that needs to operate in the real world

vrighter 2y ago
except software, of course. Nowadays it seems that software is all about creating problems to create solutions for.

krisoft 2y ago

> If I have an perfect planning/prediction system, I can throw the grungiest, worst perception data into it and it will still plan successfully despite tons of uncertainty. Not really. Even the perfect planning system will appear eratic in the presence of perception noise. It must be because it can’t create information out of nowhere. I have seen robots eratically stop because they thought that the traffic in the oncomming lane is enroaching on theirs. You can’t make the planning system ignore that because then sometimes it will collide with people playing chicken with you. Likewise I have seen robots eratically stop because they thought that a lamp post was slowly reversing out in front of them. All due to perception noise (in this case both location noise, and misclassification.) And do note that these are just the false positives. If you have a bad perception system you can also suffer from false negatives. Just experiment biases hide those. So your “perfect planning/prediction” will appear overly cautious while at the same time will be sometimes reckless. Because it doesn’t have the information to not to. You can’t magic plan your way out of that. (Unless you pipe the raw sensor data into the planner, in which case you created a second perception system you are just not calling it perception.)

YeGoblynQueenne 2y ago
>> (Unless you pipe the raw sensor data into the planner, in which case you created a second perception system you are just not calling it perception.) Like with model-free RL learning a model from pixels?

jvanderbot 2y ago

A "perfect" planning system which can handle arbitrarily bad perception is indistinguishable from a perception system. I've not seen a system that claimed to be robust to sensor noise that didn't do some filtering, estimation, or state representation internally. Those are just sensor systems inside the box.