5 ms·

One problem I encountered while working on robotics is that many commonly used algorithms yield approximate solutions with no error bounds. They work 99.99‰ of

• fluffy87 6y ago
  This is the scariest part of using machine learning as an engineer on any practical application as well. Without an error bound, ML can’t be in charge of anything that could put human lives at risk. This is also why I don’t understand all the hype about FSD / L5 autonomous driving. We don’t even know yet if such error bounds even exist, so we don’t even know if machine learning is even the right tool for FSD yet. All certification entities for control systems that put human lives at risk in aviation, automotive, etc. require those error bounds. So it actually doesn’t really matter if Tesla comes up with a “maybe L5” system, without right error bounds, their cars won’t be certified as L5 and drivers will need to keep hands on the steering wheel.
  • jaaron 6y ago
    What do you think the error bounds are for a human? I know it sounds like a flippant question, but for certain applications, if we can get a model that's better than human, then it doesn't need to be perfect. And they way we currently do this in all sorts of ways is to pair a human with a computer so that they each do what they're best at. It doesn't have to be about full automation.
    • deleted 6y ago
      [deleted]
    • de_watcher 6y ago
      Human is like your ancient software that was here since forever and somewhat worked fine. So everyone is used to it. (the difference from an actual software is that humans are based on some crazy nanotech from the future that nobody can completely control)
      • tomxor 6y ago
        > humans are based on some crazy nanotech from the future that nobody can completely control Excellent summary :D
    • tomp 6y ago
      This is another one of that "extreme tail risk" scenarios, like climate change and GMOs, that people have wildly different and contradictory reactions to. Sure, the "legacy" intelligence / climate / food could also have extreme tail risks, it's just that it's been tested for 100s of millenia... whereas new technology might be better in the average case or even 99th percentile, but the 1% (or 0.0001%) is unknown and potentially much worse. However, it seems to me that people resolve this more along ideological / political lines than with any kind of rational reasoning.
      • jnxx 6y ago
        > "extreme tail risk" scenarios, like climate change Climate change isn't a "tail risk". It is a hard wall our civilization is approaching fast. If we do not solve it, it will undo the conditions we depend on to live.
  • burntoutfire 6y ago
    What about "practical error bounds", i.e. testing the system through millions/billions of miles driven?
    • tuatoru 6y ago
      Miles driven is a useless metric. Stick your vehicle on a treadmill and have it drive a billion miles. What does that tell you? Edit to add: a better metric would be something like "billions of decisions made where human life was at stake".
      • gspr 6y ago
        It's pretty clear the they meant "distance driven in ordinary conditions where humans normally drive cars".
        • bluGill 6y ago
          That isn't the case though. Gm and Google both tell you that their incidents per mile (I think they use 100000 miles or something) is constant, because as they get better they test in harder situations. A automatic car on a desert freeway (no traffic) is easy compared to freezing rain during the afternoon rush hour in Minneapolis. (afternoon implying most people were out and so the decision to stay home isn't available). That is just on hard situation that comes to mind, I think those doing self driving cars know of others.
    • imtringued 6y ago
      Empirical evidence has shown that all Tesla cars have the same failure mode when a stationary obstacle is on the highway. This is not some fringe failure that happens because of wrong classification, it's how the Tesla Autopilot was designed to function. Tesla has claimed explicitly that it's the driver's responsibility to avoid such a situation.
      • azernik 6y ago
        Tesla is not a good representative of the field; they are waaaay behind Waymo (Google) and Cruise (GM).
  • m0zg 6y ago
    > Without an error bound, ML can’t be in charge of anything that could put human lives at risk Humans don't have "error bounds" either, and you trust them just fine.
    • Shared404 6y ago
      I personally do not trust humans. However, we do have inertia with humans running such situations, and until there is something provably/demonstrably better, I don't see the current situation changing.
      • imtringued 6y ago
        Trust is a continuum. I don't see how you can say that you do not trust humans at least to some extent, no matter how tiny that sliver of trust is.
        • Shared404 6y ago
          A good point. I do of course trust humans to an extent, but as a general rule I feel that it's been beaten into my head by enough experiences to expect for humans to mess something up. This includes myself of course.
    • foerbert 6y ago
      Humans are capable of generating and understanding creativity and complexity that are simply impossible for non-AGI automation. Even then, we don't just let people figure things out for themselves. We put them through training, and then test them. Even after that, we make them liable for negligence. I don't think it's an obvious conclusion that error bounds aren't important for automation because they aren't calculable for a human. They are just very different beasts.
      • m0zg 6y ago
        My point is not that it's not important. If someone comes up with a rigorous way to obtain error bars, that's great! I'll take it! My point is that trust should not and will not depend on it. How do you even quantify something like this to a layperson in order to persuade them? Let's do a thought experiment: let's say, we had a self-driving car that's verifiably 10x better than human on average, yet does not provide "error bars". I know we don't have one now, and unlikely to have one in the foreseeable future, but bear with me here, for the sake of argument. Would you trust it, rather than a random human Uber driver? FWIW, I'm amazed that driving cars manually is perceived as normal every time I drive one. I can easily accelerate 2+ metric tons of metal to 100+ MPH, and get distracted, launching this deadly projectile with me inside into oncoming traffic. Most roads have _no dividers_. This does happen from time to time, lots of people die. Nobody gives a shit. Humans suck so bad at so many things that robots will be better than them at a lot of fairly unconstrained tasks in the next decade or two. And I'm pretty certain they won't have error bars while doing what they do. Humans don't.
• iseanstevens 6y ago
  Though, probably humans fumble or bump things >1/10k times. I certainly do, anyways.
  • de_watcher 6y ago
    Also, replace deer with something self driving. Those things are so stupid when near the road.
• voqv 6y ago
  Afaik there is some progress being done on motion planning and control parts with regards to that, mostly using reachability analysis for continuous systems or using things like control barrier functions. I work in the perception/localization domain and I am not aware of any large developments in that direction. I do know that there are certain ML based perception systems that got some levels of ASIL certification.
• cpgxiii 6y ago
  Purely in the space of geometry and simple motion planning, there are reasonable methods with guaranteed correct solutions, provided their sensor data is correct. Of course, combining correctness (the solution is actually safe), completeness (a solution will be found if it exists), and bounded computation time (get a solution in actually useful time) is a much harder combination to achieve.
• mike_mg 6y ago
  I find the inverse surprising: that many algorithms that work on real-life robots _do_ _provide_ error bounds and their optimality / convergence properties are proven in the papers that introduce them. A great example of this is motion planning, where papers both on sample-based methods (such as SST), and on search based (descendants of the A* family) argue at length the theoretical optimality and convergence properties. On another note, I think requiring more theoretical analysis as a guarantee of safety could partially be an AI-winter meme rather than practical solution. Point in case: do people run a quick check of aerodynamics maths before boarding a flight? No - they rely mostly on the engineering and regulatory process that gradually made passenger flights safer.
  • helltone 6y ago
    It seems you are taking about theoretical error bounds, that is proofs in papers with assumptions on input probabilities etc. These don't always apply to actual implementations, in physical real robots. There is a huge gap in safety between practices in aerospace engineering and robotics.
• mihaaly 6y ago
  99.99% is pretty fine from the engineering point of view. The buildings and other constructions surrounds us have about the same theoretical reliability considering all the uncertainties involved like weather, load and impacts, long term characteristics, material and manufacturing uncertainties. Of course this centuries long trial and error supported pretty simple science of construction engineering needs to be supported by law level standards and regulations allowing certain low level uncertainty otherwise the 'engineers want to sleep at night' aspect was forcing us to make only very expensive bunkers to live in (or only sociopaths becoming engineers). There is an accepted level of risk involved in engineering. Usually mathematics and (proper) algorithms is the topic where everything is 100% (good quality actually finished work, not an early prototype released as final is assumed). It just may or may not to be fully relevant to our life.
  • marcosdumay 6y ago
    > The buildings and other constructions surrounds us have about the same theoretical reliability You are off by a few nines. Transportation machines are expected to have 5 or 6 of them. And those are the most dangerous kind we keep around. Everything else is more reliable.
    • mihaaly 6y ago
      No, I, am, not. It is about 99.998% Please do not cite numbers from different fields.
      • marcosdumay 6y ago
        By that number you mean that 2 in 10000 buildings fail at some point due to some design issue? Because building reliability is a very hard concept to define, even more in the context of machine reliability (we are discussing robots here). Anyway, machine reliability is calculated over usage, not lifetime. It gets much larger numbers.
• ragebol 6y ago
  Many localisation algorithms are probabilistic, eg. 'you are here within R of X'. A couple of times I've had a manager who could just not accept the probabilistic nature: "It's just right there!" So I to explain my manager that we just cannot do better and know for sure that the robot is really in position X, especially with the limited sensing the project would afford. Sure, you can do the classic AGV thing and add magnetic markers everywhere. Or use more sensors to get higher accuracy, but none of those were popular options.
  • de_watcher 6y ago
    Why bother, you'll hit the uncertainty principle sooner or later anyway.
• Symmetry 6y ago
  The solution generally employed is to separate humans and robots if the robots are moving at speeds or momentums that could harm a human. There are safety regulations to be complied with and the general idea is that your robot should not be able to harm a human no matter what error your software suffers from. This is sadly impossible with self driving cars but applies to most of robotics.