6 ms·

This is the scariest part of using machine learning as an engineer on any practical application as well. Without an error bound, ML can’t be in charge of anyth

This is the scariest part of using machine learning as an engineer on any practical application as well. Without an error bound, ML can’t be in charge of anything that could put human lives at risk. This is also why I don’t understand all the hype about FSD / L5 autonomous driving. We don’t even know yet if such error bounds even exist, so we don’t even know if machine learning is even the right tool for FSD yet. All certification entities for control systems that put human lives at risk in aviation, automotive, etc. require those error bounds. So it actually doesn’t really matter if Tesla comes up with a “maybe L5” system, without right error bounds, their cars won’t be certified as L5 and drivers will need to keep hands on the steering wheel.

jaaron 6y ago
What do you think the error bounds are for a human? I know it sounds like a flippant question, but for certain applications, if we can get a model that's better than human, then it doesn't need to be perfect. And they way we currently do this in all sorts of ways is to pair a human with a computer so that they each do what they're best at. It doesn't have to be about full automation.
- deleted 6y ago
  [deleted]
- de_watcher 6y ago
  Human is like your ancient software that was here since forever and somewhat worked fine. So everyone is used to it. (the difference from an actual software is that humans are based on some crazy nanotech from the future that nobody can completely control)
  - tomxor 6y ago
    > humans are based on some crazy nanotech from the future that nobody can completely control Excellent summary :D
- tomp 6y ago
  This is another one of that "extreme tail risk" scenarios, like climate change and GMOs, that people have wildly different and contradictory reactions to. Sure, the "legacy" intelligence / climate / food could also have extreme tail risks, it's just that it's been tested for 100s of millenia... whereas new technology might be better in the average case or even 99th percentile, but the 1% (or 0.0001%) is unknown and potentially much worse. However, it seems to me that people resolve this more along ideological / political lines than with any kind of rational reasoning.
  - jnxx 6y ago
    > "extreme tail risk" scenarios, like climate change Climate change isn't a "tail risk". It is a hard wall our civilization is approaching fast. If we do not solve it, it will undo the conditions we depend on to live.
burntoutfire 6y ago
What about "practical error bounds", i.e. testing the system through millions/billions of miles driven?
- tuatoru 6y ago
  Miles driven is a useless metric. Stick your vehicle on a treadmill and have it drive a billion miles. What does that tell you? Edit to add: a better metric would be something like "billions of decisions made where human life was at stake".
  - gspr 6y ago
    It's pretty clear the they meant "distance driven in ordinary conditions where humans normally drive cars".
    - bluGill 6y ago
      That isn't the case though. Gm and Google both tell you that their incidents per mile (I think they use 100000 miles or something) is constant, because as they get better they test in harder situations. A automatic car on a desert freeway (no traffic) is easy compared to freezing rain during the afternoon rush hour in Minneapolis. (afternoon implying most people were out and so the decision to stay home isn't available). That is just on hard situation that comes to mind, I think those doing self driving cars know of others.
- imtringued 6y ago
  Empirical evidence has shown that all Tesla cars have the same failure mode when a stationary obstacle is on the highway. This is not some fringe failure that happens because of wrong classification, it's how the Tesla Autopilot was designed to function. Tesla has claimed explicitly that it's the driver's responsibility to avoid such a situation.
  - azernik 6y ago
    Tesla is not a good representative of the field; they are waaaay behind Waymo (Google) and Cruise (GM).
m0zg 6y ago
> Without an error bound, ML can’t be in charge of anything that could put human lives at risk Humans don't have "error bounds" either, and you trust them just fine.
- Shared404 6y ago
  I personally do not trust humans. However, we do have inertia with humans running such situations, and until there is something provably/demonstrably better, I don't see the current situation changing.
  - imtringued 6y ago
    Trust is a continuum. I don't see how you can say that you do not trust humans at least to some extent, no matter how tiny that sliver of trust is.
    - Shared404 6y ago
      A good point. I do of course trust humans to an extent, but as a general rule I feel that it's been beaten into my head by enough experiences to expect for humans to mess something up. This includes myself of course.
- foerbert 6y ago
  Humans are capable of generating and understanding creativity and complexity that are simply impossible for non-AGI automation. Even then, we don't just let people figure things out for themselves. We put them through training, and then test them. Even after that, we make them liable for negligence. I don't think it's an obvious conclusion that error bounds aren't important for automation because they aren't calculable for a human. They are just very different beasts.
  - m0zg 6y ago
    My point is not that it's not important. If someone comes up with a rigorous way to obtain error bars, that's great! I'll take it! My point is that trust should not and will not depend on it. How do you even quantify something like this to a layperson in order to persuade them? Let's do a thought experiment: let's say, we had a self-driving car that's verifiably 10x better than human on average, yet does not provide "error bars". I know we don't have one now, and unlikely to have one in the foreseeable future, but bear with me here, for the sake of argument. Would you trust it, rather than a random human Uber driver? FWIW, I'm amazed that driving cars manually is perceived as normal every time I drive one. I can easily accelerate 2+ metric tons of metal to 100+ MPH, and get distracted, launching this deadly projectile with me inside into oncoming traffic. Most roads have _no dividers_. This does happen from time to time, lots of people die. Nobody gives a shit. Humans suck so bad at so many things that robots will be better than them at a lot of fairly unconstrained tasks in the next decade or two. And I'm pretty certain they won't have error bars while doing what they do. Humans don't.

jaaron 6y ago

What do you think the error bounds are for a human? I know it sounds like a flippant question, but for certain applications, if we can get a model that's better than human, then it doesn't need to be perfect. And they way we currently do this in all sorts of ways is to pair a human with a computer so that they each do what they're best at. It doesn't have to be about full automation.

deleted 6y ago
[deleted]
de_watcher 6y ago
Human is like your ancient software that was here since forever and somewhat worked fine. So everyone is used to it. (the difference from an actual software is that humans are based on some crazy nanotech from the future that nobody can completely control)
- tomxor 6y ago
  > humans are based on some crazy nanotech from the future that nobody can completely control Excellent summary :D
tomp 6y ago
This is another one of that "extreme tail risk" scenarios, like climate change and GMOs, that people have wildly different and contradictory reactions to. Sure, the "legacy" intelligence / climate / food could also have extreme tail risks, it's just that it's been tested for 100s of millenia... whereas new technology might be better in the average case or even 99th percentile, but the 1% (or 0.0001%) is unknown and potentially much worse. However, it seems to me that people resolve this more along ideological / political lines than with any kind of rational reasoning.
- jnxx 6y ago
  > "extreme tail risk" scenarios, like climate change Climate change isn't a "tail risk". It is a hard wall our civilization is approaching fast. If we do not solve it, it will undo the conditions we depend on to live.

burntoutfire 6y ago

What about "practical error bounds", i.e. testing the system through millions/billions of miles driven?

tuatoru 6y ago
Miles driven is a useless metric. Stick your vehicle on a treadmill and have it drive a billion miles. What does that tell you? Edit to add: a better metric would be something like "billions of decisions made where human life was at stake".
- gspr 6y ago
  It's pretty clear the they meant "distance driven in ordinary conditions where humans normally drive cars".
  - bluGill 6y ago
    That isn't the case though. Gm and Google both tell you that their incidents per mile (I think they use 100000 miles or something) is constant, because as they get better they test in harder situations. A automatic car on a desert freeway (no traffic) is easy compared to freezing rain during the afternoon rush hour in Minneapolis. (afternoon implying most people were out and so the decision to stay home isn't available). That is just on hard situation that comes to mind, I think those doing self driving cars know of others.
imtringued 6y ago
Empirical evidence has shown that all Tesla cars have the same failure mode when a stationary obstacle is on the highway. This is not some fringe failure that happens because of wrong classification, it's how the Tesla Autopilot was designed to function. Tesla has claimed explicitly that it's the driver's responsibility to avoid such a situation.
- azernik 6y ago
  Tesla is not a good representative of the field; they are waaaay behind Waymo (Google) and Cruise (GM).

m0zg 6y ago

> Without an error bound, ML can’t be in charge of anything that could put human lives at risk Humans don't have "error bounds" either, and you trust them just fine.

Shared404 6y ago
I personally do not trust humans. However, we do have inertia with humans running such situations, and until there is something provably/demonstrably better, I don't see the current situation changing.
- imtringued 6y ago
  Trust is a continuum. I don't see how you can say that you do not trust humans at least to some extent, no matter how tiny that sliver of trust is.
  - Shared404 6y ago
    A good point. I do of course trust humans to an extent, but as a general rule I feel that it's been beaten into my head by enough experiences to expect for humans to mess something up. This includes myself of course.
foerbert 6y ago
Humans are capable of generating and understanding creativity and complexity that are simply impossible for non-AGI automation. Even then, we don't just let people figure things out for themselves. We put them through training, and then test them. Even after that, we make them liable for negligence. I don't think it's an obvious conclusion that error bounds aren't important for automation because they aren't calculable for a human. They are just very different beasts.
- m0zg 6y ago
  My point is not that it's not important. If someone comes up with a rigorous way to obtain error bars, that's great! I'll take it! My point is that trust should not and will not depend on it. How do you even quantify something like this to a layperson in order to persuade them? Let's do a thought experiment: let's say, we had a self-driving car that's verifiably 10x better than human on average, yet does not provide "error bars". I know we don't have one now, and unlikely to have one in the foreseeable future, but bear with me here, for the sake of argument. Would you trust it, rather than a random human Uber driver? FWIW, I'm amazed that driving cars manually is perceived as normal every time I drive one. I can easily accelerate 2+ metric tons of metal to 100+ MPH, and get distracted, launching this deadly projectile with me inside into oncoming traffic. Most roads have _no dividers_. This does happen from time to time, lots of people die. Nobody gives a shit. Humans suck so bad at so many things that robots will be better than them at a lot of fairly unconstrained tasks in the next decade or two. And I'm pretty certain they won't have error bars while doing what they do. Humans don't.