4 ms·

> Without an error bound, ML can’t be in charge of anything that could put human lives at risk Humans don't have "error bounds" either, and you trust them just

by m0zg 6y ago

> Without an error bound, ML can’t be in charge of anything that could put human lives at risk Humans don't have "error bounds" either, and you trust them just fine.

Shared404 6y ago
I personally do not trust humans. However, we do have inertia with humans running such situations, and until there is something provably/demonstrably better, I don't see the current situation changing.
- imtringued 6y ago
  Trust is a continuum. I don't see how you can say that you do not trust humans at least to some extent, no matter how tiny that sliver of trust is.
  - Shared404 6y ago
    A good point. I do of course trust humans to an extent, but as a general rule I feel that it's been beaten into my head by enough experiences to expect for humans to mess something up. This includes myself of course.
foerbert 6y ago
Humans are capable of generating and understanding creativity and complexity that are simply impossible for non-AGI automation. Even then, we don't just let people figure things out for themselves. We put them through training, and then test them. Even after that, we make them liable for negligence. I don't think it's an obvious conclusion that error bounds aren't important for automation because they aren't calculable for a human. They are just very different beasts.
- m0zg 6y ago
  My point is not that it's not important. If someone comes up with a rigorous way to obtain error bars, that's great! I'll take it! My point is that trust should not and will not depend on it. How do you even quantify something like this to a layperson in order to persuade them? Let's do a thought experiment: let's say, we had a self-driving car that's verifiably 10x better than human on average, yet does not provide "error bars". I know we don't have one now, and unlikely to have one in the foreseeable future, but bear with me here, for the sake of argument. Would you trust it, rather than a random human Uber driver? FWIW, I'm amazed that driving cars manually is perceived as normal every time I drive one. I can easily accelerate 2+ metric tons of metal to 100+ MPH, and get distracted, launching this deadly projectile with me inside into oncoming traffic. Most roads have _no dividers_. This does happen from time to time, lots of people die. Nobody gives a shit. Humans suck so bad at so many things that robots will be better than them at a lot of fairly unconstrained tasks in the next decade or two. And I'm pretty certain they won't have error bars while doing what they do. Humans don't.
  - de_watcher 6y ago
    A human whose life is at stake controlling a car here at this moment vs a human who is far in space and time controlling someone else's car by programming it.
  - YeGoblynQueenne 6y ago
    The other poster was talking about error bounds not error bars. That means they want to have a number that tells them how often a system fails and ideally a model of how and when it fails. So they want to have a system that comes with some guarantees. It works X% of the time, fails 100-X% of the time. Note that X doesn't need to be high, or higher than a human, for example a self-driving car with a 60% failure rate is fine as long as we know when and how it fails. It's this "when and how" that's completely missing from self driving cars. And in machine learning in general we just don't have any guarantees that a model trained on some dataset will perform the same way in the real world.
  - foerbert 6y ago
    I think a lot of the nuance has been lost here. Even throwing around phrases like "10x better than a human" is already implying a lot of very vague measurement or knowledge. In what ways, and in what environments, is it better? I've dipped my toes into robotics a few times, and every time I end up being reminded of just how painful it is. Even when you're working in an idealized simulator, it's extremely easy to find a little edge case that causes completely bizarre behavior. And moving out of the simulator only makes things far, far worse. It's really easy to forget about those sorts of details and brush it away as just things to be solved while developing the automation. But they don't just go away so easily. Error bounds are there to help manage these issues and ensure we know how to best use the automation. In many instances I think people would tend to prefer the Uber driver in a moderate reading of your scenario. A human driver is likely to perform somewhat consistently and predictably. If they drift around the corner and leave a long skid mark in front of your house you can make some assumptions about how they are going to drive. If you get in and see them struggling to keep their eyes open, you can again make some assumptions about their performance. A well-rested and safe driver is extremely unlikely to suddenly throw themselves into on-coming traffic with no warning. You can refuse or stop the ride if you judge you are not safe. Automation is a different beast. It's liable to fail in ways that a human driver would not. It may be performing wonderfully until something a human would not even notice happens, and which point it may indeed throw you into on-going traffic. For example, look at adversarial examples in deep learning. As a passenger in this case you don't have a way to judge your own moment-to-moment safety. Even if it is safer on average, the sheer unpredictability and the resulting stress is likely going to eat significantly into any gains.
imtringued 6y ago
Humans have a mental model of humans, they can predict what another human is going to do with reasonable accuracy. Robots are wholly artificial so there is no existing mental model we can apply to them to understand why and how they act.
- mhb 6y ago
  You have no mental model of the crazy person who drives his car into oncoming traffic.
  - tsimionescu 6y ago
    Sure, and that's why we try not to let people with altered states of consciousness or with unpredictable rationality (children) drive.
b3kart 6y ago
Two counterpoints here: 1) Humans can estimate their own uncertainty. Ask a person to show how long a meter is, and they'll give you an estimate. Then ask them to show you the "error bounds", i.e. what they're "quite certain" the meter is longer than and shorter than. You are likely to get sensible bounds. Now, humans aren't amazing at this, but the brain does have capacity for estimating how uncertain it is. 2) No, you don't really trust humans. This very fact that humans are often imperfect in estimating their own uncertainty makes us very stupid sometimes. How many times you were sure you know something for a fact, for it to turn out to be completely false. This is why society tries to not put too much responsibility into the hands of a single person, or at least to provide help and/or safety mechanisms if that is the case.
- m0zg 6y ago
  Ask a programmer to estimate how long it'll take them to code something to see if we really have error bars on complex functions. The margin will be so wide as to be completely useless. We do have error bars on simple measurements already. They're right there in the data sheet for the sensor. What you're asking for are error bars on things several levels distant in the layers of abstraction. Humans suck at that. We only cope the same way machines do: through constant negative feedback.
  - snovv_crash 6y ago
    Programmers can give you good error bounds. Ask what the best and worst case is for solving a problem, and 80% of the time they will be within those bounds. The problem is that management doesn't want to hear about the worst case, because it looks bad for them politically.
YeGoblynQueenne 6y ago
>> Humans don't have "error bounds" either, and you trust them just fine. Yes, but humans are already here and doing all those dangerous and difficult things. But if we're going to replace them with something, it makes sense to want to replace them with something that's better than them. I mean us. Otherwise, we might as well stick with the humans. Especially since we already know how to make humans (whereas self-driving cars, not so much).

by m0zg 6y ago

> Without an error bound, ML can’t be in charge of anything that could put human lives at risk Humans don't have "error bounds" either, and you trust them just fine.

Shared404 6y ago

I personally do not trust humans. However, we do have inertia with humans running such situations, and until there is something provably/demonstrably better, I don't see the current situation changing.

imtringued 6y ago
Trust is a continuum. I don't see how you can say that you do not trust humans at least to some extent, no matter how tiny that sliver of trust is.
- Shared404 6y ago
  A good point. I do of course trust humans to an extent, but as a general rule I feel that it's been beaten into my head by enough experiences to expect for humans to mess something up. This includes myself of course.

foerbert 6y ago

Humans are capable of generating and understanding creativity and complexity that are simply impossible for non-AGI automation. Even then, we don't just let people figure things out for themselves. We put them through training, and then test them. Even after that, we make them liable for negligence. I don't think it's an obvious conclusion that error bounds aren't important for automation because they aren't calculable for a human. They are just very different beasts.

m0zg 6y ago
My point is not that it's not important. If someone comes up with a rigorous way to obtain error bars, that's great! I'll take it! My point is that trust should not and will not depend on it. How do you even quantify something like this to a layperson in order to persuade them? Let's do a thought experiment: let's say, we had a self-driving car that's verifiably 10x better than human on average, yet does not provide "error bars". I know we don't have one now, and unlikely to have one in the foreseeable future, but bear with me here, for the sake of argument. Would you trust it, rather than a random human Uber driver? FWIW, I'm amazed that driving cars manually is perceived as normal every time I drive one. I can easily accelerate 2+ metric tons of metal to 100+ MPH, and get distracted, launching this deadly projectile with me inside into oncoming traffic. Most roads have _no dividers_. This does happen from time to time, lots of people die. Nobody gives a shit. Humans suck so bad at so many things that robots will be better than them at a lot of fairly unconstrained tasks in the next decade or two. And I'm pretty certain they won't have error bars while doing what they do. Humans don't.
- de_watcher 6y ago
  A human whose life is at stake controlling a car here at this moment vs a human who is far in space and time controlling someone else's car by programming it.
- YeGoblynQueenne 6y ago
  The other poster was talking about error bounds not error bars. That means they want to have a number that tells them how often a system fails and ideally a model of how and when it fails. So they want to have a system that comes with some guarantees. It works X% of the time, fails 100-X% of the time. Note that X doesn't need to be high, or higher than a human, for example a self-driving car with a 60% failure rate is fine as long as we know when and how it fails. It's this "when and how" that's completely missing from self driving cars. And in machine learning in general we just don't have any guarantees that a model trained on some dataset will perform the same way in the real world.
- foerbert 6y ago
  I think a lot of the nuance has been lost here. Even throwing around phrases like "10x better than a human" is already implying a lot of very vague measurement or knowledge. In what ways, and in what environments, is it better? I've dipped my toes into robotics a few times, and every time I end up being reminded of just how painful it is. Even when you're working in an idealized simulator, it's extremely easy to find a little edge case that causes completely bizarre behavior. And moving out of the simulator only makes things far, far worse. It's really easy to forget about those sorts of details and brush it away as just things to be solved while developing the automation. But they don't just go away so easily. Error bounds are there to help manage these issues and ensure we know how to best use the automation. In many instances I think people would tend to prefer the Uber driver in a moderate reading of your scenario. A human driver is likely to perform somewhat consistently and predictably. If they drift around the corner and leave a long skid mark in front of your house you can make some assumptions about how they are going to drive. If you get in and see them struggling to keep their eyes open, you can again make some assumptions about their performance. A well-rested and safe driver is extremely unlikely to suddenly throw themselves into on-coming traffic with no warning. You can refuse or stop the ride if you judge you are not safe. Automation is a different beast. It's liable to fail in ways that a human driver would not. It may be performing wonderfully until something a human would not even notice happens, and which point it may indeed throw you into on-going traffic. For example, look at adversarial examples in deep learning. As a passenger in this case you don't have a way to judge your own moment-to-moment safety. Even if it is safer on average, the sheer unpredictability and the resulting stress is likely going to eat significantly into any gains.

imtringued 6y ago

Humans have a mental model of humans, they can predict what another human is going to do with reasonable accuracy. Robots are wholly artificial so there is no existing mental model we can apply to them to understand why and how they act.

mhb 6y ago
You have no mental model of the crazy person who drives his car into oncoming traffic.
- tsimionescu 6y ago
  Sure, and that's why we try not to let people with altered states of consciousness or with unpredictable rationality (children) drive.

b3kart 6y ago

Two counterpoints here: 1) Humans can estimate their own uncertainty. Ask a person to show how long a meter is, and they'll give you an estimate. Then ask them to show you the "error bounds", i.e. what they're "quite certain" the meter is longer than and shorter than. You are likely to get sensible bounds. Now, humans aren't amazing at this, but the brain does have capacity for estimating how uncertain it is. 2) No, you don't really trust humans. This very fact that humans are often imperfect in estimating their own uncertainty makes us very stupid sometimes. How many times you were sure you know something for a fact, for it to turn out to be completely false. This is why society tries to not put too much responsibility into the hands of a single person, or at least to provide help and/or safety mechanisms if that is the case.

m0zg 6y ago
Ask a programmer to estimate how long it'll take them to code something to see if we really have error bars on complex functions. The margin will be so wide as to be completely useless. We do have error bars on simple measurements already. They're right there in the data sheet for the sensor. What you're asking for are error bars on things several levels distant in the layers of abstraction. Humans suck at that. We only cope the same way machines do: through constant negative feedback.
- snovv_crash 6y ago
  Programmers can give you good error bounds. Ask what the best and worst case is for solving a problem, and 80% of the time they will be within those bounds. The problem is that management doesn't want to hear about the worst case, because it looks bad for them politically.

YeGoblynQueenne 6y ago

>> Humans don't have "error bounds" either, and you trust them just fine. Yes, but humans are already here and doing all those dangerous and difficult things. But if we're going to replace them with something, it makes sense to want to replace them with something that's better than them. I mean us. Otherwise, we might as well stick with the humans. Especially since we already know how to make humans (whereas self-driving cars, not so much).