6 ms·

The real problem with self-driving cars is the car to human handoff. Over the long term it's incredible unlikely that humans will be any good at all at remainin

by TrevorJ 11y ago

The real problem with self-driving cars is the car to human handoff. Over the long term it's incredible unlikely that humans will be any good at all at remaining aware and 'up to speed' on the current situation in the event that the car needs to give control back to the driver due to road conditions, hardware failure, or sudden situation that it cannot contend with.

adekok 11y ago
Normal desktop failure modes don't work well here. "Hi, this is your car. I've discovered that the road is icy, and am currently in the middle of a 360. Would you like to (a) take over, (b) call an ambulance, or (c) retry?" Too late! You've crashed.
- mabbo 11y ago
  Bonus points if it's Clippy, but he's a steering wheel now.
- mchahn 11y ago
  > and am currently in the middle of a 360. You have no faith in AI. The car will always be better at driving than the human. It would be much better at handling a 360 than a human ever could. Actually humans are very bad at handling 360s unless you are a pro.
  - timClicks 11y ago
    As long as the sensors, networking, compute and actuators are all functional.
  - Alupis 11y ago
    Humans are very, very good at handling unknown complex situations. In order for any AI to handle any situation, it must first be programmed to do so. If that situation was an unknown, you get undefined behavior. Undefined behavior in a car traveling down the road is not a good thing to place a bet on... especially your life. This is why majority of sane driverless vehicle talk has concluded a human driver must always be in the driver seat, ready to take control. There's an impossible volume of failure modes a driverless car must contend with, everything from someone pointing an infrared laser at the distance sensors, to complete power failure, to a sudden rockslide, a bird darting in front of the car, lines missing from a windy and hilly road, etc. In normal, daily mundane driving - driverless cars are expected to excel. Sitting in traffic is mind numbing, and mistakes happen when driver's minds wonder. In "active" driving situations, a human brain will almost always excel.
    - mikeash 11y ago
      Humans are mostly terrible at handling unknown complex situations on the road. For many human drivers on the roads right now, the response to a sudden weird situation is either going to be "brake heavily" or "swerve dangerously." You don't have to get "undefined behavior" in response to an unknown situation. The AI can simply default to braking to a stop in the absence of any better programmed alternative. This will already put it on par with most drivers. Better, in fact, because the AI can begin braking instantly. I personally find that the majority of driverless vehicle talk vastly overestimates the average human driver, and demands unreasonable perfection from the computers.
      - jzwinck 11y ago
        Not only can a computer begin braking more promptly, it can brake more accurately. Many human drivers hit the brakes 100% when they sense danger, then the full force of braking startles them (no training), then they let off again. This can increase stopping distances dramatically. Next up is per-wheel braking. Not just stability control, but actual directed thrust. The computer can control each wheel independently. Humans just have one brake pedal. Four wheel steering might even make a comeback.
      - beat 11y ago
        Remember the two second rule? Instantaneous response would be a huge improvement in many situations already.
        obsurveyor 11y ago
        With today's aggressive and impatient drivers, the 2 second rule is very hard to maintain because they'll just pass you and fill the spot, every time.
        mikeash 11y ago
        I see this stated a lot, but I think it's exaggerated. My car's adaptive cruise control follows at about 2.5 seconds when on the maximum setting, which is where I almost always keep it, including in heavy traffic. People do occasionally jump in, but then the car just backs off a bit and starts following the new car at 2.5 seconds. It's not a constant churn, and it doesn't cause any significant delay. And this is in the DC area, with dense traffic and ridiculous drivers. When people say that it doesn't work, I really think they're just saying that they can't stand having other people get ahead by breaking the rules, and they'd rather drive more dangerously than let that happen.
  - TrevorJ 11y ago
    >The car will always be better at driving than the human. I don't think this is true yet. We can interpret the incoming data incredibly well in ways the machine vision can't yet match. This is absolutely vital for driving. The difference between an empty trash bag in the road and a toddler in the road is a matter of vital importance, and correctly identifying the difference matters a lot. Humans area actually capable of being incredibly good at driving. The problem is we are also incredibly unreliable.
    - Nutmog 11y ago
      I wouldn't say the difference matters a lot. Instead humans and computers would just have different thresholds for when to stop and when to run over something. If a computer is less accurate at distinguishing them, then it'll simply err on the side of caution more often than a human. You would find there are more inconvenient stops, not that more more babies get killed. I'm sure I've stopped for empty trash bags before because I wasn't sure if they contained something solid. That doesn't mean I failed as a driver. Everything went fine. Same if it's a computer but it might stop more often than a human.
      - TrevorJ 11y ago
        This assumes that in every situation coming to a controlled stop is an option. That is almost certainly not the case. Many times the choice is between running (an object) over or making some kind of risky maneuver to avoid it.
- oblio 11y ago
  A very old joke (circa 1997, when Clippy was introduced): Two engineers are arguing: a mechanical engineer and a software engineer. The software engineer brags: If we were the ones making cars, they'd go twice as fast as they do now and they'd use half the fuel! The mechanical engineer replies: Yes, but if you'd want to break in case of an emergency you'd get a pop-up saying: Are you sure you want to break? Yes/No/Cancel.
  - mikeash 11y ago
    There's a related joke about engineers riding in a car that breaks down. The mechanical engineer says, "It's a loose belt. Let's tighten that and we'll be all set." The chemical engineer says, "No, it's fuel contamination. We need to drain the fuel, filter it, and then we can be on our way." The software engineer says, "No, all we have to do is turn the car off, get out, shut all the doors, then get back in and turn the car back on." The irony is that I now own a car for which "reboot it" is a valid troubleshooting step....
    - david-given 11y ago
      There's a related joke about engineers at an aviation software convention. This one might even be true. At a talk to an audience of avionics programmers, the speaker at one point asks: "Tell me truthfully; how many of you would be comfortable flying in a plane where your team had done the software?" Onlt a couple of people put their hands up. The speaker targets someone at the front. "You have that much confidence in your work?" "I'd be fine," they reply. "If my team wrote the software, the plane would never even leave the ground."
cortesoft 11y ago
Yeah, that is why the failure response can't be to hand control back to a human. It needs to do other things to get out of the situation, and not rely on a human taking over.
Retric 11y ago
Depends on what the failure modes are. Object avoidance and control in slippery conditions is a vastly easier problem than a 100% self driving car. So, if the default is a car that safely slows down to a stop while beeping loudly then hand-offs are a non issue. Remember, the state of the art in human driving still kills 32,000+ people per year in the US and something like 1+ million worldwide. Beating that seems very doable. EX: You show up at a concert and parking attendants are directing people to an open field. Car goes WTF, stops and hands off.
michaelbuckbee 11y ago
What you are describing is a very real issue, but I think you have to weigh it against just how bad humans are at driving in general (DUI, Texting, phone calls, distracted, tired). I wonder if even at the really primitive state of SDC if they aren't better (in terms of safety) overall than human drivers.
ska 11y ago
That assumes that you have a car to human handoff, and that it would be useful. Honestly that's a pretty strong assumption because of the reasons that you mention and related issues. The technological solution may be that there is no (or vanishingly small numbers) situation where emergency transfer is likely to improve outcomes. So you just don't do it. Non-emergency (i.e. I'm going to park now and hand off control to you) are a separate case. This may plausibly be the best technical solution - how marketable it is, that's another story.
- TrevorJ 11y ago
  My personal feeling is that you are right: the best/only way this will work long term is if it works without humans needing to be in the loop.
naasking 11y ago
> or sudden situation that it cannot contend with. The idea that humans are any better at handling situations they've not previously encountered is dubious.
- AnimalMuppet 11y ago
  Well, if you have an inference engine or something, and there's a situation that it hasn't encountered before, but there are sufficient rules to derive a solution, it may do pretty well. But if it's a completely novel situation, AIs of any flavor tend to fail pretty spectacularly. Humans also can fail spectacularly. But they also can just try something, realize quickly that it isn't working, and switch to trying something else. An AI could do the same, of course. But humans are better (at the moment at least) at eyeballing the situation and realizing that the current approach isn't working. (Or so it seems to me.)
- gr3yh47 11y ago
  Really? I would disagree whole heartedly. This might be true in X years but right now human ability to apply existing knowledge to brand new situations far surpasses that of AI
wmeredith 11y ago
Exactly. I mean, I can't speak for anyone else, but I'll be asleep in the back.
osteele 11y ago
This is anecdotal and subjective, and doesn't directly address the parent's claim (which was about “the long term”), but FWIW: I've been driving a Tesla with “Autopilot” for five months now. Since then I've probably logged about 10,000 miles of use of “Autopilot” features – lots of two-hour trips between Eastern and Western MA; a MA<->NC road trip and some shorter road trips; and lots of 30-40 minute commutes. Prior to getting the feature, I was concerned about handoff. I was concerned that it would be difficult to force myself to pay attention during those times that I wasn't directly responsible for the steering and acceleration. Instead, I've found that it's easy to pay attention, but that I shift to a more strategic mode: scanning further ahead and around for hazards. It's actual easier to maintain situational awareness than when my attention is split between what's going on in the next 10s–1m, and more tactical tasks such as manually tuning my lane position, speed, and follow distance. Driving with automation feels (I told you this was subjective) safer than manual driving. Just as in manual driving you keep your eye open for certain kinds of situations (passing a pedestrian or car on the shoulder, erratic driver ahead) and are poised to go into action, I'm especially primed to take the wheel when these risk factors add up (as well as other areas specific to this particular vehicle automation: its uncertainty about which way to steer around a small traffic island at the end of a turn; its failure to see stopped – as opposed to slow – cars just after a sharp turn or lane change; its penchant – mostly fixed now – for taking highway exits). And being ready to shift more cognitive resources over to driving is such a natural part of being a driver in any case, that I imagine this is a common experience. When autopilot is turned off or I'm in a car that doesn't have it, it feels riskier not to have that second set of “eyes” on the road. There still could be risk compensation: I could be driving faster because I feel safer, for example. I don't really trust anyone's attempts to figure out a priori whether I or the population of current or potential drivers of automated vehicles are subject to this; instead, I hope we see statistics soon. And – I have a few decades practice driving non-automated vehicles, that I can pretty effortless fall back to. I've swerved around emerging accidents a few times, and handled drivers coming into my lane and other near misses; I'm decades from having to consciously think about where to put my foot to brake. I do wonder about the hand-off to people who are just now getting their drivers licenses and are going to have almost all their experience in automated vehicles is going to work. We may be entering a (hopefully brief) pair of uncanny valleys of vehicle automation. (1) Where younger drivers don't have enough practice to deal with crises. (This has always been true of new drivers, but they'll no longer be learning at one year of experience per year; and the handoff requirement is new). And (2) the parent poster's concern, that drivers in general will zone out when the car is driving. My anecdotal evidence, above, is that this isn't yet a problem with the state of the art, but that's just a matter of whether this is the year that happens.
wolframarnold 11y ago
This is why Google canned the whole idea of requiring a human to be alert and ready to take over. They want to get rid of the steering wheel and pedals, because they found in their extensive road testing that their own employees just didn't pay as much attention as they wanted them to. This is still facing regulatory hurdles, obviously.

by TrevorJ 11y ago

adekok 11y ago

Normal desktop failure modes don't work well here. "Hi, this is your car. I've discovered that the road is icy, and am currently in the middle of a 360. Would you like to (a) take over, (b) call an ambulance, or (c) retry?" Too late! You've crashed.

mabbo 11y ago
Bonus points if it's Clippy, but he's a steering wheel now.
mchahn 11y ago
> and am currently in the middle of a 360. You have no faith in AI. The car will always be better at driving than the human. It would be much better at handling a 360 than a human ever could. Actually humans are very bad at handling 360s unless you are a pro.
- timClicks 11y ago
  As long as the sensors, networking, compute and actuators are all functional.
- Alupis 11y ago
  Humans are very, very good at handling unknown complex situations. In order for any AI to handle any situation, it must first be programmed to do so. If that situation was an unknown, you get undefined behavior. Undefined behavior in a car traveling down the road is not a good thing to place a bet on... especially your life. This is why majority of sane driverless vehicle talk has concluded a human driver must always be in the driver seat, ready to take control. There's an impossible volume of failure modes a driverless car must contend with, everything from someone pointing an infrared laser at the distance sensors, to complete power failure, to a sudden rockslide, a bird darting in front of the car, lines missing from a windy and hilly road, etc. In normal, daily mundane driving - driverless cars are expected to excel. Sitting in traffic is mind numbing, and mistakes happen when driver's minds wonder. In "active" driving situations, a human brain will almost always excel.
  - mikeash 11y ago
    Humans are mostly terrible at handling unknown complex situations on the road. For many human drivers on the roads right now, the response to a sudden weird situation is either going to be "brake heavily" or "swerve dangerously." You don't have to get "undefined behavior" in response to an unknown situation. The AI can simply default to braking to a stop in the absence of any better programmed alternative. This will already put it on par with most drivers. Better, in fact, because the AI can begin braking instantly. I personally find that the majority of driverless vehicle talk vastly overestimates the average human driver, and demands unreasonable perfection from the computers.
    - jzwinck 11y ago
      Not only can a computer begin braking more promptly, it can brake more accurately. Many human drivers hit the brakes 100% when they sense danger, then the full force of braking startles them (no training), then they let off again. This can increase stopping distances dramatically. Next up is per-wheel braking. Not just stability control, but actual directed thrust. The computer can control each wheel independently. Humans just have one brake pedal. Four wheel steering might even make a comeback.
    - beat 11y ago
      Remember the two second rule? Instantaneous response would be a huge improvement in many situations already.
      - obsurveyor 11y ago
        With today's aggressive and impatient drivers, the 2 second rule is very hard to maintain because they'll just pass you and fill the spot, every time.
        mikeash 11y ago
        I see this stated a lot, but I think it's exaggerated. My car's adaptive cruise control follows at about 2.5 seconds when on the maximum setting, which is where I almost always keep it, including in heavy traffic. People do occasionally jump in, but then the car just backs off a bit and starts following the new car at 2.5 seconds. It's not a constant churn, and it doesn't cause any significant delay. And this is in the DC area, with dense traffic and ridiculous drivers. When people say that it doesn't work, I really think they're just saying that they can't stand having other people get ahead by breaking the rules, and they'd rather drive more dangerously than let that happen.
- TrevorJ 11y ago
  >The car will always be better at driving than the human. I don't think this is true yet. We can interpret the incoming data incredibly well in ways the machine vision can't yet match. This is absolutely vital for driving. The difference between an empty trash bag in the road and a toddler in the road is a matter of vital importance, and correctly identifying the difference matters a lot. Humans area actually capable of being incredibly good at driving. The problem is we are also incredibly unreliable.
  - Nutmog 11y ago
    I wouldn't say the difference matters a lot. Instead humans and computers would just have different thresholds for when to stop and when to run over something. If a computer is less accurate at distinguishing them, then it'll simply err on the side of caution more often than a human. You would find there are more inconvenient stops, not that more more babies get killed. I'm sure I've stopped for empty trash bags before because I wasn't sure if they contained something solid. That doesn't mean I failed as a driver. Everything went fine. Same if it's a computer but it might stop more often than a human.
    - TrevorJ 11y ago
      This assumes that in every situation coming to a controlled stop is an option. That is almost certainly not the case. Many times the choice is between running (an object) over or making some kind of risky maneuver to avoid it.
oblio 11y ago
A very old joke (circa 1997, when Clippy was introduced): Two engineers are arguing: a mechanical engineer and a software engineer. The software engineer brags: If we were the ones making cars, they'd go twice as fast as they do now and they'd use half the fuel! The mechanical engineer replies: Yes, but if you'd want to break in case of an emergency you'd get a pop-up saying: Are you sure you want to break? Yes/No/Cancel.
- mikeash 11y ago
  There's a related joke about engineers riding in a car that breaks down. The mechanical engineer says, "It's a loose belt. Let's tighten that and we'll be all set." The chemical engineer says, "No, it's fuel contamination. We need to drain the fuel, filter it, and then we can be on our way." The software engineer says, "No, all we have to do is turn the car off, get out, shut all the doors, then get back in and turn the car back on." The irony is that I now own a car for which "reboot it" is a valid troubleshooting step....
  - david-given 11y ago
    There's a related joke about engineers at an aviation software convention. This one might even be true. At a talk to an audience of avionics programmers, the speaker at one point asks: "Tell me truthfully; how many of you would be comfortable flying in a plane where your team had done the software?" Onlt a couple of people put their hands up. The speaker targets someone at the front. "You have that much confidence in your work?" "I'd be fine," they reply. "If my team wrote the software, the plane would never even leave the ground."

cortesoft 11y ago

Yeah, that is why the failure response can't be to hand control back to a human. It needs to do other things to get out of the situation, and not rely on a human taking over.

Retric 11y ago

Depends on what the failure modes are. Object avoidance and control in slippery conditions is a vastly easier problem than a 100% self driving car. So, if the default is a car that safely slows down to a stop while beeping loudly then hand-offs are a non issue. Remember, the state of the art in human driving still kills 32,000+ people per year in the US and something like 1+ million worldwide. Beating that seems very doable. EX: You show up at a concert and parking attendants are directing people to an open field. Car goes WTF, stops and hands off.

michaelbuckbee 11y ago

What you are describing is a very real issue, but I think you have to weigh it against just how bad humans are at driving in general (DUI, Texting, phone calls, distracted, tired). I wonder if even at the really primitive state of SDC if they aren't better (in terms of safety) overall than human drivers.

ska 11y ago

That assumes that you have a car to human handoff, and that it would be useful. Honestly that's a pretty strong assumption because of the reasons that you mention and related issues. The technological solution may be that there is no (or vanishingly small numbers) situation where emergency transfer is likely to improve outcomes. So you just don't do it. Non-emergency (i.e. I'm going to park now and hand off control to you) are a separate case. This may plausibly be the best technical solution - how marketable it is, that's another story.

TrevorJ 11y ago
My personal feeling is that you are right: the best/only way this will work long term is if it works without humans needing to be in the loop.

naasking 11y ago

> or sudden situation that it cannot contend with. The idea that humans are any better at handling situations they've not previously encountered is dubious.

AnimalMuppet 11y ago
Well, if you have an inference engine or something, and there's a situation that it hasn't encountered before, but there are sufficient rules to derive a solution, it may do pretty well. But if it's a completely novel situation, AIs of any flavor tend to fail pretty spectacularly. Humans also can fail spectacularly. But they also can just try something, realize quickly that it isn't working, and switch to trying something else. An AI could do the same, of course. But humans are better (at the moment at least) at eyeballing the situation and realizing that the current approach isn't working. (Or so it seems to me.)
gr3yh47 11y ago
Really? I would disagree whole heartedly. This might be true in X years but right now human ability to apply existing knowledge to brand new situations far surpasses that of AI

wmeredith 11y ago

Exactly. I mean, I can't speak for anyone else, but I'll be asleep in the back.

osteele 11y ago

This is anecdotal and subjective, and doesn't directly address the parent's claim (which was about “the long term”), but FWIW: I've been driving a Tesla with “Autopilot” for five months now. Since then I've probably logged about 10,000 miles of use of “Autopilot” features – lots of two-hour trips between Eastern and Western MA; a MA<->NC road trip and some shorter road trips; and lots of 30-40 minute commutes. Prior to getting the feature, I was concerned about handoff. I was concerned that it would be difficult to force myself to pay attention during those times that I wasn't directly responsible for the steering and acceleration. Instead, I've found that it's easy to pay attention, but that I shift to a more strategic mode: scanning further ahead and around for hazards. It's actual easier to maintain situational awareness than when my attention is split between what's going on in the next 10s–1m, and more tactical tasks such as manually tuning my lane position, speed, and follow distance. Driving with automation feels (I told you this was subjective) safer than manual driving. Just as in manual driving you keep your eye open for certain kinds of situations (passing a pedestrian or car on the shoulder, erratic driver ahead) and are poised to go into action, I'm especially primed to take the wheel when these risk factors add up (as well as other areas specific to this particular vehicle automation: its uncertainty about which way to steer around a small traffic island at the end of a turn; its failure to see stopped – as opposed to slow – cars just after a sharp turn or lane change; its penchant – mostly fixed now – for taking highway exits). And being ready to shift more cognitive resources over to driving is such a natural part of being a driver in any case, that I imagine this is a common experience. When autopilot is turned off or I'm in a car that doesn't have it, it feels riskier not to have that second set of “eyes” on the road. There still could be risk compensation: I could be driving faster because I feel safer, for example. I don't really trust anyone's attempts to figure out a priori whether I or the population of current or potential drivers of automated vehicles are subject to this; instead, I hope we see statistics soon. And – I have a few decades practice driving non-automated vehicles, that I can pretty effortless fall back to. I've swerved around emerging accidents a few times, and handled drivers coming into my lane and other near misses; I'm decades from having to consciously think about where to put my foot to brake. I do wonder about the hand-off to people who are just now getting their drivers licenses and are going to have almost all their experience in automated vehicles is going to work. We may be entering a (hopefully brief) pair of uncanny valleys of vehicle automation. (1) Where younger drivers don't have enough practice to deal with crises. (This has always been true of new drivers, but they'll no longer be learning at one year of experience per year; and the handoff requirement is new). And (2) the parent poster's concern, that drivers in general will zone out when the car is driving. My anecdotal evidence, above, is that this isn't yet a problem with the state of the art, but that's just a matter of whether this is the year that happens.

wolframarnold 11y ago

This is why Google canned the whole idea of requiring a human to be alert and ready to take over. They want to get rid of the steering wheel and pedals, because they found in their extensive road testing that their own employees just didn't pay as much attention as they wanted them to. This is still facing regulatory hurdles, obviously.