5 ms·

What befuddles me is that in all these discussions about self-driving cars seemingly no one refers to the massive body of knowledge in this area that comes from

by cletus 9y ago

What befuddles me is that in all these discussions about self-driving cars seemingly no one refers to the massive body of knowledge in this area that comes from the aviation world. I've posted variants of this same comment several times and I'm starting to feel like a broken record. Look at studies of efforts to make planes safer by removing the human element. While efforts like autopilot have made things safer it reaches the point where more automation can reduce safety as pilots are no longer alert and/or don't trust the instruments and/or can't fully manually override the automation. Call it the uncanny valley of automation safety. Bridging that last few percent for true automation (ie where vehicles aren't designed to have drivers or pilots at all) is going to be _incredibly_ difficult, to the point where I'm not convinced it won't require something resembling a general AI. All of this is why I think driverless cars are going to take much longer than many expect.

deleted 9y ago
[deleted]
tlb 9y ago
There's a big difference: commercial pilots are highly trained, even-tempered, and take their job seriously. Most drivers are lazy, distracted, and apt to do something stupid in an emergency. It's very hard to make something safer than a commercial pilot. It's much easier to make something safer than a typical driver.
- erobbins 9y ago
  Yeah that validates his point. If the arguably most highly trained vehicle operators around lose situational awareness and fail to recover after automation fails, what do you think will happen with untrained drivers?
  - bdamm 9y ago
    That's actually a confirmation bias being missed. Aircraft systems fail and commercial pilots take over and land without mishap all the time. Components inside airplanes, at airports, and even inside air traffic control can and do fail. One example is the attack on the Chicago air traffic control system. Dozens and perhaps even over a hundred aircraft suddenly were flying around with no oversight. Every single pilot took local control, negotiated with the other pilots, and collectively were able to either land or divert without incident.
- archagon 9y ago
  > Most drivers are lazy, distracted, and apt to do something stupid in an emergency. Er, citation needed. I think the vast majority of drivers are good drivers — otherwise, vehicular transport would be a disaster.
  - titanomachy 9y ago
    Yeah, that's true at least where I live. Most times I drive I'll see someone do something stupid or inattentive (or even do something stupid myself!), but I see thousands of people driving normally, i.e. driving well. Countless times I've seen pedestrians or cyclists throw themselves in front of traffic without warning, and every time the drivers have stopped without incident. A collision is by far the more exceptional case.
  - tlb 9y ago
    There are around 1.25M vehicle fatalities every year worldwide [0]. It is a disaster. Driving has killed more people than the world wars. "Good drivers" -- we have no benchmark to measure against. Maybe it's amazing that 10x more people aren't killed, or maybe it's dismal that anyone is killed at all. When we have autonomous vehicles, we'll have a reference to compare against. I predict that "bad" will be the only word to describe the current situation. [0] https://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
- throw2016 9y ago
  most drivers are lazy, distracted and accident prone. If this had any element of truth roads would be empty. This kind of sweeping and bigoted dismissal of other people is a bit too self serving in the context of self driving cars and is made too casually and too often on HN now to allow balanced discussion.
- reubenswartz 9y ago
  I agree. I think it will be very hard to make something that drives better than a really good human driver who is focused on driving. Getting better than the 50+% of people I see with their phone clapped to their faces while they drive (despite that being illegal in Austin) is much easier (though still difficult) task. The other thing is that something going wrong with a plane in the air is a pretty big deal. You can't just pull over and wait. If you assume a "first, do no harm" principle of robotics for driverless cars, the failure mode should in most cases by "pull over and wait." This can still cause problems in many situations, but people do it now.
bluthru 9y ago
>or don't trust the instruments Thankfully this isn't as big of an issue with driving on the ground. Airplanes don't have sensors that give them the same precision as a car's wheel rotation or proximity to nearby objects.
- niftich 9y ago
  On the contrary, this is a much larger issue on the ground. A commercial airplane spends most of its flight time in clear air, in an airway assigned to it by 24/7 air traffic control, safely separated from large objects with which it can collide, and largely safe from disruptions which necessitate sudden control inputs to avoid an immediate crash. Cars don't operate in a comparable problem-space.
  - bluthru 9y ago
    Proximity to other objects doesn't matter when your air speed indicator isn't accurate and you can't get lift. https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447
cdolan 9y ago
This is a very important topic that I am surprised does not receive enough coverage. Thank you for bringing it up. It will be particularly interesting if accident blame is placed on the 'dumb cars', and then insurance companies do a 180 and charge MORE for 'dumb cars' operated by humans. Once they put this information in their pricing models, I assume its 'stuck' in there until the next major NTSB report is published. As complacency sets in over a couple months & years, the accident rates will likely swing from "dumb" human operated machines back to "Level 4 Highly Intelligent Teslas/UBERS/Argo AI", and that market might get a real shock when the pendulum comes back their way!
mdekkers 9y ago
Call it the uncanny valley of automation safety. I am not disputing your assessment, but please don't discount liability. Planes can pretty much fly themselves today - there are no significant technology issues with the idea of "taxi away, take off, fly to destination, land, taxi to gate". all of this happens in what is perhaps the most regulated traffic environment on the planet. The issue is with creating the code that deals with "oh shit" scenarios. Whilst is is probably possible, and even feasible, to write code to cover every possible failure scenario, who is going to be left holding the can when this fails (all systems have a non-zero probability of failure)? Who will be held responsible? The outsourced company that coded the avionics/flight control software? The airplane manufacturer? The airline company? The poor fucker that wrote the failing logic tree that was supposed to deal with that specific failure scenario, but was forced to work overtime the 47th day in a row when that particular code was cut? It is a liability nightmare, and when you add up the cost of creating a software system that must never fail, the increased insurance premiums, the PR/marketing work to convince the unwashed masses that this is actually safer, and the whole rest of the circus required to make this a reality, you will find that pilot costs are not all that bad. Especially since pilots have significant downward pressure on real earnings these days anyway.
- cletus 9y ago
  > Planes can pretty much fly themselves today but > The issue is with creating the code that deals with "oh shit" scenarios. So they fly themselves except they don't? That's kind of my point: what makes anyone think truly driverless cars are going to happen anytime soon when a human is required to deal with these "oh shit" scenarios? What's more, I think the "oh shit" scenarios for cars are FAR more complicated. With planes someone else deals with scheduling for take off and landing. While in flight, the plane simply needs to not fly into other objects and maintain speed, direction and altitude. As for liability, I agree. It's a nightmare, particularly when the standard will probably be "did the software cause injury or death?" when the standard should be "what is the incidence of injury or death compared to a human driver?" I mean that'll be little comfort to the family of someone killed in an accident. We humans seem to have a weird tolerance humans negligently killing other humans.
StephenMelon 9y ago
I agree, the devil is going to be in the almost infinite edge cases, the visual negotiation that goes on between drivers at box junctions, dealing with bad or aggressive drivers who ignore right of way or tailgate, ethical decisions in all the “Kobiashi Maru” no-win situations (extreme weather, black ice, highway pile up, mechanical failures). An advanced AI may well be able to identify whether an object coming towards the windscreen is a bird, bat, leaf or a rock, but what will its intuition be about how much of a problem it is likely to be? Should it swerve to avoid a raccoon and risk whiplash for passengers? Should it aim to avoid large insects if the owner is vegan? Also, people are very used to mild lawbreaking. We expect a cab driver to double park and let us out the car if there are no available parking spaces, but would an AI be authorised to bend the law or would it have to find the nearest parking space, which may be 5 blocks away and could be taken by the time it gets there? I suspect we will get autonomous drone-like flying cars long before we get full autonomy in city centres or rural areas, because flying through a mostly obstacle-free space with an ability to avoid collisions on 3 axis seems much more reachable?
- jacquesm 9y ago
  > Should it aim to avoid large insects if the owner is vegan? Only if the other cars on the road around it are also owned by vegans.

by cletus 9y ago

deleted 9y ago

[deleted]

tlb 9y ago

There's a big difference: commercial pilots are highly trained, even-tempered, and take their job seriously. Most drivers are lazy, distracted, and apt to do something stupid in an emergency. It's very hard to make something safer than a commercial pilot. It's much easier to make something safer than a typical driver.

erobbins 9y ago
Yeah that validates his point. If the arguably most highly trained vehicle operators around lose situational awareness and fail to recover after automation fails, what do you think will happen with untrained drivers?
- bdamm 9y ago
  That's actually a confirmation bias being missed. Aircraft systems fail and commercial pilots take over and land without mishap all the time. Components inside airplanes, at airports, and even inside air traffic control can and do fail. One example is the attack on the Chicago air traffic control system. Dozens and perhaps even over a hundred aircraft suddenly were flying around with no oversight. Every single pilot took local control, negotiated with the other pilots, and collectively were able to either land or divert without incident.
archagon 9y ago
> Most drivers are lazy, distracted, and apt to do something stupid in an emergency. Er, citation needed. I think the vast majority of drivers are good drivers — otherwise, vehicular transport would be a disaster.
- titanomachy 9y ago
  Yeah, that's true at least where I live. Most times I drive I'll see someone do something stupid or inattentive (or even do something stupid myself!), but I see thousands of people driving normally, i.e. driving well. Countless times I've seen pedestrians or cyclists throw themselves in front of traffic without warning, and every time the drivers have stopped without incident. A collision is by far the more exceptional case.
- tlb 9y ago
  There are around 1.25M vehicle fatalities every year worldwide [0]. It is a disaster. Driving has killed more people than the world wars. "Good drivers" -- we have no benchmark to measure against. Maybe it's amazing that 10x more people aren't killed, or maybe it's dismal that anyone is killed at all. When we have autonomous vehicles, we'll have a reference to compare against. I predict that "bad" will be the only word to describe the current situation. [0] https://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate https://en.wikipedia.org/wiki/List_of_countries_by_traffic-r...
throw2016 9y ago
most drivers are lazy, distracted and accident prone. If this had any element of truth roads would be empty. This kind of sweeping and bigoted dismissal of other people is a bit too self serving in the context of self driving cars and is made too casually and too often on HN now to allow balanced discussion.
reubenswartz 9y ago
I agree. I think it will be very hard to make something that drives better than a really good human driver who is focused on driving. Getting better than the 50+% of people I see with their phone clapped to their faces while they drive (despite that being illegal in Austin) is much easier (though still difficult) task. The other thing is that something going wrong with a plane in the air is a pretty big deal. You can't just pull over and wait. If you assume a "first, do no harm" principle of robotics for driverless cars, the failure mode should in most cases by "pull over and wait." This can still cause problems in many situations, but people do it now.

bluthru 9y ago

>or don't trust the instruments Thankfully this isn't as big of an issue with driving on the ground. Airplanes don't have sensors that give them the same precision as a car's wheel rotation or proximity to nearby objects.

niftich 9y ago
On the contrary, this is a much larger issue on the ground. A commercial airplane spends most of its flight time in clear air, in an airway assigned to it by 24/7 air traffic control, safely separated from large objects with which it can collide, and largely safe from disruptions which necessitate sudden control inputs to avoid an immediate crash. Cars don't operate in a comparable problem-space.
- bluthru 9y ago
  Proximity to other objects doesn't matter when your air speed indicator isn't accurate and you can't get lift. https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447

cdolan 9y ago

This is a very important topic that I am surprised does not receive enough coverage. Thank you for bringing it up. It will be particularly interesting if accident blame is placed on the 'dumb cars', and then insurance companies do a 180 and charge MORE for 'dumb cars' operated by humans. Once they put this information in their pricing models, I assume its 'stuck' in there until the next major NTSB report is published. As complacency sets in over a couple months & years, the accident rates will likely swing from "dumb" human operated machines back to "Level 4 Highly Intelligent Teslas/UBERS/Argo AI", and that market might get a real shock when the pendulum comes back their way!

mdekkers 9y ago

Call it the uncanny valley of automation safety. I am not disputing your assessment, but please don't discount liability. Planes can pretty much fly themselves today - there are no significant technology issues with the idea of "taxi away, take off, fly to destination, land, taxi to gate". all of this happens in what is perhaps the most regulated traffic environment on the planet. The issue is with creating the code that deals with "oh shit" scenarios. Whilst is is probably possible, and even feasible, to write code to cover every possible failure scenario, who is going to be left holding the can when this fails (all systems have a non-zero probability of failure)? Who will be held responsible? The outsourced company that coded the avionics/flight control software? The airplane manufacturer? The airline company? The poor fucker that wrote the failing logic tree that was supposed to deal with that specific failure scenario, but was forced to work overtime the 47th day in a row when that particular code was cut? It is a liability nightmare, and when you add up the cost of creating a software system that must never fail, the increased insurance premiums, the PR/marketing work to convince the unwashed masses that this is actually safer, and the whole rest of the circus required to make this a reality, you will find that pilot costs are not all that bad. Especially since pilots have significant downward pressure on real earnings these days anyway.

cletus 9y ago
> Planes can pretty much fly themselves today but > The issue is with creating the code that deals with "oh shit" scenarios. So they fly themselves except they don't? That's kind of my point: what makes anyone think truly driverless cars are going to happen anytime soon when a human is required to deal with these "oh shit" scenarios? What's more, I think the "oh shit" scenarios for cars are FAR more complicated. With planes someone else deals with scheduling for take off and landing. While in flight, the plane simply needs to not fly into other objects and maintain speed, direction and altitude. As for liability, I agree. It's a nightmare, particularly when the standard will probably be "did the software cause injury or death?" when the standard should be "what is the incidence of injury or death compared to a human driver?" I mean that'll be little comfort to the family of someone killed in an accident. We humans seem to have a weird tolerance humans negligently killing other humans.

StephenMelon 9y ago

I agree, the devil is going to be in the almost infinite edge cases, the visual negotiation that goes on between drivers at box junctions, dealing with bad or aggressive drivers who ignore right of way or tailgate, ethical decisions in all the “Kobiashi Maru” no-win situations (extreme weather, black ice, highway pile up, mechanical failures). An advanced AI may well be able to identify whether an object coming towards the windscreen is a bird, bat, leaf or a rock, but what will its intuition be about how much of a problem it is likely to be? Should it swerve to avoid a raccoon and risk whiplash for passengers? Should it aim to avoid large insects if the owner is vegan? Also, people are very used to mild lawbreaking. We expect a cab driver to double park and let us out the car if there are no available parking spaces, but would an AI be authorised to bend the law or would it have to find the nearest parking space, which may be 5 blocks away and could be taken by the time it gets there? I suspect we will get autonomous drone-like flying cars long before we get full autonomy in city centres or rural areas, because flying through a mostly obstacle-free space with an ability to avoid collisions on 3 axis seems much more reachable?

jacquesm 9y ago
> Should it aim to avoid large insects if the owner is vegan? Only if the other cars on the road around it are also owned by vegans.