5 ms·

>The autonomous vehicle rounds a corner and detects a crosswalk full of children. It brakes, but your lane is unexpectedly full of sand from a recent rock slide

>The autonomous vehicle rounds a corner and detects a crosswalk full of children. It brakes, but your lane is unexpectedly full of sand from a recent rock slide. It can't get traction. Why did an autonomous car with a map that tells it there's a crosswalk around a blind corner round the blind corner so fast it couldn't stop if there was unexpected debris on the road? What a stupid scenario, autonomous cars have infinite patience, and when the people inside them can watch netflix all the way to work they won't care much about a few extra seconds either. The obvious solution is to program the car to never take any blind corner, mapped crosswalk or not, so fast it can't stop on a dime. These things are going to have to work in winter in Canada. Low traction is a regular problem they'll have to solve. That'll mean every auto that takes this corner will do it very slowly, but that's fine because it can report back to the citywide network that this corner is unsafe, meaning all other autos will avoid it if they can (so no congestion), and as a plus it can be registered for modification with the local council. Maybe a networkable sensor can be erected, allowing the cars to see around the corner. Taking corners faster than you can stop if there's a problem is a very human thing to do. One of the key benefits of autos is they won't do that.

tunesmith 11y ago
That's not really the point - knocking down that example doesn't knock down every possible scenario where uncomfortable questions come up. We were talking about the Trolley Problem at a recent dinner with friends, and one of the guests remarked about how it was like Star Trek; "The Needs Of The Many" versus "The Prime Directive". So it's a general argument about where a car has the capability of making an uncomfortable utilitarian choice (The Needs Of The Many), or of simply shutting off any ability to affect matters and let nature take its course (The Prime Directive). The uncomfortable bit there is that with the latter, there's still that actual choice (by the designers or whoever) to follow the Prime Directive - it's arguably not "natural" anymore.
Animats 11y ago
"The obvious solution is to program the car to never take any blind corner, mapped crosswalk or not, so fast it can't stop on a dime." In general, that's right. There is no reason for the vehicle to out-drive its unobstructed sensor range. One effect, which the Google cars already demonstrate, is that their behavior at corners with obstructed visibility is markedly more conservative than when the LIDAR can see down the cross street. That's a good thing. Having once coded this for a DARPA Grand Challenge vehicle, that's how you look at the problem. You're always plotting stopping distance against the map constructed from sensor data. You figure out how far it's safe to advance given sensor data, and compute a top speed from that. We did this every 100ms. If the map wasn't filled in with new safe space into which it could advance, the vehicle would slow down and stop at the end of the safe area. So it never out-drove its sensor range. Remember, this was for off-road operations, with obstacles and cliffs; we had to be very conservative. Humans tend to get into a mindset of not wanting to slow down. Software doesn't have that problem. Constantly having a plan available for an emergency stop is not a problem for software.
giancarlostoro 11y ago
I'm wondering why in that scenario the car doesn't honk the horn at all? Are all those children death and blind? No adults are supervising those kids either? I just don't understand. I hope the safety standardization of autonomous cars is a joint agreement that involves the companies involved, and the public, especially if a real scenario occurs...
- RamenJunkie_ 11y ago
  They can be deaf and blind if it helps push anti self driving agenda. Even if a self driving car runs over a pack of school children daily, it will still be less fatalities than humans cause.
crististm 11y ago
The AI used in these cars is probability based. They can't be held accountable in the sense that they can't explain their decisions rationally. You can't actually predict what you'll get by slight modifications of probabilities when adding new test cases. Gerry Sussman explained this recently way better than I can.
- adrianN 11y ago
  Your brain is also probability based and you don't have an audit trail for your thoughts written to a black box.
  - dasil003 11y ago
    So what is really needed is to add post-hoc rationalization engines to these self-driving cars to facilitate satisfying our sense of justice.
    - TeMPOraL 11y ago
      That was actually my first reaction to reading about Sussman's AI ideas in an article[0] that was on HN recently[1]. Propagators seems neat, but the quote: "Sure, when you're running the program, that whole thing is a black box. So is your brain. But you can explain to me the reasoning of why you did something. At that point, being able to inspect the symbolic reasoning of the system is all you have." made me immediately notice that there's a reason we have negative connotations with the world "rationalization" - because most of the time, post-fact rationalizations are bullshit and do not reveal one's actual thoughts at the time of the event, but what one wishes one thought at that time. Just bolting a "reasoning explainer" on top of a Bayesian inference system won't fare any better than human rationalization does. [0] - http://dustycloud.org/blog/sussman-on-ai/ http://dustycloud.org/blog/sussman-on-ai/ [1] - https://news.ycombinator.com/item?id=10388795 https://news.ycombinator.com/item?id=10388795
- mtgx 11y ago
  I think the auto/software makers should take responsibility for it, though, as some already have said they will. Just like when you have bad brakes in a car, it's the auto maker that is responsible not the supplier of that component. Also, they can't just say "it happens, what are you going to do?"
mtgx 11y ago
I'm much more worried about self-driving cars getting hacked than "whether it will decide to kill me" in a scenario that is only going to get less likely as more and more self-driving vehicles are on the road.
ubersync 11y ago
I think it was more of an example of an imaginary emergency situation. You can hard program the corners, but you can't successfully predict all kinds of emergency situations that may arrive. Another example in that article is better: A rogue motorcyclist just turning blindly in front of you. Maybe, the only two choices are either you run them over, or steer your car off the road.
- Sir_Substance 11y ago
  Why are you so close to the motorbike that if he stops, you'll collide with him? If he's coming out of a blind sidestreet, why are you approaching so fast and with so little lateral room that you can't dodge a surprise motorcyclist? These are all human patience related problems. An auto should be perpetually correcting to make sure that it's current minimum collision avoidance distance is smaller than it's current visual clear zone.
  - quicklyfrozen 11y ago
    So every time there's oncoming traffic (or a person on a sidewalk nearby) you'd have to stop until they were past, just in case they decide to turn/step in front of you. As long as there are other actors in the world who may behave irrationally I don't think you have to work too hard to come up with either/or scenarios for the AI to deal with. And even if the irrationality is removed there's still a chance of mechanical failure (e.g. tire blowout causes loss of control) which other AIs may need to make decisions on how to avoid.
RamenJunkie_ 11y ago
This is my entire problem with this bull shit question. It applies human limitations and distractions to a computer. The scenario will never come.because the autonomous car sees everything with way greater clarity than any human. And it will never miss a person crossing the street because its rubbernecking at am accident or gawking at a hot girl or answering a test or trying to turn up the radio because its favorite song is on.
xbmcuser 11y ago
There won't be a blind corner as the thousands of other autonomous Google cars passing by would already be napping and sending live data. Probably even some of the children crossing the street will have Google devices aw ding the data to Google and through to the car.
xbmcuser 11y ago
There won't be a blind corner as the thousands of other autonomous Google cars passing by would already be napping and sending live data. Probably even some of the children crossing the street will have Google devices aw ding the data to Google and through to the car.

tunesmith 11y ago

That's not really the point - knocking down that example doesn't knock down every possible scenario where uncomfortable questions come up. We were talking about the Trolley Problem at a recent dinner with friends, and one of the guests remarked about how it was like Star Trek; "The Needs Of The Many" versus "The Prime Directive". So it's a general argument about where a car has the capability of making an uncomfortable utilitarian choice (The Needs Of The Many), or of simply shutting off any ability to affect matters and let nature take its course (The Prime Directive). The uncomfortable bit there is that with the latter, there's still that actual choice (by the designers or whoever) to follow the Prime Directive - it's arguably not "natural" anymore.

Animats 11y ago

"The obvious solution is to program the car to never take any blind corner, mapped crosswalk or not, so fast it can't stop on a dime." In general, that's right. There is no reason for the vehicle to out-drive its unobstructed sensor range. One effect, which the Google cars already demonstrate, is that their behavior at corners with obstructed visibility is markedly more conservative than when the LIDAR can see down the cross street. That's a good thing. Having once coded this for a DARPA Grand Challenge vehicle, that's how you look at the problem. You're always plotting stopping distance against the map constructed from sensor data. You figure out how far it's safe to advance given sensor data, and compute a top speed from that. We did this every 100ms. If the map wasn't filled in with new safe space into which it could advance, the vehicle would slow down and stop at the end of the safe area. So it never out-drove its sensor range. Remember, this was for off-road operations, with obstacles and cliffs; we had to be very conservative. Humans tend to get into a mindset of not wanting to slow down. Software doesn't have that problem. Constantly having a plan available for an emergency stop is not a problem for software.

giancarlostoro 11y ago

I'm wondering why in that scenario the car doesn't honk the horn at all? Are all those children death and blind? No adults are supervising those kids either? I just don't understand. I hope the safety standardization of autonomous cars is a joint agreement that involves the companies involved, and the public, especially if a real scenario occurs...

RamenJunkie_ 11y ago
They can be deaf and blind if it helps push anti self driving agenda. Even if a self driving car runs over a pack of school children daily, it will still be less fatalities than humans cause.

crististm 11y ago

The AI used in these cars is probability based. They can't be held accountable in the sense that they can't explain their decisions rationally. You can't actually predict what you'll get by slight modifications of probabilities when adding new test cases. Gerry Sussman explained this recently way better than I can.

adrianN 11y ago
Your brain is also probability based and you don't have an audit trail for your thoughts written to a black box.
- dasil003 11y ago
  So what is really needed is to add post-hoc rationalization engines to these self-driving cars to facilitate satisfying our sense of justice.
  - TeMPOraL 11y ago
    That was actually my first reaction to reading about Sussman's AI ideas in an article[0] that was on HN recently[1]. Propagators seems neat, but the quote: "Sure, when you're running the program, that whole thing is a black box. So is your brain. But you can explain to me the reasoning of why you did something. At that point, being able to inspect the symbolic reasoning of the system is all you have." made me immediately notice that there's a reason we have negative connotations with the world "rationalization" - because most of the time, post-fact rationalizations are bullshit and do not reveal one's actual thoughts at the time of the event, but what one wishes one thought at that time. Just bolting a "reasoning explainer" on top of a Bayesian inference system won't fare any better than human rationalization does. [0] - http://dustycloud.org/blog/sussman-on-ai/ http://dustycloud.org/blog/sussman-on-ai/ [1] - https://news.ycombinator.com/item?id=10388795 https://news.ycombinator.com/item?id=10388795
mtgx 11y ago
I think the auto/software makers should take responsibility for it, though, as some already have said they will. Just like when you have bad brakes in a car, it's the auto maker that is responsible not the supplier of that component. Also, they can't just say "it happens, what are you going to do?"

mtgx 11y ago

I'm much more worried about self-driving cars getting hacked than "whether it will decide to kill me" in a scenario that is only going to get less likely as more and more self-driving vehicles are on the road.

ubersync 11y ago

I think it was more of an example of an imaginary emergency situation. You can hard program the corners, but you can't successfully predict all kinds of emergency situations that may arrive. Another example in that article is better: A rogue motorcyclist just turning blindly in front of you. Maybe, the only two choices are either you run them over, or steer your car off the road.

Sir_Substance 11y ago
Why are you so close to the motorbike that if he stops, you'll collide with him? If he's coming out of a blind sidestreet, why are you approaching so fast and with so little lateral room that you can't dodge a surprise motorcyclist? These are all human patience related problems. An auto should be perpetually correcting to make sure that it's current minimum collision avoidance distance is smaller than it's current visual clear zone.
- quicklyfrozen 11y ago
  So every time there's oncoming traffic (or a person on a sidewalk nearby) you'd have to stop until they were past, just in case they decide to turn/step in front of you. As long as there are other actors in the world who may behave irrationally I don't think you have to work too hard to come up with either/or scenarios for the AI to deal with. And even if the irrationality is removed there's still a chance of mechanical failure (e.g. tire blowout causes loss of control) which other AIs may need to make decisions on how to avoid.

RamenJunkie_ 11y ago

This is my entire problem with this bull shit question. It applies human limitations and distractions to a computer. The scenario will never come.because the autonomous car sees everything with way greater clarity than any human. And it will never miss a person crossing the street because its rubbernecking at am accident or gawking at a hot girl or answering a test or trying to turn up the radio because its favorite song is on.

xbmcuser 11y ago

There won't be a blind corner as the thousands of other autonomous Google cars passing by would already be napping and sending live data. Probably even some of the children crossing the street will have Google devices aw ding the data to Google and through to the car.

xbmcuser 11y ago