4 ms·

> On the other hand, things like self-driving will remain unsolvable because the problem is fundamentally ill-defined; we don't even know what is a good driving

> On the other hand, things like self-driving will remain unsolvable because the problem is fundamentally ill-defined; we don't even know what is a good driving. Humans arrive safe and unhurt (as much as possible, especially while human drivers remain on the road) at their destination with minimal violation of the locality's established rules of the road. No? (Though now that I've written what amounts to a utility function, I fear what sort of paperclips may come out of it.)

matheusmoreira 7y ago
> minimal violation of the locality's established rules of the road I wish. I've come to the conclusion that the only true rule of the road is: don't crash. As long as no actual collisions occur, people are totally fine with doing whatever they want and bending the rules for their own convenience. I can no longer predict the behavior of other drivers. Even something as basic as the turn signal is unreliable since people are forgetful.
- funnybeam 7y ago
  I think you are right. In fact, thinking some more, it would seem that the whole point of the ‘rules of the road’ is simply to prevent crashes - we have just built up a whole load of protocols in order to achieve that
Randomini 7y ago
Drivers (both AI and human) may face problems that are essentially ethical trolley problems. While many of these choices are clearly artificial to the point of ridiculousness, the one that gets me best is "should a self-driving car drive itself off a cliff, killing its only passenger, or hit and kill some >1 number of pedestrians?". While the external observation may be "minimising deaths is preferable, so drive off that cliff", are people willing to use a vehicle that might intentionally kill them as an intrinsic part of its operation? Or will market forces result in self-driving cars that make more selfish choices being more popular, potentially resulting in suboptimal prisoners-dilemma style results? There are also different ethical norms in different cultures about preferences (https://www.wired.com/story/trolley-problem-teach-self-driving-car-engineers/ https://www.wired.com/story/trolley-problem-teach-self-drivi...). While these are edge cases, they're the edge cases people are worried about, and the source of the ill-definedness: "unhurt as much as possible" implicitly chooses some ethical tradeoff that people can easily have different answers to.
- bonoboTP 7y ago
  Also such meek and suicidal cars would get abused to no end. Just imagine all the assholes today that pass cars in turns with bad visibility or bike in crazy ways. Today they are still paying some attention, because they may easily get killed if the other drivers don't notice quickly enough what they are doing. With meek AIs on the road you can do anything (as long as you bunch up in large enough groups).

matheusmoreira 7y ago

> minimal violation of the locality's established rules of the road I wish. I've come to the conclusion that the only true rule of the road is: don't crash. As long as no actual collisions occur, people are totally fine with doing whatever they want and bending the rules for their own convenience. I can no longer predict the behavior of other drivers. Even something as basic as the turn signal is unreliable since people are forgetful.

funnybeam 7y ago
I think you are right. In fact, thinking some more, it would seem that the whole point of the ‘rules of the road’ is simply to prevent crashes - we have just built up a whole load of protocols in order to achieve that

Randomini 7y ago

Drivers (both AI and human) may face problems that are essentially ethical trolley problems. While many of these choices are clearly artificial to the point of ridiculousness, the one that gets me best is "should a self-driving car drive itself off a cliff, killing its only passenger, or hit and kill some >1 number of pedestrians?". While the external observation may be "minimising deaths is preferable, so drive off that cliff", are people willing to use a vehicle that might intentionally kill them as an intrinsic part of its operation? Or will market forces result in self-driving cars that make more selfish choices being more popular, potentially resulting in suboptimal prisoners-dilemma style results? There are also different ethical norms in different cultures about preferences (https://www.wired.com/story/trolley-problem-teach-self-driving-car-engineers/ https://www.wired.com/story/trolley-problem-teach-self-drivi...). While these are edge cases, they're the edge cases people are worried about, and the source of the ill-definedness: "unhurt as much as possible" implicitly chooses some ethical tradeoff that people can easily have different answers to.

bonoboTP 7y ago
Also such meek and suicidal cars would get abused to no end. Just imagine all the assholes today that pass cars in turns with bad visibility or bike in crazy ways. Today they are still paying some attention, because they may easily get killed if the other drivers don't notice quickly enough what they are doing. With meek AIs on the road you can do anything (as long as you bunch up in large enough groups).