6 ms·

I don't think this is a question: I wouldn't get in a self-driving car unless protecting my safety was its number one priority.

ocdtrekkie 10y ago
The problem is, if it's a cloud connected car, or an artificially intelligent car, or even just a closed source car... how can you know if your car is going to decide to protect you over someone else?
- thatswrong0 10y ago
  I think the incentives are aligned for companies to try their best to provide this for their users. If even one fatal accident is documented where the car valued the lives of others over the passenger of the car, people would switch car providers.
- mhb 10y ago
  Contracts? Testing?
semi-extrinsic 10y ago
Would you want your car to prioritise saving just you over saving two random children? Say your car detects a high speed head-on collision with another non-autonomous car is imminent, your car detects one adult and two children in the other car, and it determines it can only avoid collision by ditching your car, risking your life. What should it do? There is plenty evidence that many adults will sacrifice their life to save children.
- thatswrong0 10y ago
  Whatever results in me having the highest chances of survival.
  - the8472 10y ago
    Careful with your 3 laws. If you prioritize safety above all else then your car locks you in once you enter, then orders food online so you won't starve. Not driving around minimizes the chance of accidents after all.
    - thatswrong0 10y ago
      See my above comment about "within reason".
the8472 10y ago
Why? There are many many tradeoffs that are made to day about your safety. They're not built as massive as they could be in the name of fuel-efficiency. They don't use the strongest known materials to keep production costs low. And active safety systems that override your stupid decisions are more likely to be found in expensive cars, i.e. cheaper ones again make a tradeoff for price. But none of these are really important because they only apply in events that already have a relatively low base rate: accidents. Any ethics system that may or may not have to sacrifice your life for a school bus full of nuns will only be forced to make decision with an even lower baserate: accidents + a bunch of additional conditions where the only way to save your live would be risking others, with a reasonable confidence to boot. Elevating this small aspect over everything else seems silly.
- thatswrong0 10y ago
  I didn't think I had to clarify, but here we are. We were talking specifically about the "moral compass" of the car in (rare) situations where it had to choose between my life and others. When it comes to those situations, I think it should always choose to protect my safety. But I guess I'll qualify my original statement: "within reason".
  - the8472 10y ago
    > I think it should always choose to protect my safety. Well, your thinking might be irrelevant. Let us assume the following points: - we live in a democracy with self-interest driven voters. - most commonly the victims in accidents are pedestrians, not occupants of the vehicle. measured by frequency and severity - not every citizen is a car-user (even in an autonomous-vehicle world). e.g. some may prefer bicycles, especially children From that we can conclude it is in every citizen's interest that cars do not prioritize occupants of the vehicles, because even if the car takes evasive maneuvers (deprioritizing the occupants) the overall trade-off matrix will still be in favor of the occupant. This arises because we're not actually dealing with a the trolley problem but with more complex continuous variables where 100%-confidence of 100%loss (death) is only an extreme point. Additionally not every citizen will have any interest at all in being protected while inside a car because they're always outside the car. Thus the overall interest of the citizens will be to prioritize pedestrians over car occupants. And this being the majority view means means cars with selfish behavior will banned. Also note that in this scenario your own self-interest aligns with that of everyone else when you're outside a car. It only mismatches while you're inside a car. But in the end this is all a storm in a teacup. When a system gets to a point where it has to make such tradeoffs then you already failed. And ideally this happens very very rarely. So you just end up shifting 0.0001% risk factors around. There are more important things.
e40 10y ago
Do you practice that when you drive? If you saw people in front of your car and the only way to not hit them was to put yourself in more harm, would you drive right over them?
- thatswrong0 10y ago
  If we're talking about driving a car off a cliff or into oncoming traffic vs. hitting a person, absolutely I'm going to hit the person. Anyways, the main thing here is that I don't want my car making some calculated moral decision.. 3 nuns vs. my current occupant. It should obviously make a reasonable effort to avoid hitting meatbags, but if doing so puts the occupant in overwhelming harms way, it shouldn't. Anyways, I think this is going to be a 0.00001% thing, and in the case that the car may need to swerve (dangerously) to avoid hitting an obstacle, it would be able to do it with precision and care that I'm simply not capable of.. Think swerving on a highway. Pretty dangerous for me to do since I can't check my blind spots that quickly, but it always can.

ocdtrekkie 10y ago

The problem is, if it's a cloud connected car, or an artificially intelligent car, or even just a closed source car... how can you know if your car is going to decide to protect you over someone else?

thatswrong0 10y ago
I think the incentives are aligned for companies to try their best to provide this for their users. If even one fatal accident is documented where the car valued the lives of others over the passenger of the car, people would switch car providers.
mhb 10y ago
Contracts? Testing?

semi-extrinsic 10y ago

Would you want your car to prioritise saving just you over saving two random children? Say your car detects a high speed head-on collision with another non-autonomous car is imminent, your car detects one adult and two children in the other car, and it determines it can only avoid collision by ditching your car, risking your life. What should it do? There is plenty evidence that many adults will sacrifice their life to save children.

thatswrong0 10y ago
Whatever results in me having the highest chances of survival.
- the8472 10y ago
  Careful with your 3 laws. If you prioritize safety above all else then your car locks you in once you enter, then orders food online so you won't starve. Not driving around minimizes the chance of accidents after all.
  - thatswrong0 10y ago
    See my above comment about "within reason".

the8472 10y ago

Why? There are many many tradeoffs that are made to day about your safety. They're not built as massive as they could be in the name of fuel-efficiency. They don't use the strongest known materials to keep production costs low. And active safety systems that override your stupid decisions are more likely to be found in expensive cars, i.e. cheaper ones again make a tradeoff for price. But none of these are really important because they only apply in events that already have a relatively low base rate: accidents. Any ethics system that may or may not have to sacrifice your life for a school bus full of nuns will only be forced to make decision with an even lower baserate: accidents + a bunch of additional conditions where the only way to save your live would be risking others, with a reasonable confidence to boot. Elevating this small aspect over everything else seems silly.

thatswrong0 10y ago
I didn't think I had to clarify, but here we are. We were talking specifically about the "moral compass" of the car in (rare) situations where it had to choose between my life and others. When it comes to those situations, I think it should always choose to protect my safety. But I guess I'll qualify my original statement: "within reason".
- the8472 10y ago
  > I think it should always choose to protect my safety. Well, your thinking might be irrelevant. Let us assume the following points: - we live in a democracy with self-interest driven voters. - most commonly the victims in accidents are pedestrians, not occupants of the vehicle. measured by frequency and severity - not every citizen is a car-user (even in an autonomous-vehicle world). e.g. some may prefer bicycles, especially children From that we can conclude it is in every citizen's interest that cars do not prioritize occupants of the vehicles, because even if the car takes evasive maneuvers (deprioritizing the occupants) the overall trade-off matrix will still be in favor of the occupant. This arises because we're not actually dealing with a the trolley problem but with more complex continuous variables where 100%-confidence of 100%loss (death) is only an extreme point. Additionally not every citizen will have any interest at all in being protected while inside a car because they're always outside the car. Thus the overall interest of the citizens will be to prioritize pedestrians over car occupants. And this being the majority view means means cars with selfish behavior will banned. Also note that in this scenario your own self-interest aligns with that of everyone else when you're outside a car. It only mismatches while you're inside a car. But in the end this is all a storm in a teacup. When a system gets to a point where it has to make such tradeoffs then you already failed. And ideally this happens very very rarely. So you just end up shifting 0.0001% risk factors around. There are more important things.

e40 10y ago

Do you practice that when you drive? If you saw people in front of your car and the only way to not hit them was to put yourself in more harm, would you drive right over them?

thatswrong0 10y ago
If we're talking about driving a car off a cliff or into oncoming traffic vs. hitting a person, absolutely I'm going to hit the person. Anyways, the main thing here is that I don't want my car making some calculated moral decision.. 3 nuns vs. my current occupant. It should obviously make a reasonable effort to avoid hitting meatbags, but if doing so puts the occupant in overwhelming harms way, it shouldn't. Anyways, I think this is going to be a 0.00001% thing, and in the case that the car may need to swerve (dangerously) to avoid hitting an obstacle, it would be able to do it with precision and care that I'm simply not capable of.. Think swerving on a highway. Pretty dangerous for me to do since I can't check my blind spots that quickly, but it always can.