5 ms·
Tyre blows, the car starts sweering to the left into the oncoming lane, the computer can either: 1) decide that a critical error has occurred and shut down 2)
by gambiting 11y ago
Tyre blows, the car starts sweering to the left into the oncoming lane, the computer can either:
1) decide that a critical error has occurred and shut down
2) apply full brakes on the remaining wheels, sending the car into a spin and killing a group of pedestrians on the pavement.
3) let the car continue, smashing into a minivan with a family inside it. The minivan is also auto-controlled but has no time to change course
4) try steering further left, hitting the barriers/trees, possibly killing its own occupants.
Now, situations 2-4 can be simulated before they happen with enough computational power. Therefore, the computer would have to take a conscious/programmed decision who to kill, and that is a huge deal morally, because at some point someone somewhere will have to write code that will literally be evaluating value of human life.
And before you say that humans would do no better in this situation - that is true, but no one will judge you for a decision you took in the split second you had, you would do what felt right at the time. The computer is programmed a certain way and you can always sue saying that if it did X then person Y would still be alive - and if there's one thing that companies fear it's litigation.
- Razengan 11y agoWell, obviously the car's own occupants should always hold the highest priority for that car. Otherwise it is a terrible terrible product if the code even contains the possibility that it could ever prioritize anything over the lives of its owners. In your scenario, smashing into another vehicle will be preferable to mowing down pedestrians; that vehicle's program just might be able to save itself, or the two vehicles might even communicate to minimize the damage to each other, and in any case a vehicle will most certainly absorb more of the impact than an exposed pedestrian could. People on foot, who have nothing to do with the traffic, should never have to deal with the vehicles and the consequences of their programs. Number 2 from your scenario should not even be an option to the program.
- gambiting 11y agoWhile I agree with you, there still is a possibility of conflict in what you are saying. What if hitting a pedestrian is the only way to avoid killing occupants of the vehicle? On one hand, like you said - the vehicle should always prioritize its owners, but at the same time, it should have some overriding directive that says "never hit a pedestrian at all costs". One of those has to give - who decides which one?
- Razengan 11y agoThe first priority should always be your occupants. Hardcoded. Otherwise, can you imagine the first incident in which some passengers died because the code prioritized the life of someone else? Would anyone want to buy an autocar after that? "YOUR CAR COULD CHOOSE TO KILL YOU", the headlines will say. The second priority should be the safety of pedestrians who are off the road. Until we have computational power at the level of divine prescience, I don't think you can do much else in scenarios like the one you described, other than have a chain of basic priorities that falls back on "stop everything." Eventually though, powerful-enough AI may be able to simulate all outcomes down to a very granular level, such as the difference between outright killing someone or just injuring their limbs to the point of crippling them for life. Many factors will need to be considered then; who appears to be the weakest individual among the unavoidable targets? Are there any acquaintances of the owners among them? Is there a hospital nearby that could tend to their injuries? Does it look like a suicide attempt?
- randomThoughts9 11y agoThe government might and probably will change this rule. I can imagine that after the first couple of accidents where the algorithm decides to kill N pedestrians to save 1 driver, the government could switch to a "utilitarian" point of view - minimize the number of lost lives - and will enforce this by law.
- TeMPOraL 11y agoI hope one day people will actually grow to appreciate that viewpoint and care about others too. Personally, I find the concept of a car that would consciously chose to save you at the expense of 5 pedestrians to be morally wrong.
- ewindisch 11y agoRight. We had an extensive discussion about this at BarCamp Philly this year. The discussion was on machine ethics and we spoke largely about these scenarios of driverless vehicles. I made the argument that machine ethics aren't a real thing. Companies will do whatever they can to avoid litigation, then the government will step in with regulations as the closest approximation we'll get to ethical behavior.
- taneq 11y agoAnd this is why driverless cars will never (or at least not soon) attempt to optimise trolley-problem-style for number of human lives saved or similar. The algorithms chosen will optimise for one thing and one thing only: Legal defensibility for the car's actions.
- ohitsdom 11y agoThis scenario is why a fully autonomous (and networked) grid could be a beautiful thing. You car takes option #3, but all oncoming traffic is already breaking and realigning to let you pass through the lane unharmed, and you car brings itself to a stop in the opposite shoulder.
- jameshart 11y ago5) apply simple ABS'd braking to the three remaining wheels, maintaining control and bringing the car to a safe stop. Cars do no just flip over and explode for no reason. Life is not a fast and furious movie.