8 ms·

> I think the ethical question is greatly overblown. Not at all. One programmer is going to make choices that affect ~ a billion vehicles one day, so something

> I think the ethical question is greatly overblown. Not at all. One programmer is going to make choices that affect ~ a billion vehicles one day, so something that doesn't happen 99.9999% per hour will happen 100 times per hour. And it's going to be more mundane things, like programming the speed limit 1 mph slower results in 500 less deaths per year, or using drm/copyright to stop poor people from getting driving software, thus killing thousands of people.

Retric 10y ago
Programmers are not going to decide what is an acceptable MPH vs posted speed or if the car will use DRM. Much like how programmers at Volkswagen, Audi, etc are not the ones that chose to cheat emissions tests. The real choice is going to be what to do about soft obstacles. Hitting a person or deer can easily kill passengers and hitting trees is actually normally safer thus making the discussion fairly moot. People are going to want to hack self driving cars so they can speed, which is more inline with this moral discussion. Not because it's safe for passengers, but because it's also dangerous for other people.
- twblalock 10y ago
  Programmers have to make decisions about those things all the time. It isn't like the product team defines every nitty-gritty detail of implementation -- especially for machine learning and AI algorithms.
  - Retric 10y ago
    Sure, in the how cautious should I be in rain situation. Not, it's a sunny day and open road, the speed limit is 65 what should I do situation. PS: I expect company's to punt with a user selectable: how much to speed buttons.
    - twblalock 10y ago
      You're wrong. Programmers cannot escape making decisions when they design these algorithms, no matter the situation they are designed for.
      - Retric 10y ago
        A lot of AI is based around training sets not traditional Algorithms like your thinking of. There are simply to many edge cases to deal with by hand. Yes, there is also a lot of hand coding in these systems, but good training sets are very critical.
        twblalock 10y ago
        The choice of those training sets, including their contents, their size, how they are consumed, and the algorithms they are fed to, affects the output. Therefore, programmers make decisions. It is unavoidable.
        Retric 10y ago
        I don't think the people doing a million miles driving Google cars around are classified as programmers.
        twblalock 10y ago
        Actually, some of them are programmers. Even if they weren't, it is unrealistic to assume that programmers have no input into the decisions that go into building the training set, which in this case involves where, and in what conditions, the prototype cars are driven.
        Retric 10y ago
        Granted, though plenty of Uber drivers are probably also programmers, and I have made plenty of business decisions I probably should not have. But, I would hope this is held to a higher standard.
mikeash 10y ago
"Per hour" is rather ridiculous. I'm talking about per crash. There are a couple million vehicle crashes in the US each year, so if it really is 99.9999%, that's two or three ethical crises per year here. Among a billion vehicles it would be maybe ten per year. And honestly I'd be surprised if it's that many. When have you ever heard of a crash where there was any kind of ethical dilemma in the response? People keep imagining scenarios, but I've never heard of one happening in real life. As for speed limits and such, I'm talking about the car's ethical choices in crashes, not the ethics of the programmers.
- foota 10y ago
  I'm actually familiar with a situation like this, the person was on a highway at night and came around a corner to see a deer lying in the road and a crowd right behind the deer. There was a couple of people off to either side. They couldn't brake in time to stop and not harm the people on the other side of the deer.
  - mikeash 10y ago
    What's the ethical dilemma there? From your description it sounds like they were just screwed regardless.
    - maxerickson 10y ago
      Also, an autonomous system would probably get information about that sort of problem and slow down prior to going around the turn. At least, they will if we don't screw it up.
      - Zak 10y ago
        Human drivers can already get information about that sort of problem, e.g. by using Waze. This requires that a human who uses the app has reported a hazard (the act of which could sometimes result in distracted driving), however, I would expect autonomous cars to tell each other about such things in the future.
    - 10y ago
acbabis 10y ago
If the cars obey the posted speed limit and slow down and split-the-difference when on curves or near hazards (such as children running on the sidewalk), the hypothetical moral dilemmas shouldn't even happen. So far, I've only heard of one accident involving a Google car, and there wasn't a fatality.
- sokoloff 10y ago
  There's only been one accident that was the Google car's fault. Google cars have been involved in more than a dozen collisions in total.
- CM30 10y ago
  Natural environment/human/animal behaviour related issues kind of throw a wrench in that idea. Yes, in normal conditions you shouldn't ever be in a situation where the 'trolley problem' is relevant. But it's also quite easy to imagine external factors that might make it unavoidable. Like say, a car running out of power in the middle of the road, falling trees, buildings, fences or other obstacles, people or animals in the road, natural disasters or freak weather conditions. All of these situations open up a possibility that the car won't have enough time to stop regardless of the speed limit.
kybernetikos 10y ago
To me the question is relatively uninteresting because it seems obvious. As long as cars belong to individuals they have the responsibility to favor the interests of those individuals as far as law permits. Where and how much the community is to be preferred over the individual, the community should specify in laws. People running code that breaks those laws should be liable for that codes actions. (I could imagine an ethics slider settable at the owners choice within the legal limits that allows them to discount the lives of the passengers compared to the lives of others, on the condition that they notify their other passengers). It's possible that ultimately the right choice is to move to a different model, but in that case, the AI should not be owned by the individual who bought the car and has a reasonable expectation that it work on their behalf. The ethical decision on the car is no different than the ethical decision that we ask the human driver to make in similar situations (which as the gp noted are incredibly rare), or that we trust the human driver to make when we are passengers in their car. In fact I even doubt that it makes sense to legislate it, since we haven't so far for e.g. uber drivers, but perhaps testing will indicate that we should. In terms of the speed limits, society has already decided to accept a huge number of deaths per year that would almost all be preventable with a speed limit of 35mph, so I don't see how the introduction of AI drivers affects this question. I would like to see laws that say that all source code for self driving cars must be publicly auditable and pass tests before being approved for road use. Running code that has not been safety tested by the state will leave you uninsurable and personally liable for its actions. We should aim to make running the test suite as accessible as possible in order to allow innovation, but it should still give us a high confidence that the driver behaves correctly according to the law. I take pretty much all of this from analogy with human drivers: when we're driven by someone else, we have relatively little control over their ethical slider setting, we have high speed limits for humans because we think the convenience of the many is more important than the lives of the few, we don't let brand new humans drive without having some indication that they'll make the right decisions. None of these problems seem to be particularly changed by the introduction of AI.
- patrickmay 10y ago
  > As long as cars belong to individuals they have the responsibility to favor the interests of those individuals as far as law permits. Well put. I would not buy a car that wasn't designed to protect its occupants first and foremost. My family is more important to me than abstract, unlikely to occur ethical dilemmas. Having that choice programmed in actually makes it easier for self-driving cars to predict the behavior of other cars on the road. It's simple and probably at least a near-global maxima for optimal car fitness.
WalterBright 10y ago
Engineers make such decisions all the time. There are always tradeoffs with respect to safety/cost/utility/etc.

Retric 10y ago

Programmers are not going to decide what is an acceptable MPH vs posted speed or if the car will use DRM. Much like how programmers at Volkswagen, Audi, etc are not the ones that chose to cheat emissions tests. The real choice is going to be what to do about soft obstacles. Hitting a person or deer can easily kill passengers and hitting trees is actually normally safer thus making the discussion fairly moot. People are going to want to hack self driving cars so they can speed, which is more inline with this moral discussion. Not because it's safe for passengers, but because it's also dangerous for other people.

twblalock 10y ago
Programmers have to make decisions about those things all the time. It isn't like the product team defines every nitty-gritty detail of implementation -- especially for machine learning and AI algorithms.
- Retric 10y ago
  Sure, in the how cautious should I be in rain situation. Not, it's a sunny day and open road, the speed limit is 65 what should I do situation. PS: I expect company's to punt with a user selectable: how much to speed buttons.
  - twblalock 10y ago
    You're wrong. Programmers cannot escape making decisions when they design these algorithms, no matter the situation they are designed for.
    - Retric 10y ago
      A lot of AI is based around training sets not traditional Algorithms like your thinking of. There are simply to many edge cases to deal with by hand. Yes, there is also a lot of hand coding in these systems, but good training sets are very critical.
      - twblalock 10y ago
        The choice of those training sets, including their contents, their size, how they are consumed, and the algorithms they are fed to, affects the output. Therefore, programmers make decisions. It is unavoidable.
        Retric 10y ago
        I don't think the people doing a million miles driving Google cars around are classified as programmers.
        twblalock 10y ago
        Actually, some of them are programmers. Even if they weren't, it is unrealistic to assume that programmers have no input into the decisions that go into building the training set, which in this case involves where, and in what conditions, the prototype cars are driven.
        Retric 10y ago
        Granted, though plenty of Uber drivers are probably also programmers, and I have made plenty of business decisions I probably should not have. But, I would hope this is held to a higher standard.

mikeash 10y ago

"Per hour" is rather ridiculous. I'm talking about per crash. There are a couple million vehicle crashes in the US each year, so if it really is 99.9999%, that's two or three ethical crises per year here. Among a billion vehicles it would be maybe ten per year. And honestly I'd be surprised if it's that many. When have you ever heard of a crash where there was any kind of ethical dilemma in the response? People keep imagining scenarios, but I've never heard of one happening in real life. As for speed limits and such, I'm talking about the car's ethical choices in crashes, not the ethics of the programmers.

foota 10y ago
I'm actually familiar with a situation like this, the person was on a highway at night and came around a corner to see a deer lying in the road and a crowd right behind the deer. There was a couple of people off to either side. They couldn't brake in time to stop and not harm the people on the other side of the deer.
- mikeash 10y ago
  What's the ethical dilemma there? From your description it sounds like they were just screwed regardless.
  - maxerickson 10y ago
    Also, an autonomous system would probably get information about that sort of problem and slow down prior to going around the turn. At least, they will if we don't screw it up.
    - Zak 10y ago
      Human drivers can already get information about that sort of problem, e.g. by using Waze. This requires that a human who uses the app has reported a hazard (the act of which could sometimes result in distracted driving), however, I would expect autonomous cars to tell each other about such things in the future.
  - 10y ago

acbabis 10y ago

If the cars obey the posted speed limit and slow down and split-the-difference when on curves or near hazards (such as children running on the sidewalk), the hypothetical moral dilemmas shouldn't even happen. So far, I've only heard of one accident involving a Google car, and there wasn't a fatality.

sokoloff 10y ago
There's only been one accident that was the Google car's fault. Google cars have been involved in more than a dozen collisions in total.
CM30 10y ago
Natural environment/human/animal behaviour related issues kind of throw a wrench in that idea. Yes, in normal conditions you shouldn't ever be in a situation where the 'trolley problem' is relevant. But it's also quite easy to imagine external factors that might make it unavoidable. Like say, a car running out of power in the middle of the road, falling trees, buildings, fences or other obstacles, people or animals in the road, natural disasters or freak weather conditions. All of these situations open up a possibility that the car won't have enough time to stop regardless of the speed limit.

kybernetikos 10y ago

To me the question is relatively uninteresting because it seems obvious. As long as cars belong to individuals they have the responsibility to favor the interests of those individuals as far as law permits. Where and how much the community is to be preferred over the individual, the community should specify in laws. People running code that breaks those laws should be liable for that codes actions. (I could imagine an ethics slider settable at the owners choice within the legal limits that allows them to discount the lives of the passengers compared to the lives of others, on the condition that they notify their other passengers). It's possible that ultimately the right choice is to move to a different model, but in that case, the AI should not be owned by the individual who bought the car and has a reasonable expectation that it work on their behalf. The ethical decision on the car is no different than the ethical decision that we ask the human driver to make in similar situations (which as the gp noted are incredibly rare), or that we trust the human driver to make when we are passengers in their car. In fact I even doubt that it makes sense to legislate it, since we haven't so far for e.g. uber drivers, but perhaps testing will indicate that we should. In terms of the speed limits, society has already decided to accept a huge number of deaths per year that would almost all be preventable with a speed limit of 35mph, so I don't see how the introduction of AI drivers affects this question. I would like to see laws that say that all source code for self driving cars must be publicly auditable and pass tests before being approved for road use. Running code that has not been safety tested by the state will leave you uninsurable and personally liable for its actions. We should aim to make running the test suite as accessible as possible in order to allow innovation, but it should still give us a high confidence that the driver behaves correctly according to the law. I take pretty much all of this from analogy with human drivers: when we're driven by someone else, we have relatively little control over their ethical slider setting, we have high speed limits for humans because we think the convenience of the many is more important than the lives of the few, we don't let brand new humans drive without having some indication that they'll make the right decisions. None of these problems seem to be particularly changed by the introduction of AI.

patrickmay 10y ago
> As long as cars belong to individuals they have the responsibility to favor the interests of those individuals as far as law permits. Well put. I would not buy a car that wasn't designed to protect its occupants first and foremost. My family is more important to me than abstract, unlikely to occur ethical dilemmas. Having that choice programmed in actually makes it easier for self-driving cars to predict the behavior of other cars on the road. It's simple and probably at least a near-global maxima for optimal car fitness.

WalterBright 10y ago

Engineers make such decisions all the time. There are always tradeoffs with respect to safety/cost/utility/etc.