9 ms·
I think the ethical question is greatly overblown. The odds of being in a situation where there's any sort of ethical choice to make are so remote as to not rea
by mikeash 10y ago
I think the ethical question is greatly overblown. The odds of being in a situation where there's any sort of ethical choice to make are so remote as to not really be worth considering. 99.9999% of the time, the correct choice when faced with an impending crash will be to brake as hard as the driving surface allows and steer to avoid obstacles as much as possible.
In any case, tinkering and commons already intersect to an extent, for example with emissions. It's not uncommon for emissions to be favored over performance, for example, and some people hack their cars to favor performance instead. Or they just like being polluting assholes, as is the case with "rolling coal." But it's not a big deal, some jurisdictions do periodic testing and others just rely on the fact that such people are relatively rare.
- rando3826 10y ago> I think the ethical question is greatly overblown. Not at all. One programmer is going to make choices that affect ~ a billion vehicles one day, so something that doesn't happen 99.9999% per hour will happen 100 times per hour. And it's going to be more mundane things, like programming the speed limit 1 mph slower results in 500 less deaths per year, or using drm/copyright to stop poor people from getting driving software, thus killing thousands of people.
- Retric 10y agoProgrammers are not going to decide what is an acceptable MPH vs posted speed or if the car will use DRM. Much like how programmers at Volkswagen, Audi, etc are not the ones that chose to cheat emissions tests. The real choice is going to be what to do about soft obstacles. Hitting a person or deer can easily kill passengers and hitting trees is actually normally safer thus making the discussion fairly moot. People are going to want to hack self driving cars so they can speed, which is more inline with this moral discussion. Not because it's safe for passengers, but because it's also dangerous for other people.
- twblalock 10y agoProgrammers have to make decisions about those things all the time. It isn't like the product team defines every nitty-gritty detail of implementation -- especially for machine learning and AI algorithms.
- Retric 10y agoSure, in the how cautious should I be in rain situation. Not, it's a sunny day and open road, the speed limit is 65 what should I do situation. PS: I expect company's to punt with a user selectable: how much to speed buttons.
- twblalock 10y agoYou're wrong. Programmers cannot escape making decisions when they design these algorithms, no matter the situation they are designed for.
- Retric 10y agoA lot of AI is based around training sets not traditional Algorithms like your thinking of. There are simply to many edge cases to deal with by hand. Yes, there is also a lot of hand coding in these systems, but good training sets are very critical.
- twblalock 10y agoThe choice of those training sets, including their contents, their size, how they are consumed, and the algorithms they are fed to, affects the output. Therefore, programmers make decisions. It is unavoidable.
- Retric 10y agoI don't think the people doing a million miles driving Google cars around are classified as programmers.
- mikeash 10y ago"Per hour" is rather ridiculous. I'm talking about per crash. There are a couple million vehicle crashes in the US each year, so if it really is 99.9999%, that's two or three ethical crises per year here. Among a billion vehicles it would be maybe ten per year. And honestly I'd be surprised if it's that many. When have you ever heard of a crash where there was any kind of ethical dilemma in the response? People keep imagining scenarios, but I've never heard of one happening in real life. As for speed limits and such, I'm talking about the car's ethical choices in crashes, not the ethics of the programmers.
- foota 10y agoI'm actually familiar with a situation like this, the person was on a highway at night and came around a corner to see a deer lying in the road and a crowd right behind the deer. There was a couple of people off to either side. They couldn't brake in time to stop and not harm the people on the other side of the deer.
- mikeash 10y agoWhat's the ethical dilemma there? From your description it sounds like they were just screwed regardless.
- maxerickson 10y agoAlso, an autonomous system would probably get information about that sort of problem and slow down prior to going around the turn. At least, they will if we don't screw it up.
- Zak 10y agoHuman drivers can already get information about that sort of problem, e.g. by using Waze. This requires that a human who uses the app has reported a hazard (the act of which could sometimes result in distracted driving), however, I would expect autonomous cars to tell each other about such things in the future.
- 10y ago
- acbabis 10y agoIf the cars obey the posted speed limit and slow down and split-the-difference when on curves or near hazards (such as children running on the sidewalk), the hypothetical moral dilemmas shouldn't even happen. So far, I've only heard of one accident involving a Google car, and there wasn't a fatality.
- sokoloff 10y agoThere's only been one accident that was the Google car's fault. Google cars have been involved in more than a dozen collisions in total.
- CM30 10y agoNatural environment/human/animal behaviour related issues kind of throw a wrench in that idea. Yes, in normal conditions you shouldn't ever be in a situation where the 'trolley problem' is relevant. But it's also quite easy to imagine external factors that might make it unavoidable. Like say, a car running out of power in the middle of the road, falling trees, buildings, fences or other obstacles, people or animals in the road, natural disasters or freak weather conditions. All of these situations open up a possibility that the car won't have enough time to stop regardless of the speed limit.
- kybernetikos 10y agoTo me the question is relatively uninteresting because it seems obvious. As long as cars belong to individuals they have the responsibility to favor the interests of those individuals as far as law permits. Where and how much the community is to be preferred over the individual, the community should specify in laws. People running code that breaks those laws should be liable for that codes actions. (I could imagine an ethics slider settable at the owners choice within the legal limits that allows them to discount the lives of the passengers compared to the lives of others, on the condition that they notify their other passengers). It's possible that ultimately the right choice is to move to a different model, but in that case, the AI should not be owned by the individual who bought the car and has a reasonable expectation that it work on their behalf. The ethical decision on the car is no different than the ethical decision that we ask the human driver to make in similar situations (which as the gp noted are incredibly rare), or that we trust the human driver to make when we are passengers in their car. In fact I even doubt that it makes sense to legislate it, since we haven't so far for e.g. uber drivers, but perhaps testing will indicate that we should. In terms of the speed limits, society has already decided to accept a huge number of deaths per year that would almost all be preventable with a speed limit of 35mph, so I don't see how the introduction of AI drivers affects this question. I would like to see laws that say that all source code for self driving cars must be publicly auditable and pass tests before being approved for road use. Running code that has not been safety tested by the state will leave you uninsurable and personally liable for its actions. We should aim to make running the test suite as accessible as possible in order to allow innovation, but it should still give us a high confidence that the driver behaves correctly according to the law. I take pretty much all of this from analogy with human drivers: when we're driven by someone else, we have relatively little control over their ethical slider setting, we have high speed limits for humans because we think the convenience of the many is more important than the lives of the few, we don't let brand new humans drive without having some indication that they'll make the right decisions. None of these problems seem to be particularly changed by the introduction of AI.
- patrickmay 10y ago> As long as cars belong to individuals they have the responsibility to favor the interests of those individuals as far as law permits. Well put. I would not buy a car that wasn't designed to protect its occupants first and foremost. My family is more important to me than abstract, unlikely to occur ethical dilemmas. Having that choice programmed in actually makes it easier for self-driving cars to predict the behavior of other cars on the road. It's simple and probably at least a near-global maxima for optimal car fitness.
- WalterBright 10y agoEngineers make such decisions all the time. There are always tradeoffs with respect to safety/cost/utility/etc.
- saidajigumi 10y ago> I think the ethical question is greatly overblown. Agreed. At least at the level these cars are likely to enter the market, the whole concept of what the car "knows" isn't likely to extend to "other people", "my passengers", "inanimate objects" and a probability matrix of injury/death/damage among these categories. Instead of this heavily anthropomorphized view, cars are much more likely to have a highly simplified, "bug-like" level of information to work from. This isn't harping on a "technology is neutral" vibe, which I view as obviously false. Just that many of the high-minded philosophical models presented to date don't appear to map well to a likely actual embodiment of "ethical" decisions that may get embedded into vehicular software. If the model is a bad fit for the real-world problem(s) encountered, it's simply not going to be very useful for informing solutions to that problem.
- jackhack 10y agoMore likely than one might want to think. A car traveling the same direction swerves into your lane, crowding you off the road. Does your self-driving car allow the collision and push the offending car back in order to hold your lane? Or does it avoid the collision and run you off the road? (As a person who drives in the mountains, this could have deadly results. If you live in the plains of Kansas, maybe not so much.)
- marcosdumay 10y agoIt does not run off the road... And it pushes the break hard. What kind of ethical dilemma should it have on that example?
- Houshalter 10y agoI don't think it's overblown at all. There would be large advantages to programming your car to be more aggressive and risky.
- CodeCube 10y agoI'm curious to hear what these advantages would be ... not saying there aren't, just having a hard time thinking about what you might mean.
- Houshalter 10y agoGetting to your destination faster. Driving slightly faster than the speed limit, keeping less distance from cars, cutting them off, etc. I'm not sure if anyone would be willing or able to reprogram a self driving car to do that. But I do think its plausible people would abuse self driving cars. E.g. carelessly walking into traffic knowing the self driving cars will reliably still stop and be passive.
- femto 10y agoKeep in mind that self-driving cars will be loaded with sensors. Car's will basically be running continuous surveillance on their neighbours, and any "rogue" car will probably be reported by surrounding vehicles.
- robryan 10y agoIt is no different to a human driver deciding to speed.
- eru 10y agoWow, I just looked up "rolling coal". I can maybe get the political protest aspect---even if I disagree, but blowing the smoke at pedestrians, cyclist or even just other cars seems like one of the most dickish things ever.
- ams6110 10y agoI've seen it happen firsthand a couple of times. In both cases it was done after the cyclists stayed two abreast instead of singling up to let traffic pass.
- eru 10y agoDoes the law say anything about having to get into single file? (In any case, the law doesn't provide for this case of vigilantism. And two wrong wouldn't make a right.)
- mikeash 10y agoOf course not. The law typically requires slow traffic to pull over when safe and allow faster traffic to pass when there are more than a few cars backed up behind, but that's a bit different. But naturally the lack of legal standing doesn't stop drivers who think they have the right to pass all bicyclists as soon as possible.
- Thlom 10y agoCyclists, especially in a group should stay two or three abreast as it significantly reduces the passing length for overtaking cars. The only reason people complain about cyclists two abreast is that they want to overtake when it's not really safe to do so.
- astronautjones 10y agoIt's a pinnacle of ignorance
- kbart 10y agoIs it even legal in USA? In EU, that would make your card road illegal instantly.
- crispyambulance 10y agoOK, but given that there will likely be vast numbers of self-driving cars (tens to hundreds of millions), 99.9999% leave room for a small but steady stream of horrific cases where the machine and its maker are called into question. In wildly litigious societies like the USA, this could very well go beyond "not worth considering" and land firmly in the "existential threat" territory as far as self-driving car makers are concerned.
- mikeash 10y agoThere are about 30,000 traffic fatalities per year in the US. If ethical problems come up 0.0001% of the time, then that's one ethically questionable fatality every 33 years. I think the industry can withstand this. Especially when self-driving cars drop the annual fatality rate from 30,000 to 3,000, or whatever it ends up being.
- crispyambulance 10y agoI (and the person I was responding to) was being loose with the 99.9999 % figure. That number was pulled out of thin air. The point is there are "a lot" of cars and "many" accidents "some" of which will present an ethical challenge to self-driving cars. It does not take many accidents for these concerns to become a big problem for auto-makers given the litigious proclivities of places like the USA. Admittedly, if it really was every "33 years" that would be a good place to be. At this point, however, it remains to be proven that self-driving cars will even lower fatality rates.
- mikeash 10y agoOf course it's loose and pulled out of thin air. My point is that I think the "some" crashes which present an ethical challenge will be extremely rare. One in a million is pulled from my ass, but I think it's in the right ballpark. It's true that it's not proven that self-driving cars will lower fatality rates, but I'm perfectly comfortable with assuming they will. For one thing, nobody is going to put them in service until they're at least as safe as humans, so it's pretty much a tautology that they'll be safer; the relevant question is actually whether they'll ever be put in service. And considering how bad human drivers are, and how easily preventible many traffic fatalities are (about a third of traffic deaths in the US involve alcohol, another 10% or so involve distractions, and many others are due to something ridiculous like driving on the wrong side of the road), the bar is not really all that high. As long as self-driving cars don't get drunk, don't take their eyes off the road to text, and can reliably follow basic rules of the road, they'll already have a huge leg up.
- gohrt 10y agohttps://en.wikipedia.org/wiki/Rolling_coal https://en.wikipedia.org/wiki/Rolling_coal Amazing. Intentional, premeditated, assault and battery and vandalism carries a penalty of just $5,000.
- mc32 10y agoTrue. But pollution does not have a direct consequence _today_. If someone were to mod so that their vehicle would make every effort to protect the occupant regardless of fault, then that could have immediate grievous consequences, although rare they might be. Another more annoying aspect would be to mod your vehicle to have emergency vehicle preference, among other vehicles. I.e. impersonate/invehicleate (?) an emergency vehicle to get a nice right of way.
- TeMPOraL 10y ago> as is the case with "rolling coal." https://en.wikipedia.org/wiki/Rolling_coal https://en.wikipedia.org/wiki/Rolling_coal "Targets of coal rolling often include owners of hybrid vehicles (hence the nickname "Prius duster") as well as bicyclists and pedestrians." People really do shit like this out of spite? I have no words.
- mikeash 10y agoWelcome to America, where a big chunk of the population thinks that caring about the environment is a scam, and pollution regulations are just a way for liberals to acquire power.
- doriangray1 10y agoI respectfully disagree. Car crashes killed 38000 people in 2015, and injured around 8 million in the US alone. Designing any vehicle involves ethical choices about safety and the US has developed traffic laws over time to try to protect the safety of drivers. Autonomous vehicles will surely save lives compared to thier manual counterparts, but we shouldn't discount the ethical questions involved. Everyone has the Right to own property and to modify the property that they own. If I wanted to tear my car's muffler out and remove the break lights, that would be my constitutional right. However, the above right does not allow me to go driving on public roads at nighttime - because that would be violating other Driver's rights to safety. It is the same concept with self driving cars, and our legal system will eventually develope an enforceable code which (hopefully) will protect the rights of everyone in an equal measure. *I am an Environmental Law professional.
- mikeash 10y agoI totally agree that there are plenty of ethical choices in general for self-driving cars. I'm only addressing the idea that self-driving cars will need to make ethical choices about who to kill in a crash. That specific ethical conundrum is constantly brought up in discussions about autonomous vehicles, and I really don't see it happening with any frequency.
- gerbilly 10y ago>I think the ethical question is greatly overblown. I agree, especially since when I sit on my porch I see every other person on my street texting and driving, yet no one cares about this much greater hazard.
- afarrell 10y ago> 99.9999% of the time There are 109,277 workers in the US who drive to work and back on their own[1]. 6 nines means a quandry once every 10 weekdays...which I guess is actually pretty rare all things considered. Okay carry on. [1] page 7 of https://www.census.gov/hhes/commuting/files/2014/acs-32.pdf https://www.census.gov/hhes/commuting/files/2014/acs-32.pdf
- mikeash 10y agoEr, that number is in thousands. It's 109 million. But I'm talking about crashes, not normal drives. I think ethical considerations will be one-in-a-million events for crashes, not for normal drives.
- daveheq 10y agoMy guess is if the car had to decide whether to kill you or not, you'd think that choice was worth considering.