17 ms·

The complicated ethics of Tesla’s Autopilot

102 points by NN88 7y ago

mantap 7y ago
What kinds of accidents is autopilot going to prevent that automated accident avoidance cannot? Why does it have to drive the car to achieve the safety benefits? If you watch videos of autopilot preventing accidents on YouTube, it involves two stages: first the car detects a situation is likely to lead to a collision, then it brakes and steers the car to avoid it. There is no reason for autopilot to be controlling the vehicle before anticipating a collision. By contrast the deaths involving autopilot have all revolved around autopilot failing to perceive some object, or swerving the car into it. I don't know of any deaths that were caused by Tesla's accident avoidance feature.
- SilasX 7y ago
  >What kinds of accidents is autopilot going to prevent that automated accident avoidance cannot? Why does it have to drive the car to achieve the safety benefits? Accidents from people who shouldn't have been driving in the first place. Part of the benefit of self-driving cars is not needing a qualified human, so the blind, elderly, children, drunk, etc can [still be] drive[n around].
  - blowski 7y ago
    And people who are not very good at driving! Myself included.
    - Jamwinner 7y ago
      Thats why taxis and mass transit exist. If you suck at driving, please don't.
  - deleted 7y ago
    [deleted]
  - linuxftw 7y ago
    Maybe society should invest in providing services for these people rather than enriching more corporations. There are plenty of unemployed people, we can ferry a lot of passengers, no need for robots on the streets.
    - damnyou 7y ago
      Hmm, unemployment in the US is very low. What is happening is underemployment through gig economy jobs like Uber. You seem to be advocating for more of that.
  - rhino369 7y ago
    There is no reason to believe that a shitty driver with good automated accident avoidance would be worse than a self-driving car. Especially since no self-driving car is better than a shitty driver in 2019.
- mschuster91 7y ago
  > What kinds of accidents is autopilot going to prevent that automated accident avoidance cannot? Many accidents in stop-and-go traffic jams, people falling asleep due to being tired or bored (yes, that is a real issue) and veering off course, pedestrians stepping in front of the vehicle...
  - sz4kerto 7y ago
    These are all present (in one way or another) in many cars that don't drive themselves otherwise. (They're not that great at the moment, but these are part of accident avoidance, not self driving.)
  - braythwayt 7y ago
    Automated accident avoidance covers accidents in stop-and-go traffic jams. My 2012 Volvo already a rudimentary system along these lines that is designed specifically for stop-and-go traffic. Other Volvos have tools for detecting drivers losing alertness. Technology for helping to keep cars in their lanes already exists. Technology for stooping cars when pedestrians and bicycles appear in front and/or on the side when making a turn at an intersection already exists. Most of these systems need to get much, much better, but I think it's very reasonable to separate them from the general capability of driving a car autonomously from point A to point B. If we can develop a car to drive autonomously from point A to point B, and if we are granting that this system can also solve the problems you list above, I believe we can also develop most of the safety features that do not involve driving the car autonomously from A to B. I think this is more than a minor point to quibble over, as there are many extremely difficult problems to solve for fully autonomous driving that are unrelated to this kind of safety. For example, autonomous driving involves recognizing when the map is wrong. Despite driving maps being a so-called mature technology, a variety of map programs tell me to turn left on certain streets during prohibited times. There is a road that was closed at one end more than a decade ago, and when summoning a ride-share to an address on that street, we have to always text them not to follow the map directions, because it tries to send the car down the closed road, and today's ride-sharing drivers are terrible navigators and literally get lost. Solving autonomous driving solves problems like those AND safety problems, true. But if we just want to solve the safety problems, we can, and we can solve them faster and sooner by focusing on them.
- notus 7y ago
  If you have all the cars driving themselves they can function more like a swarm/grid of vehicles managed externally vs individual agents working against each other. This means you can have cars going like 80-100 mph but driving bumper to bumper.
  - jliptzin 7y ago
    We will have flying cars long before that happens
  - beojan 7y ago
    You mean like a train? Why not have trains instead?
    - lallysingh 7y ago
      Stops.
    - beerandt 7y ago
      Rails are expensive and limiting.
    - smachiz 7y ago
      Because trains can't take exits... unless the whole train wants to.
    - naavis 7y ago
      All cars are not going to the same place.
    - standyro 7y ago
      I love trains but for some stupid reason >$200 million per mile is the average cost for urban rail. We already sank a huge upfront investment into the interstate highway system, so we'll have to settle for a private enterprise solution with semi-autonomous vehicles. At least, until the US Congress becomes less dysfunctional...
  - dopylitty 7y ago
- Robotbeat 7y ago
  The automated accident avoidance systems will never become advanced enough without something like autopilot driving them.
- siliconc0w 7y ago
  Tesla wouldn't be where it is if it just had mere accident avoidance. Fewer would buy their cars and so they'd have worse training data and less money/viability. They probably wouldn't exist. The author is essentially correct but you don't see the whole picture because there aren't news articles when a Toyota or BMW LKAS system fails and causes an accident. We'd just have incumbent car manufacturers with worse systems. In fact I'd probably go the other way and advocate for requiring these systems on cars similar to seatbelts. At the very least collision accident avoidance and blind spot detection. Some manufacturers still make them options or, worse, reserve them for the highest premium trims which is kinda bullshit given the technology really isn't that expensive.
  - mantap 7y ago
    I don't think Tesla can claim credit for development of collision avoidance. They have been vocal proponents of it, but so have Volvo. Tesla just happened to start manufacturing mass market vehicles around the time of the ML revolution in the 2010s.
magwa101 7y ago
I honestly hate this "false baseline" story line that applies to almost all anti AI/automation comments. A person who is focused may be better than self driving vehicle but even that is debatable. However, given real life self driving are already much safer than real life human drivers. Think about pedestrians and bike rider safety too!!
- circlefavshape 7y ago
  > However, given real life self driving are already much safer than real life human drivers Citation needed
  - notus 7y ago
    The citation is that there is a wikipedia list of all the fatalities from self driving cars with only 6 entries on it, and most of them were due to human error still. https://en.wikipedia.org/wiki/List_of_self-driving_car_fatalities https://en.wikipedia.org/wiki/List_of_self-driving_car_fatal...
    - untog 7y ago
      That is not a good citation given the number of autonomous cars on the road compared to the number of user-driven cars.
    - simion314 7y ago
      You only forgot to consider that there are still drivers in the cars, so each time the human intervene it could have prevented an accident , I am not sure what is the average victims per accident say 2 , you can multiply it with the number of time the drivers had to take over and you have an upper limit, if you don't like this upper limit then we need better numbers but this companies keep them secret or under report the problems.
    - sorokod 7y ago
      According to this [1], fatality rates vary a lot from country to country. For example, in Germany and UK the rate is about 1/3 of the US rate. Will you cherry pick the geography to make your argument? [1] https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic...
claudeganon 7y ago
I’m very skeptical of this authoritarian market logic whereby we are told we must sacrifice others lives for the sake of “progress,” which is almost exclusively tied up with one person or business’ individual profit. While I don’t think autonomous cars are a particularly good investment for the future of transportation, does anyone doubt that with massive, publicly-funded investment we could not achieve them? And what’s more on terms where whatever “costs” were associated with them were subject to democratic checks and debate?
- Bud 7y ago
  1. We have democratic checks and debate now, regarding this issue. 2. I see no evidence that "massive, publicly-funded investment" would achieve this technological goal faster or better than via capitalistic means. Do you have any? 3. Lives will be lost regardless of how the tech is developed. Surely this is obvious. So what are you even talking about?
  - sverige 7y ago
    Re: 1. above, who do we sue when the autopilot kills someone? The owner of the vehicle? The manufacturer? The driver behind the wheel (if there is one and it's not the same as the owner)?
    - revscat 7y ago
      That is an area that is still being debated. I suspect it will depend on the level of autonomy the car claims to provide. If it is level 5, and perhaps doesn't even include a steering wheel or other means of controlling the vehicle, then there would be no rational basis to hold anyone inside liable for failure. If, on the other hand, it is level 3 or 4, and does provide a steering wheel, and prominently informs the user that they are ultimately responsible for the vehicle, then the driver would be liable in this case. This technology is still evolving, though, and the law will need to be adjusted accordingly.
    - fizx 7y ago
      Ideally, every self-driving car would have secondary insurance baked into its price. If someone dies, it's an insurance payout of a standardized amount.
xyzzy2020 7y ago
I don't get why people are up in arms over this: the average person drives like an I D I O T. A Tesla auto-pilot drives above average. And right now they are driving the worse they will ever drive. Therefore: more auto-pilot, less human death.
- peteradio 7y ago
  DATA OR GTFO
  - xyzzy2020 7y ago
    I know
- simion314 7y ago
  I don't think this is true, there are maybe many young drivers and some drunk drivers that cause a lot of accidents. But consider this, you are a decent driver and you need to send your children somewhere. Do you drive him yourself because you know that you will not speed or text or be drunk or you send it with a robot that is better then an idiot but worse then you ??? Sure if you were drunk or tired is safer to send him with the robot.
  - xyzzy2020 7y ago
    Yes that is the physiology barrier that you describe that will be difficult to overcome. The flaw is everyone thinks they will be a better driver than an AI, even though very few actually will be. So if I had to choose my children driving with an "average joe" / friend / etc vs an AI, I would say: the data says the AI will crash 25% less, therefore it is safer.
    - simion314 7y ago
      I assume you are better then a drunk teen that comes from a party and has 3 months of experience, The most road deaths that happen here in Romania are caused by young drivers cumming from parties late at night, maybe drunk, the car is full with people so you have 1 crash causing a lot of deaths, So the average driver from the stats is a terrible driver because the stats are skewed hard by the inexperienced, drunk or tired drivers. You can have both this next two facts true at the same time 1 replacing all drivers with AI is x% safer 2 replacing you an experienced responsible driver with an AI is less safer Does it make sense ?
Bud 7y ago
Anyone have a link to get past the paywall for this article?
- neonate 7y ago
  http://archive.is/N3bF3 http://archive.is/N3bF3
tcskeptic 7y ago
My general expectation is that Autonomous vehicles will at some point become safer than human drivers. However, even when that happens, when they make errors they will be errors that seem stupid and incomprehensible to humans. This will be because the failure mode of an ML driving model will be utterly different to the failure mode of human beings. This will lead to a lot of fear and uncertainty, humans want to comprehend what might kill them, and autonomous driving, while safer will kill them in bizarre ways.
- Whatarethese 7y ago
  I have the same assumption when a dumb human is tailgating and rear ends the car in from of them. That has been solved with frontal radar and emergency braking.
  - sokoloff 7y ago
    “been solved”? What percentage of the overall fleet is equipped for automated frontal collision avoidance? Under 5%? Under 2%?
    - standyro 7y ago
      Every Model 3 has a forward collision avoidance that has a better reaction time than I do. I know because it's saved my life already when I narrowly avoided a suddenly stopped car in front of me. Tesla's Autopilot (not full self driving) has its issues, mainly with vague lane markings, but collision avoidance isn't one of them. I know plenty of drivers who fall victim and cause accidents because of the same issues that Autopilot has: poor lane markings, not seeing cars that are immobile on the side of the highway.
      - sokoloff 7y ago
        There are over 1000 times more "not Model 3"s in the US than Model 3s. That was my only point about "this has been solved". It's not a knock against the Model 3.
- 7y ago
tannerc 7y ago
In most complex systems the reality is people will encounter harm. The commercial airline industry is a good example of progress through crashing. In 2017 there were zero reported accidentally deaths [1]. Compared to 1972, one of the most dangerous years to fly, where more than 2,300 accidental deaths occurred on commercial flights. How did we go from thousands of deaths to zero? Primarily through crashing. Then working to evaluate what went wrong and improve the design, processes, and programs around commercial aviation. As one pilot explains [2]: "One of the ways we’ve become so safe was to realize that our efforts were never going to be good enough." So yes, I think technologies like autopilot will save lives. But it won't be able to do it magically, unfortunately some harm will occur because we—in all our human wisdom—just can never accurately predict all possibilities. It's the same problem most self-driving companies are facing today, and the reason promises of self-driving being here today have fallen short. The world is far, far more complex than we realize... especially when we put a computer out into it and tell it to "go forth." https://www.reuters.com/article/us-aviation-safety/2017-safest-year-on-record-for-commercial-passenger-air-travel-groups-idUSKBN1EQ17L https://www.reuters.com/article/us-aviation-safety/2017-safe... https://www.usatoday.com/story/travel/columnist/cox/2018/01/07/ask-captain-why-aviation-so-safe-2017/1005183001/ https://www.usatoday.com/story/travel/columnist/cox/2018/01/...
- brosinante 7y ago
  I guess this is how people justify killing for peace. Tesla stock holders must be celebrating that their cars get to run a few people over in the name of "progress". (Well ok, they get to celebrate because their failed pet project gets to have another go - potato/potato)
  - everdev 7y ago
    If you find something that's pure good, let me know. There's always a price to pay with any activity. The question is just if the positive outweighs the negative. If you can't see both sides the you're probably polarized and biased to one approach. I think everyone wants better transportation and fewer deaths. But 0 deaths with an emerging driving technology sounds really hard. There are just so many edge cases and physics is pretty unforgiving on our bodies past 40mph.
simion314 7y ago
The problem is that "it could" there is no proof that after Tesla tests on public roads won't fail or go bankrupt. I am not aware of any proof that neural networks combined with current tech is enough. I know there are some misleading statistics around but those are not correct are biased and is not a proof.
- bob_theslob646 7y ago
  >The problem is that "it could" there is no proof that after Tesla tests on public roads won't fail or go bankrupt. Tesla will just raise more capital. It's unlikely that they will go bankrupt anytime soon. Unfortunately, there are nowhere near the misfortune of Faraday future.( They just announced bankruptcy today)
  - simion314 7y ago
    Assuming Tesla will do excellent, it could be Uber,Waymo or company X in country Y where some politicians will sign a piece of paper letting them killing people on the streets and later X fails because some bizarre reason or maybe they wanted to use n cameras but they had no idea that you needed double that so all that testing and people killed was for nothing. Then you have 12 companies testing with real people and not sharing data so killing about 12 times more people then needed.
krm01 7y ago
People were having the same discussions about airbags. They ended up saving quite a number of lives. The media should do more to contribute to a better world vs writing clickbaity articles that make adrevenue. It’s astonishing that somethinf like drunk driving for example, is portrayed less scary than autonomous vehicles.
omarforgotpwd 7y ago
Cool! That's me.
Beltiras 7y ago
If anyone else balked at the pilot story, here it is (and it's a bit juicy): https://en.wikipedia.org/wiki/Northwest_Airlines_Flight_188 https://en.wikipedia.org/wiki/Northwest_Airlines_Flight_188
omarforgotpwd 7y ago
Hey, cool! That's me in the story
jwr 7y ago
What bothers me about autonomous-driving software is that while you might argue that drivers are free to take the risk, I as a pedestrian (or another driver) did not sign up for the risk. There is an entirely new set of failure modes that we are not familiar with, and particularly as a pedestrian, it worries me greatly.
zarro 7y ago
The title makes it appear as if "Sacrifices" must be made, which is either a disingenuous click-bait or misguided, as its categorically untrue. The whole process of implementation of autopilot is based on the logic that its implementation is scaled as a function of the number of lives it SAVES through its utility. Meaning that as the percent automation increases, you should be able to see a net DECREASE in casualties in automated vehicles compared to non-automated vehicles. If we are not seeing this, then the rate of implementation may be too high, and the software needs to be corrected until it is. Nothing in life is perfect, there will always be accidents, the only question is can we create tools which will decrease the percent chance of them happening.
shrubble 7y ago
We could fix autopilot problems across the entire USA in approximately one year. However it would not have a winner take all, Wall Street frothiness pump and dump aspect to it. You put rfid tags in the asphalt, on the curbs and some kind of beacon on the telephone poles, or a 'wire in the road' (like a smarter version of the working system that GM built in the 70s) and then, because it is publicly owned infrastructure, everyone can use it. That goes against the current hypocritical faux-libertarianism that is espoused by some of our tech elites-but it would in fact work. I said a year, because that is how long it would take to install it. Edit to add: for fun, you can go on Mouser or Digikey and determine how to put the system together with off the shelf parts. Just add weather proofing...
- rasz 7y ago
  US is known for their world class infrastructure, and its regular maintenance. https://www.businessinsider.com/asce-gives-us-infrastructure-a-d-2017-3?IR=T https://www.businessinsider.com/asce-gives-us-infrastructure...
andrewla 7y ago
Systems created by humans adapt to humans, especially where they carry significant downside risks. The risk of death or injury is high enough in driving motor vehicles that we've taken a ton of precautions (licensing drivers, criminalizing reckless behavior, road signs, stop lights, intersections design, freeway design) to try to minimize those risks, and the designs have adapted to how humans drive. Because there is skin in the game, people drive carefully -- drivers who are consistently reckless are removed from the system. Pedestrians and other non-drivers have adapted their behavior as well to reduce the risks. Autonomous drivers have the problem that they do not carry skin in the game -- they are not removed from the system for bad behavior. In the short term that just gives us an ethical quandary about who is responsible for deaths caused by autonomous systems. In the long term it presents much higher risks as non-autonomous-drivers (both drivers who are not autonomous and humans who are not driving) will adapt their behavior to their mental model of the behavior of autonomous vehicles. Pedestrians will adapt their behavior correspondingly, creating even more dangerous scenarios because the interaction with sufficiently complex system is inherently unpredictable. We rely on the fact that we can build an approximate mental model of human behavior as it pertains to other drivers to predict what their behavior will be, and we constrain that behavior via rules and legal deterrents so that they are more predictable. When we are 50% autonomous, what happens to someone how deliberately tries to cross a busy highway? Other systems, like trains, we have warnings for -- "implacable vehicle coming that will destroy anything left in its path", but for autonomous vehicles? The priority on keeping humans alive will cause situations like this to require building stricter behavioral systems to try to limit people's behavior, but the complexity of the underlying behavior is real -- that is, most of the time, crossing an autonomous freeway is actually a safe operation because of the way that they cars are designed -- so there's no feedback that will cause pedestrians to modulate their behavior. Similarly, the driving systems themselves are designed with some elements of human behavior implicitly included -- as these behaviors drift due to the coupling of complex systems, it's not clear how we can directly adapt the behavior of the autonomous drivers.
duaoebg 7y ago
Like many people I accept that there are unavoidable deaths in the name of progress. What I don't like is that I cannot opt out of this experiment. I believe many of these deaths are needless and due to cost savings and sloppiness. Tesla to me seems less like a center of excellence and more like a group of people who can put up with Elon. Presumably made easier by telling him what he wants to hear. While the average driver may be bad, the average hour driven is by good drivers. Plus I don't like the idea of a Tesla can crashing into me. Seems like something that could be weaponized.
ryanmarsh 7y ago
I’ve posted this several times on HN but I’ll post it again. I once had the opportunity to interview a renowned cancer doctor and researcher at MD Anderson hospital. He told me that we would never again see the kinds of advances we saw in the early days (70’s and 80’s) of cancer research because those days were the Wild West. He basically said they did many reckless things, but that’s how they stumbled upon treatments such as the protocol for using steroids with chemo for acute lymphoblastic leukemia. Ok, so high risk experimentation lead to exaptation and innovation. Not sure I want to see that happen with self driving cars but the other part of me knows that’s how complex systems “evolve”.
CrackerNews 7y ago
This headline reads like the justifications for the rapid industrialization programs of the USSR and Communist China.
loceng 7y ago
It reminds me of how the violence-harm that exists in a democracy vs. a tyranny isn't evenly distributed: there may be very little violence (due to control mechanisms) in a tyrant controlled state - at least until you're their target and then slaughter, mass murder can occur.
6gvONxR4sf7o 7y ago
Think how many lives we could save by loosing regulations on medical experimentation. If we killed not that many people we could save billions. Good thing we already have decades (even centuries) of research ethics we’ve developed to deal with situations like this. Too bad Tesla is ignoring them. The proposal in the article to treat these like (potential) medical breakthroughs is spot on. I think we all agree that autonomous cars will eventually save lives, but that was true 50 years ago too. The question is whether these autonomous cars will save lives. We’ve developed the ethics. We’ve developed the standards of evidence. In seems like a no brained to require them here.
sorokod 7y ago
Wouldn't public transport "save millions" much more efficiently?
- twic 7y ago
  Yes, but we haven't worked out how use public transport to divert billions of dollars of investment into paying machine learning specialists telephone number salaries, so it's not actually plausible.
  - sorokod 7y ago
    Sarcasm aside, the discussion seems to be driven by technology fetish and a promise to buy / sell more stuff.
    - twic 7y ago
      Looks like HN has a new strapline!
  - standyro 7y ago
    Sounds like the next YC startup...
deleted 7y ago
[deleted]
Piskvorrr 7y ago
The ending is telling: [I trust the AV, but not enough to risk hitting a cyclist right during the interview. More safety...at the expense of people sans Teslas] (paraphrasing mine)
neonate 7y ago
http://archive.is/N3bF3 http://archive.is/N3bF3
zaroth 7y ago
NHTSA found that the Economic and Societal impact of Motor Vehicle Crashes, when taking into account harm from pain & suffering and lost wages, is about $877 billion per year. [1] That means we have a tremendous incentive to make investments which make the roads safer, and lower the crash rate. The single biggest technological advancement to make this possible (without eliminating cars) will be autonomous driving. Today the lowest hanging fruit for self-driving is algorithmic work which allows the car to navigate safely based on its own sensor suite, and mostly by ignoring map data which could be unreliable. Reliance on highly accurate maps and roadway infrastructure is not really a pressing concern, because the cost/benefits of investing in improving the internal algorithm is much higher than the cost/benefits of adding roadway infrastructure for such a small percentage of vehicles to leverage. I believe there will be a cross-over point, where safety for a significant number of vehicles can be significantly improved by making significant changes to road infrastructure to support self-driving. When that day comes, I hope that the USG will be ready to invest on the order of $1 trillion to deploy that infrastructure and ultimately, mandate all new vehicles on public roads are self-driving. I've said it in many comments before - it is a moral imperative to get computers and not humans "behind the wheel". We can afford to spend trillions of dollars to get there, but that's not to say we get there faster by spending trillions of dollars today. [1] - https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/2015sae-blincoe-costs_of_crashes2010.pdf https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/2015sae-blin...
kwhitefoot 7y ago
This is a really misleading and inflammatory article. " reading books, napping, strumming a ukulele, or having sex." Perhaps it was like that years ago, long before the Model 3 but it isn't like that now. My 2015 Model S would complain long before I could complete a sex act. Or perhaps Bloomberg's hack suffers from premature ejaculation.
- CobrastanJorji 7y ago
  The author is alluding to a specific incident, which was posted to porn sites. Elon Musk commented about it on Twitter (because of course he did), saying "Turns out there’s more ways to use Autopilot than we imagined."
zaroth 7y ago
I take issue with the headline. The fact that crashes still do and will occur even while AutoPilot is on does not mean that AutoPilot is legally or ethically responsible for the crash, until the system is advertised as a Level 4 system. Only at that point we can look at the accident rate of the L4 system and decide if those accidents are the fault of the autonomy, or if the algorithms could be improved to have avoided an accident that was even another driver's fault. Distracted driving is a leading cause of crashes today, and particularly if you use AutoPilot, you really see the ridiculous numbers of distracted drivers all around you. Many drivers choose to use their phones instead of looking at the road, whether they have AutoPilot or not. It's dangerous in both cases, but less-so if you have AutoPilot on. In either case, in a L2 system, if the car crashes while the driver is on their cell phone, it’s the driver’s fault regardless of whether there was AutoPilot enabled or not.
russellbeattie 7y ago
Caution: Extreme opinion below. Artificial Intelligence is a tool. Just like anyone who uses a hammer, backhoe or dump truck is responsible for what happens when they are using it, the same is true for AI. As a tool, all AI systems should be designed to always do the most harm to the user of the AI first. It should be embedded into every autopilot-like system, and users should be aware of that choice. It's the only moral and ethically correct solution. Let's say your AI-controlled car is driving at speed and turning the corner, there suddenly appears a little girl in the road in front of you. It can either swerve into a wall or run down the girl. A human might not be able to make the decision in time, but an AI system would. It needs to be programmed to always hit the wall. Though this might seem extreme, the opposite is completely immoral. If it was a human driving, one could assume bad luck to be put in that situation: aka innocent until proven guilty. But for an AI system, we have to assume the opposite: The AI should always be presumed to be faulty. Therefore the user of that AI is culpable of putting it in that situation. And just like any other tool, the manufacturer of that tool is legally responsible for its quality and reliability. The opposite of the above means we'll all be riding around in autonomous tanks, aggressively maneuvering around each other at higher and higher speeds, pedestrians and others be damned.
mindfulplay 7y ago
All of these efforts will eventually result in an amazing world where there are a lot fewer or zero crashes. Perhaps even change how cities or homes are built/organized. No doubt. But the really troubling thing is the attitude and approach: this is not an "experimental software rollout" that somehow Tesla seems to advocate. You cannot use live human beings in real streets with very little safety net to "roll out" your self driving torpedoes. That just doesn't make sense and seems very irresponsible. I would rather not have silicon valley mentality when it comes to cars or healthcare or even banking. Real people's lives are at stake.

102 points by NN88 7y ago

What kinds of accidents is autopilot going to prevent that automated accident avoidance cannot? Why does it have to drive the car to achieve the safety benefits? If you watch videos of autopilot preventing accidents on YouTube, it involves two stages: first the car detects a situation is likely to lead to a collision, then it brakes and steers the car to avoid it. There is no reason for autopilot to be controlling the vehicle before anticipating a collision. By contrast the deaths involving autopilot have all revolved around autopilot failing to perceive some object, or swerving the car into it. I don't know of any deaths that were caused by Tesla's accident avoidance feature.

SilasX 7y ago
>What kinds of accidents is autopilot going to prevent that automated accident avoidance cannot? Why does it have to drive the car to achieve the safety benefits? Accidents from people who shouldn't have been driving in the first place. Part of the benefit of self-driving cars is not needing a qualified human, so the blind, elderly, children, drunk, etc can [still be] drive[n around].
- blowski 7y ago
  And people who are not very good at driving! Myself included.
  - Jamwinner 7y ago
    Thats why taxis and mass transit exist. If you suck at driving, please don't.
- deleted 7y ago
  [deleted]
- linuxftw 7y ago
  Maybe society should invest in providing services for these people rather than enriching more corporations. There are plenty of unemployed people, we can ferry a lot of passengers, no need for robots on the streets.
  - damnyou 7y ago
    Hmm, unemployment in the US is very low. What is happening is underemployment through gig economy jobs like Uber. You seem to be advocating for more of that.
- rhino369 7y ago
  There is no reason to believe that a shitty driver with good automated accident avoidance would be worse than a self-driving car. Especially since no self-driving car is better than a shitty driver in 2019.
mschuster91 7y ago
> What kinds of accidents is autopilot going to prevent that automated accident avoidance cannot? Many accidents in stop-and-go traffic jams, people falling asleep due to being tired or bored (yes, that is a real issue) and veering off course, pedestrians stepping in front of the vehicle...
- sz4kerto 7y ago
  These are all present (in one way or another) in many cars that don't drive themselves otherwise. (They're not that great at the moment, but these are part of accident avoidance, not self driving.)
- braythwayt 7y ago
  Automated accident avoidance covers accidents in stop-and-go traffic jams. My 2012 Volvo already a rudimentary system along these lines that is designed specifically for stop-and-go traffic. Other Volvos have tools for detecting drivers losing alertness. Technology for helping to keep cars in their lanes already exists. Technology for stooping cars when pedestrians and bicycles appear in front and/or on the side when making a turn at an intersection already exists. Most of these systems need to get much, much better, but I think it's very reasonable to separate them from the general capability of driving a car autonomously from point A to point B. If we can develop a car to drive autonomously from point A to point B, and if we are granting that this system can also solve the problems you list above, I believe we can also develop most of the safety features that do not involve driving the car autonomously from A to B. I think this is more than a minor point to quibble over, as there are many extremely difficult problems to solve for fully autonomous driving that are unrelated to this kind of safety. For example, autonomous driving involves recognizing when the map is wrong. Despite driving maps being a so-called mature technology, a variety of map programs tell me to turn left on certain streets during prohibited times. There is a road that was closed at one end more than a decade ago, and when summoning a ride-share to an address on that street, we have to always text them not to follow the map directions, because it tries to send the car down the closed road, and today's ride-sharing drivers are terrible navigators and literally get lost. Solving autonomous driving solves problems like those AND safety problems, true. But if we just want to solve the safety problems, we can, and we can solve them faster and sooner by focusing on them.
notus 7y ago
If you have all the cars driving themselves they can function more like a swarm/grid of vehicles managed externally vs individual agents working against each other. This means you can have cars going like 80-100 mph but driving bumper to bumper.
- jliptzin 7y ago
  We will have flying cars long before that happens
- beojan 7y ago
  You mean like a train? Why not have trains instead?
  - lallysingh 7y ago
    Stops.
  - beerandt 7y ago
    Rails are expensive and limiting.
  - smachiz 7y ago
    Because trains can't take exits... unless the whole train wants to.
  - naavis 7y ago
    All cars are not going to the same place.
  - standyro 7y ago
    I love trains but for some stupid reason >$200 million per mile is the average cost for urban rail. We already sank a huge upfront investment into the interstate highway system, so we'll have to settle for a private enterprise solution with semi-autonomous vehicles. At least, until the US Congress becomes less dysfunctional...
- dopylitty 7y ago
Robotbeat 7y ago
The automated accident avoidance systems will never become advanced enough without something like autopilot driving them.
siliconc0w 7y ago
Tesla wouldn't be where it is if it just had mere accident avoidance. Fewer would buy their cars and so they'd have worse training data and less money/viability. They probably wouldn't exist. The author is essentially correct but you don't see the whole picture because there aren't news articles when a Toyota or BMW LKAS system fails and causes an accident. We'd just have incumbent car manufacturers with worse systems. In fact I'd probably go the other way and advocate for requiring these systems on cars similar to seatbelts. At the very least collision accident avoidance and blind spot detection. Some manufacturers still make them options or, worse, reserve them for the highest premium trims which is kinda bullshit given the technology really isn't that expensive.
- mantap 7y ago
  I don't think Tesla can claim credit for development of collision avoidance. They have been vocal proponents of it, but so have Volvo. Tesla just happened to start manufacturing mass market vehicles around the time of the ML revolution in the 2010s.

magwa101 7y ago

I honestly hate this "false baseline" story line that applies to almost all anti AI/automation comments. A person who is focused may be better than self driving vehicle but even that is debatable. However, given real life self driving are already much safer than real life human drivers. Think about pedestrians and bike rider safety too!!

circlefavshape 7y ago
> However, given real life self driving are already much safer than real life human drivers Citation needed
- notus 7y ago
  The citation is that there is a wikipedia list of all the fatalities from self driving cars with only 6 entries on it, and most of them were due to human error still. https://en.wikipedia.org/wiki/List_of_self-driving_car_fatalities https://en.wikipedia.org/wiki/List_of_self-driving_car_fatal...
  - untog 7y ago
    That is not a good citation given the number of autonomous cars on the road compared to the number of user-driven cars.
  - simion314 7y ago
    You only forgot to consider that there are still drivers in the cars, so each time the human intervene it could have prevented an accident , I am not sure what is the average victims per accident say 2 , you can multiply it with the number of time the drivers had to take over and you have an upper limit, if you don't like this upper limit then we need better numbers but this companies keep them secret or under report the problems.
  - sorokod 7y ago
    According to this [1], fatality rates vary a lot from country to country. For example, in Germany and UK the rate is about 1/3 of the US rate. Will you cherry pick the geography to make your argument? [1] https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate https://en.m.wikipedia.org/wiki/List_of_countries_by_traffic...

claudeganon 7y ago

I’m very skeptical of this authoritarian market logic whereby we are told we must sacrifice others lives for the sake of “progress,” which is almost exclusively tied up with one person or business’ individual profit. While I don’t think autonomous cars are a particularly good investment for the future of transportation, does anyone doubt that with massive, publicly-funded investment we could not achieve them? And what’s more on terms where whatever “costs” were associated with them were subject to democratic checks and debate?

Bud 7y ago
1. We have democratic checks and debate now, regarding this issue. 2. I see no evidence that "massive, publicly-funded investment" would achieve this technological goal faster or better than via capitalistic means. Do you have any? 3. Lives will be lost regardless of how the tech is developed. Surely this is obvious. So what are you even talking about?
- sverige 7y ago
  Re: 1. above, who do we sue when the autopilot kills someone? The owner of the vehicle? The manufacturer? The driver behind the wheel (if there is one and it's not the same as the owner)?
  - revscat 7y ago
    That is an area that is still being debated. I suspect it will depend on the level of autonomy the car claims to provide. If it is level 5, and perhaps doesn't even include a steering wheel or other means of controlling the vehicle, then there would be no rational basis to hold anyone inside liable for failure. If, on the other hand, it is level 3 or 4, and does provide a steering wheel, and prominently informs the user that they are ultimately responsible for the vehicle, then the driver would be liable in this case. This technology is still evolving, though, and the law will need to be adjusted accordingly.
  - fizx 7y ago
    Ideally, every self-driving car would have secondary insurance baked into its price. If someone dies, it's an insurance payout of a standardized amount.

xyzzy2020 7y ago

I don't get why people are up in arms over this: the average person drives like an I D I O T. A Tesla auto-pilot drives above average. And right now they are driving the worse they will ever drive. Therefore: more auto-pilot, less human death.

peteradio 7y ago
DATA OR GTFO
- xyzzy2020 7y ago
  I know
simion314 7y ago
I don't think this is true, there are maybe many young drivers and some drunk drivers that cause a lot of accidents. But consider this, you are a decent driver and you need to send your children somewhere. Do you drive him yourself because you know that you will not speed or text or be drunk or you send it with a robot that is better then an idiot but worse then you ??? Sure if you were drunk or tired is safer to send him with the robot.
- xyzzy2020 7y ago
  Yes that is the physiology barrier that you describe that will be difficult to overcome. The flaw is everyone thinks they will be a better driver than an AI, even though very few actually will be. So if I had to choose my children driving with an "average joe" / friend / etc vs an AI, I would say: the data says the AI will crash 25% less, therefore it is safer.
  - simion314 7y ago
    I assume you are better then a drunk teen that comes from a party and has 3 months of experience, The most road deaths that happen here in Romania are caused by young drivers cumming from parties late at night, maybe drunk, the car is full with people so you have 1 crash causing a lot of deaths, So the average driver from the stats is a terrible driver because the stats are skewed hard by the inexperienced, drunk or tired drivers. You can have both this next two facts true at the same time 1 replacing all drivers with AI is x% safer 2 replacing you an experienced responsible driver with an AI is less safer Does it make sense ?

Bud 7y ago

Anyone have a link to get past the paywall for this article?

neonate 7y ago
http://archive.is/N3bF3 http://archive.is/N3bF3

tcskeptic 7y ago

My general expectation is that Autonomous vehicles will at some point become safer than human drivers. However, even when that happens, when they make errors they will be errors that seem stupid and incomprehensible to humans. This will be because the failure mode of an ML driving model will be utterly different to the failure mode of human beings. This will lead to a lot of fear and uncertainty, humans want to comprehend what might kill them, and autonomous driving, while safer will kill them in bizarre ways.

Whatarethese 7y ago
I have the same assumption when a dumb human is tailgating and rear ends the car in from of them. That has been solved with frontal radar and emergency braking.
- sokoloff 7y ago
  “been solved”? What percentage of the overall fleet is equipped for automated frontal collision avoidance? Under 5%? Under 2%?
  - standyro 7y ago
    Every Model 3 has a forward collision avoidance that has a better reaction time than I do. I know because it's saved my life already when I narrowly avoided a suddenly stopped car in front of me. Tesla's Autopilot (not full self driving) has its issues, mainly with vague lane markings, but collision avoidance isn't one of them. I know plenty of drivers who fall victim and cause accidents because of the same issues that Autopilot has: poor lane markings, not seeing cars that are immobile on the side of the highway.
    - sokoloff 7y ago
      There are over 1000 times more "not Model 3"s in the US than Model 3s. That was my only point about "this has been solved". It's not a knock against the Model 3.
7y ago

tannerc 7y ago

In most complex systems the reality is people will encounter harm. The commercial airline industry is a good example of progress through crashing. In 2017 there were zero reported accidentally deaths [1]. Compared to 1972, one of the most dangerous years to fly, where more than 2,300 accidental deaths occurred on commercial flights. How did we go from thousands of deaths to zero? Primarily through crashing. Then working to evaluate what went wrong and improve the design, processes, and programs around commercial aviation. As one pilot explains [2]: "One of the ways we’ve become so safe was to realize that our efforts were never going to be good enough." So yes, I think technologies like autopilot will save lives. But it won't be able to do it magically, unfortunately some harm will occur because we—in all our human wisdom—just can never accurately predict all possibilities. It's the same problem most self-driving companies are facing today, and the reason promises of self-driving being here today have fallen short. The world is far, far more complex than we realize... especially when we put a computer out into it and tell it to "go forth." https://www.reuters.com/article/us-aviation-safety/2017-safest-year-on-record-for-commercial-passenger-air-travel-groups-idUSKBN1EQ17L https://www.reuters.com/article/us-aviation-safety/2017-safe... https://www.usatoday.com/story/travel/columnist/cox/2018/01/07/ask-captain-why-aviation-so-safe-2017/1005183001/ https://www.usatoday.com/story/travel/columnist/cox/2018/01/...

brosinante 7y ago
I guess this is how people justify killing for peace. Tesla stock holders must be celebrating that their cars get to run a few people over in the name of "progress". (Well ok, they get to celebrate because their failed pet project gets to have another go - potato/potato)
- everdev 7y ago
  If you find something that's pure good, let me know. There's always a price to pay with any activity. The question is just if the positive outweighs the negative. If you can't see both sides the you're probably polarized and biased to one approach. I think everyone wants better transportation and fewer deaths. But 0 deaths with an emerging driving technology sounds really hard. There are just so many edge cases and physics is pretty unforgiving on our bodies past 40mph.

simion314 7y ago

The problem is that "it could" there is no proof that after Tesla tests on public roads won't fail or go bankrupt. I am not aware of any proof that neural networks combined with current tech is enough. I know there are some misleading statistics around but those are not correct are biased and is not a proof.

bob_theslob646 7y ago
>The problem is that "it could" there is no proof that after Tesla tests on public roads won't fail or go bankrupt. Tesla will just raise more capital. It's unlikely that they will go bankrupt anytime soon. Unfortunately, there are nowhere near the misfortune of Faraday future.( They just announced bankruptcy today)
- simion314 7y ago
  Assuming Tesla will do excellent, it could be Uber,Waymo or company X in country Y where some politicians will sign a piece of paper letting them killing people on the streets and later X fails because some bizarre reason or maybe they wanted to use n cameras but they had no idea that you needed double that so all that testing and people killed was for nothing. Then you have 12 companies testing with real people and not sharing data so killing about 12 times more people then needed.

krm01 7y ago

People were having the same discussions about airbags. They ended up saving quite a number of lives. The media should do more to contribute to a better world vs writing clickbaity articles that make adrevenue. It’s astonishing that somethinf like drunk driving for example, is portrayed less scary than autonomous vehicles.

omarforgotpwd 7y ago

Cool! That's me.

Beltiras 7y ago

If anyone else balked at the pilot story, here it is (and it's a bit juicy): https://en.wikipedia.org/wiki/Northwest_Airlines_Flight_188 https://en.wikipedia.org/wiki/Northwest_Airlines_Flight_188

omarforgotpwd 7y ago

Hey, cool! That's me in the story

jwr 7y ago

What bothers me about autonomous-driving software is that while you might argue that drivers are free to take the risk, I as a pedestrian (or another driver) did not sign up for the risk. There is an entirely new set of failure modes that we are not familiar with, and particularly as a pedestrian, it worries me greatly.

zarro 7y ago

The title makes it appear as if "Sacrifices" must be made, which is either a disingenuous click-bait or misguided, as its categorically untrue. The whole process of implementation of autopilot is based on the logic that its implementation is scaled as a function of the number of lives it SAVES through its utility. Meaning that as the percent automation increases, you should be able to see a net DECREASE in casualties in automated vehicles compared to non-automated vehicles. If we are not seeing this, then the rate of implementation may be too high, and the software needs to be corrected until it is. Nothing in life is perfect, there will always be accidents, the only question is can we create tools which will decrease the percent chance of them happening.

shrubble 7y ago

We could fix autopilot problems across the entire USA in approximately one year. However it would not have a winner take all, Wall Street frothiness pump and dump aspect to it. You put rfid tags in the asphalt, on the curbs and some kind of beacon on the telephone poles, or a 'wire in the road' (like a smarter version of the working system that GM built in the 70s) and then, because it is publicly owned infrastructure, everyone can use it. That goes against the current hypocritical faux-libertarianism that is espoused by some of our tech elites-but it would in fact work. I said a year, because that is how long it would take to install it. Edit to add: for fun, you can go on Mouser or Digikey and determine how to put the system together with off the shelf parts. Just add weather proofing...

rasz 7y ago
US is known for their world class infrastructure, and its regular maintenance. https://www.businessinsider.com/asce-gives-us-infrastructure-a-d-2017-3?IR=T https://www.businessinsider.com/asce-gives-us-infrastructure...

andrewla 7y ago

Systems created by humans adapt to humans, especially where they carry significant downside risks. The risk of death or injury is high enough in driving motor vehicles that we've taken a ton of precautions (licensing drivers, criminalizing reckless behavior, road signs, stop lights, intersections design, freeway design) to try to minimize those risks, and the designs have adapted to how humans drive. Because there is skin in the game, people drive carefully -- drivers who are consistently reckless are removed from the system. Pedestrians and other non-drivers have adapted their behavior as well to reduce the risks. Autonomous drivers have the problem that they do not carry skin in the game -- they are not removed from the system for bad behavior. In the short term that just gives us an ethical quandary about who is responsible for deaths caused by autonomous systems. In the long term it presents much higher risks as non-autonomous-drivers (both drivers who are not autonomous and humans who are not driving) will adapt their behavior to their mental model of the behavior of autonomous vehicles. Pedestrians will adapt their behavior correspondingly, creating even more dangerous scenarios because the interaction with sufficiently complex system is inherently unpredictable. We rely on the fact that we can build an approximate mental model of human behavior as it pertains to other drivers to predict what their behavior will be, and we constrain that behavior via rules and legal deterrents so that they are more predictable. When we are 50% autonomous, what happens to someone how deliberately tries to cross a busy highway? Other systems, like trains, we have warnings for -- "implacable vehicle coming that will destroy anything left in its path", but for autonomous vehicles? The priority on keeping humans alive will cause situations like this to require building stricter behavioral systems to try to limit people's behavior, but the complexity of the underlying behavior is real -- that is, most of the time, crossing an autonomous freeway is actually a safe operation because of the way that they cars are designed -- so there's no feedback that will cause pedestrians to modulate their behavior. Similarly, the driving systems themselves are designed with some elements of human behavior implicitly included -- as these behaviors drift due to the coupling of complex systems, it's not clear how we can directly adapt the behavior of the autonomous drivers.

duaoebg 7y ago

Like many people I accept that there are unavoidable deaths in the name of progress. What I don't like is that I cannot opt out of this experiment. I believe many of these deaths are needless and due to cost savings and sloppiness. Tesla to me seems less like a center of excellence and more like a group of people who can put up with Elon. Presumably made easier by telling him what he wants to hear. While the average driver may be bad, the average hour driven is by good drivers. Plus I don't like the idea of a Tesla can crashing into me. Seems like something that could be weaponized.

ryanmarsh 7y ago

I’ve posted this several times on HN but I’ll post it again. I once had the opportunity to interview a renowned cancer doctor and researcher at MD Anderson hospital. He told me that we would never again see the kinds of advances we saw in the early days (70’s and 80’s) of cancer research because those days were the Wild West. He basically said they did many reckless things, but that’s how they stumbled upon treatments such as the protocol for using steroids with chemo for acute lymphoblastic leukemia. Ok, so high risk experimentation lead to exaptation and innovation. Not sure I want to see that happen with self driving cars but the other part of me knows that’s how complex systems “evolve”.

CrackerNews 7y ago

This headline reads like the justifications for the rapid industrialization programs of the USSR and Communist China.

loceng 7y ago

It reminds me of how the violence-harm that exists in a democracy vs. a tyranny isn't evenly distributed: there may be very little violence (due to control mechanisms) in a tyrant controlled state - at least until you're their target and then slaughter, mass murder can occur.

6gvONxR4sf7o 7y ago

Think how many lives we could save by loosing regulations on medical experimentation. If we killed not that many people we could save billions. Good thing we already have decades (even centuries) of research ethics we’ve developed to deal with situations like this. Too bad Tesla is ignoring them. The proposal in the article to treat these like (potential) medical breakthroughs is spot on. I think we all agree that autonomous cars will eventually save lives, but that was true 50 years ago too. The question is whether these autonomous cars will save lives. We’ve developed the ethics. We’ve developed the standards of evidence. In seems like a no brained to require them here.

sorokod 7y ago

Wouldn't public transport "save millions" much more efficiently?

twic 7y ago
Yes, but we haven't worked out how use public transport to divert billions of dollars of investment into paying machine learning specialists telephone number salaries, so it's not actually plausible.
- sorokod 7y ago
  Sarcasm aside, the discussion seems to be driven by technology fetish and a promise to buy / sell more stuff.
  - twic 7y ago
    Looks like HN has a new strapline!
- standyro 7y ago
  Sounds like the next YC startup...

deleted 7y ago

[deleted]

Piskvorrr 7y ago

The ending is telling: [I trust the AV, but not enough to risk hitting a cyclist right during the interview. More safety...at the expense of people sans Teslas] (paraphrasing mine)

neonate 7y ago

http://archive.is/N3bF3 http://archive.is/N3bF3

zaroth 7y ago

NHTSA found that the Economic and Societal impact of Motor Vehicle Crashes, when taking into account harm from pain & suffering and lost wages, is about $877 billion per year. [1] That means we have a tremendous incentive to make investments which make the roads safer, and lower the crash rate. The single biggest technological advancement to make this possible (without eliminating cars) will be autonomous driving. Today the lowest hanging fruit for self-driving is algorithmic work which allows the car to navigate safely based on its own sensor suite, and mostly by ignoring map data which could be unreliable. Reliance on highly accurate maps and roadway infrastructure is not really a pressing concern, because the cost/benefits of investing in improving the internal algorithm is much higher than the cost/benefits of adding roadway infrastructure for such a small percentage of vehicles to leverage. I believe there will be a cross-over point, where safety for a significant number of vehicles can be significantly improved by making significant changes to road infrastructure to support self-driving. When that day comes, I hope that the USG will be ready to invest on the order of $1 trillion to deploy that infrastructure and ultimately, mandate all new vehicles on public roads are self-driving. I've said it in many comments before - it is a moral imperative to get computers and not humans "behind the wheel". We can afford to spend trillions of dollars to get there, but that's not to say we get there faster by spending trillions of dollars today. [1] - https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/2015sae-blincoe-costs_of_crashes2010.pdf https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/2015sae-blin...

kwhitefoot 7y ago

This is a really misleading and inflammatory article. " reading books, napping, strumming a ukulele, or having sex." Perhaps it was like that years ago, long before the Model 3 but it isn't like that now. My 2015 Model S would complain long before I could complete a sex act. Or perhaps Bloomberg's hack suffers from premature ejaculation.

CobrastanJorji 7y ago
The author is alluding to a specific incident, which was posted to porn sites. Elon Musk commented about it on Twitter (because of course he did), saying "Turns out there’s more ways to use Autopilot than we imagined."

zaroth 7y ago

I take issue with the headline. The fact that crashes still do and will occur even while AutoPilot is on does not mean that AutoPilot is legally or ethically responsible for the crash, until the system is advertised as a Level 4 system. Only at that point we can look at the accident rate of the L4 system and decide if those accidents are the fault of the autonomy, or if the algorithms could be improved to have avoided an accident that was even another driver's fault. Distracted driving is a leading cause of crashes today, and particularly if you use AutoPilot, you really see the ridiculous numbers of distracted drivers all around you. Many drivers choose to use their phones instead of looking at the road, whether they have AutoPilot or not. It's dangerous in both cases, but less-so if you have AutoPilot on. In either case, in a L2 system, if the car crashes while the driver is on their cell phone, it’s the driver’s fault regardless of whether there was AutoPilot enabled or not.

russellbeattie 7y ago

Caution: Extreme opinion below. Artificial Intelligence is a tool. Just like anyone who uses a hammer, backhoe or dump truck is responsible for what happens when they are using it, the same is true for AI. As a tool, all AI systems should be designed to always do the most harm to the user of the AI first. It should be embedded into every autopilot-like system, and users should be aware of that choice. It's the only moral and ethically correct solution. Let's say your AI-controlled car is driving at speed and turning the corner, there suddenly appears a little girl in the road in front of you. It can either swerve into a wall or run down the girl. A human might not be able to make the decision in time, but an AI system would. It needs to be programmed to always hit the wall. Though this might seem extreme, the opposite is completely immoral. If it was a human driving, one could assume bad luck to be put in that situation: aka innocent until proven guilty. But for an AI system, we have to assume the opposite: The AI should always be presumed to be faulty. Therefore the user of that AI is culpable of putting it in that situation. And just like any other tool, the manufacturer of that tool is legally responsible for its quality and reliability. The opposite of the above means we'll all be riding around in autonomous tanks, aggressively maneuvering around each other at higher and higher speeds, pedestrians and others be damned.

mindfulplay 7y ago

All of these efforts will eventually result in an amazing world where there are a lot fewer or zero crashes. Perhaps even change how cities or homes are built/organized. No doubt. But the really troubling thing is the attitude and approach: this is not an "experimental software rollout" that somehow Tesla seems to advocate. You cannot use live human beings in real streets with very little safety net to "roll out" your self driving torpedoes. That just doesn't make sense and seems very irresponsible. I would rather not have silicon valley mentality when it comes to cars or healthcare or even banking. Real people's lives are at stake.