17 ms·

Tesla’s Autopilot found partly to blame for 2018 crash on the 405

152 points by casefields 7y ago

qzx_pierri 7y ago
$0.75 has been deposited into your Rivian Wallet™
Confusedcius 7y ago
The driver was found using his phone, over-relied on the autopilot system, and lied about what he was doing when being questioned. Sadly, we can't 100% trust these automated driving systems yet so folks need to stay attentive behind the wheel. The more accidents like this will cause lawmakers to create laws that can potentially slow down automated driving development.
- birdyrooster 7y ago
  If we want to be fair to consumers and bystanders, Tesla needs their product to safely stop the reckless behavior (not simply warn the user). If it can't do that, it needs to disable AutoPilot. AutoPilot was marketed in a way which led to Tesla selling their product to people with unwittingly dangerous uses in mind. It is unacceptable for Tesla to enable this reckless driving. Tesla has the ability to affect the scale of this problem most readily and relying on individual consumers is not in the best interests of public health.
  - gpm 7y ago
    Why can't I say literally the exact same thing about all cars? Let's hold people responsible for their actions, not objects. It's one thing to say "autopilot makes it hard to focus", it's another thing to say "autopilot made me pick up the phone while driving, and lie about it".
    - birdyrooster 7y ago
      Tesla's Auto Pilot website makes claims that other auto manufacturers don't make like: "With the new Tesla Vision cameras, sensors and computing power, your Tesla will navigate tighter, more complex roads." There is a lot implied in the short phrasing of "navigate tighter, more complex roads". It also begs the question that Tesla's capabilities were sufficient in the first place. "All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat." Why make potentially misleading and untested future-looking statements when consumer safety should be the focus of a system like this? https://www.tesla.com/autopilot https://www.tesla.com/autopilot
  - zaroth 7y ago
    This might sound like a good idea in principle — in effect, don’t allow people to use the system unless they are more attentive then even most drivers are normally. The problem is the natural outcome is that people will then simply just stop using the system, and end up less safe because of it, causing more accidents and possibly costing lives. In the interest of public health, the system has to be better than the average driver, and also not annoyingly unusable. So there is a balancing act. Tesla will already disable AP if you keep your hands off the wheel for too long. It even locks you out for the rest of the drive as a “penalty box” so you can’t just turn it back on. As I drive down the highway on AP, the number of cars drifting in their lanes, crossing the lines, changing lanes without signaling, not maintaining proper speed, driving aggressively, driving for an exit at the last minute, panic breaking, etc. is actually somewhat mind blowing. It’s amazing there aren’t more accidents. Versus the Model 3 which maintains speed and centers in its lane more precisely than I even do manually, for mile after mile. What’s also remarkable is the constant progression of new features, better feature detection, more complex navigation and route planning which has happened over just the last 12 months. Tesla AP now in 2019 is significantly improved from what it looked like even July 2018, and v10 will just continue the progression. Tesla safety stats are hard to compare to the general public, but what I can say definitely is a Tesla on AP on the highway is way more consistent and predictable of a driver than any human. It’s better to be in the Tesla in AP on the highway, but I imagine it’s also better to be driving behind or beside the Tesla in that same situation because it’s not driving distracted like almost everyone else on the road.
- c3534l 7y ago
  The best way to stay attentive behind the wheel is to actually be the one driving. There's virtually no way of being both attentive and passive for long periods of time. If a driver can't check their email or whatever on their phone while autopilot is on, then autopilot is not safe to put in cars. And while I'm okay with not-exactly-safe for most things people willingly consume or use, driving is not an area where it's okay to roll out a feature that may cause people to stop paying attention to the road.
  - diminoten 7y ago
    It is certainly okay when the rates at which these things happen is lower than the rates at which non-self driving cars have accidents. A 0% accident rate is unrealistic and unfair.
    - macintux 7y ago
      Statistics makes for a tough sell to the survivors. “Yes, this autonomous vehicle killed your son who just happened to be in the wrong place at the wrong time, but someone else’s son would have died in an unrelated accident somewhere else were it not for autonomy.”
      - kemitche 7y ago
        Assume for a moment that vaccines could actually cause autism at a small rate. Your argument applies there. "Yes your son became autistic, but someone else's son somewhere else didn't get measles and die." I understand there are human emotions involved, but my opinion is that significantly reducing death count is still a better outcome. If we can't accept that noticeably better systems will still have faults, and that those faults will result in a different, but smaller number of deaths, then we're content asking for more deaths than necessary.
        ipython 7y ago
        You don’t need to assume. With any medical procedure, there is risk involved. Including routine vaccination. That’s why there is a dedicated program to both report and compensate people who suffer adverse reactions from vaccines. See https://www.hrsa.gov/vaccine-compensation/index.html https://www.hrsa.gov/vaccine-compensation/index.html
- vkou 7y ago
  What is the point of a self-driving system where you need to stay attentive behind the wheel? The physical act of turning a wheel isn't the hard part of driving. Paying attention is.
  - danhak 7y ago
    > The physical act of turning a wheel isn't the hard part of driving. Paying attention is. As an avid user of Tesla Autopilot I'll disagree with this. On a long road trip, monitoring my surroundings and making sure everything is generally ok is far less fatiguing than making constant microadjustments to steering, speed and following distance. Even while paying close attention to the ride, long trips (especially in stop-and-go traffic) become significantly less stressful and exhausting.
    - vkou 7y ago
      > is far less fatiguing than making constant microadjustments to steering, speed and following distance. Adaptive cruise control and lane assist, available in every modern car under the sun will make microadjustments to steering, speed, and following distance all day long. What exactly does autopilot bring to the table?
      - macintux 7y ago
        I also wonder whether “dumber” systems are easier to predict. I’d feel much more comfortable knowing what the car won’t do (change lanes on me, e.g.). (Update: to be clear, I’m not inherently opposed to a car changing lanes if I need it to, but it seems easier to deal with a potential emergency, and easier to persuade myself to pay more attention, if I know definitively it won’t do something that sophisticated.)
      - kinghajj 7y ago
        That's all Autopilot is--Tesla's branding for adaptive cruise and lane keeping. It's just as useful as competitor's versions.
    - olliej 7y ago
- olliej 7y ago
  If you create a system that by design will cause the driver to lose focus, you are responsible for the driver losing focus. Saying “we told the users that they must pay attention while they are not driving”, or “we told them that the system called ‘auto-pilot’ is not actually able to autopilot the car”, and “we advertised the car’s self driving ability but had documentation saying it doesn’t actually have that ability” then you are responsible for that system driver behaving exactly as you would predict. Their have been numerous studies that show that people stop paying attention when they are given menial or non-focus dependent tasks. Saying that the only safe way to use autopilot is by not being a human is fundamentally the same as saying “the brakes in the car only work if you press them continuously for 5s or apply 300lbs of force”.
  - JeffL 7y ago
    My experience with autopilot isn't that I lose focus as much as I focus on something else. Instead of focusing on keeping in my lane, not going too fast, and if the car ahead of me is braking, I'm focusing on the road ahead -- is there construction coming up? Is there debris in the road? Is there a wall ahead that gets too close to the road? Is there anything weird up ahead that needs attention? You learn what AP can and can't handle and you keep an eye out for anything out of the ordinary, which on long road trips is significantly less fatiguing and probably safer.
    - munmaek 7y ago
      > as much as I focus on something else. Yes, so people ending up focusing on things like cellphones and email.
      - JeffL 7y ago
        There's obviously going to be some of that from people in every sort of car. The hope is that over all, people with AP are safer. I personally feel like I am.
- Traster 7y ago
  >Sadly, we can't 100% trust these automated driving systems yet so folks need to stay attentive behind the wheel. And companies selling the systems need to stop calling them "Autopilot". >The more accidents like this will cause lawmakers to create laws that can potentially slow down automated driving development. You'll actually hear this from people working on self-driving at most companies. That they have deliberately been conservative about their roll out and claims because they don't want to destroy the concept of self-driving before they can achieve it. Tesla has done a great job of damaging the reputation of self-driving cars for everyone.
  - nocitrek 7y ago
    Realisticly, Tesla has done more for self-driving cars than any company in the world. It is almost synonym with self-driving car. Mistakes are inevitable no matter what you do, so I would be careful with your last statement.
hnburnsy 7y ago
Report... Highway Accident Brief: Rear-End Collision Between a Car Operating with Advanced Driver Assistance Systems and a Stationary Fire Truck https://www.ntsb.gov/investigations/AccidentReports/Pages/HAB1907.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HA...
- c22 7y ago
  Thanks for the link. Interestingly, the NTSB's page appears unable to scroll at all with javascript disabled, but I was still able to view all the content by resizing the viewport(?) in both Firefox and Chrome. A very odd design decision...
dreamcompiler 7y ago
Fire truck driver here: People crash into parked fire trucks and ambulances and cop cars all the damn time. That's why we park the Big Red Truck at an angle behind the accident -- so that when the 2-ton car crashes into the 30-ton truck, the car will bounce off and the people behind the truck (us) won't be injured. I don't disagree that Tesla's software is partly to blame here, but the null experiment also has a lousy track record.
- smt88 7y ago
  The null experiment is the human driver of that particular Tesla, not an average of all drivers
  - schiffern 7y ago
    So if the crash didn't involve "the human driver of that particular Tesla" but instead was an identical incident with a different Tesla driver, we wouldn't be having this conversation? Because that seems highly improbable. This particular driver isn't what makes the story special or newsworthy. The nature of the incident is. Our choice of statistical cohort should reflect this data selection effect, ie "all Autopilot miles driven," not "all miles driven by <person's name>."
    - smt88 7y ago
      I disagree because we're comparing what this individual driver would have done without a feature advertised as Autopilot. I'm a partner in an auto insurance company, and most of our no-accident records will stay that way, statistically speaking. If this driver had a one in a billion miles chance of crashing and Autopilot (with an inattentve driver, as they all tend to be) had a one in a million miles chance, then Autopilot is decreasing safety. At a certain level of safety, Autopilot would make everyone safer, but it's definitely not there yet.
      - chingjun 7y ago
        How do you know that this individual driver has a one in a billion miles chance of crashing? By statistics from other human drivers I assume?
Animats 7y ago
Figure 7 depicts the movement of the Tesla and the two lead vehicles in the last 15 seconds before the crash. In the left panel—covering 15 to 8 seconds before impact—the Tesla is accelerating from 9 to 18 mph while following a lead vehicle (red car) at 30 to 46 feet.In the middle panel—covering 7 to 4 seconds before impact—the Tesla is traveling at a constant 21 mph while following a lead vehicle (green car) at a distance decreasing from 148 to 108 feet. In the right panel—covering the last 3 seconds before impact—the Tesla is accelerating from 21 to 31mph, with no lead vehicle and a forward collision warning half a second before impact. ... About 0.49 second (490 milliseconds) before the crash, the system detected a stationary object in the Tesla’s path. The forward collision warning activated, displaying a visual warning and sounding an auditory warning to the driver. By the moment of impact, the Tesla had accelerated to a speed of 30.9 mph. ... AEB did not activate during the event, and data show no driver-applied braking or steering before the crash. Tesla’s AEB is a radar/camera fusion system that is designed for front-to-rear collision mitigation or avoidance. According to the company, the system requires agreement from both the radar and the camera to initiate AEB; complex or unusual vehicle shapes can delay or prevent the system from classifying vehicles as targets or threats. [1] Notice the last line. The system still has to recognize an obstacle as being visually car-like before braking to avoid hitting it. If it's not recognized as a vehicle, the car will not stop. This was in January 2018, but it was a 2014 vehicle, which would mean the old Mobileye vision based collision system, right? We know Mobileye works that way, trying to draw boxes around car rear ends, because people have bought them and made videos showing what they see. Did that get fixed? Can it be fixed with a vision-based system? This is the big weakness of a no-LIDAR system. Here, the driver had 3 seconds to react, because the car ahead turned out of the lane to evade the fire truck well before reaching it. Compare this 2017 video of a Tesla hitting a construction barrier. The car ahead didn't turn until just before the barrier, only giving the driver under 1 second to react.[2] Same failure to detect a lane obstruction, but not enough time for the driver to do anything about it. [1] https://www.ntsb.gov/investigations/AccidentReports/Reports/HAB1907.pdf https://www.ntsb.gov/investigations/AccidentReports/Reports/... [2] https://www.youtube.com/watch?v=VTdcWnGnnJQ https://www.youtube.com/watch?v=VTdcWnGnnJQ
- matt-attack 7y ago
  It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding. What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how critical all of our human understanding is to driving.
jakeogh 7y ago
This shifting blame to the cloud is a bad trend. It's the driver's fault.
- olliej 7y ago
  ... but the driver was the car...
  - jakeogh 7y ago
    heh. You know it's a pre-programmed computer on wheels. It has no self.
  - sjwright 7y ago
    Not legally. Has Autopilot got a driver’s license? The fact that software was in control at the time is no more relevant than the guy who asks his twelve year old son in the passenger seat to reach over and steer while the driver rifles through stuff on the back seat. Responsibility for an accident doesn’t ever rest with whatever entity is steering the car, it rests with the licensed driver. (Which could be software, when it gets good enough.) Anything else is madness.
    - wbl 7y ago
      This report is about determining what features are inherently dangerous. It is very relevant that autopilot can't detect vehicles.
      - jakeogh 7y ago
        A common marketing trick is to steal a word by naming your product it. Suddenly you have Autopilot when it's really "Autopilot". Or better "Tesla Autopilot".
        sjwright 7y ago
        Yes, a respectable company like Chrysler wouldn't use such an inaccurate term. https://i2.wp.com/www.curbsideclassic.com/wp-content/uploads/2018/01/Chrysler-1958-Auto-Pilot-Brochure-03.jpg https://i2.wp.com/www.curbsideclassic.com/wp-content/uploads...
kazinator 7y ago
> The vehicle’s design “permitted the driver to disengage from the driving task” Which vehicle's design does not permit a driver to disengage from the driving task?
- draw_down 7y ago
  You guys are funny, I'll give you that!
- olliej 7y ago
  Autopilot. By design it encourages drivers to stop paying any attention - the fact that it has a “signal the driver” to try and maintain driver focus indicates that the core design produces this behavior. If the core design produces the same behavior, in a large enough portion of the user base to require such a feature, that’s a fairly good indication that the core design is causing a problem.
  - kazinator 7y ago
    Every car has a measure of "autopilot" called "wheel alignment". The car was built before Alta Vista was eaten by Google, yet goes dead straight on the freeway while you rummage through the glove compartment for your second favorite CD. Maybe cars should have wacky alignment that pulls randomly left and right; that would teach drivers to keep their hands on the wheel and pay attention at all times.
  - chrisseaton 7y ago
    So are you saying Autopilot does or doesn’t allow you to disengage? Because you said it didn’t then argued it did...
  - m463 7y ago
    You are answering the opposite of the question. all cars allow you to stop paying attention. I think the idea behind autopilot is that it reduces your workload (like cruise control, or automatic wipers, or automatic headlights or any other labor-saving device)
alexanderh 7y ago
Even with Autopilot, everyone I know who owns one: Its their little baby... they don't even dare risk it, at all. These people doing stuff like this are either really stupid, or really rich... or both. Because I have never seen anyone I know dare drive recklessly with autopilot.
- ModsCtrlideas 7y ago
  I was looking to get a leaf or Cadillac in a few years when their autopilot is on used vehicles. Isn't the point that its... Autopilot?
StopWithThe405 7y ago
Saying "the <highway/interstate number>" has got to be the most disgusting thing to come out of CA aside from CA residents themselves.
torpfactory 7y ago
I think we should look at this probabilistically. Yes, autopilot should be improved to prevent this, but no system is perfect. Certainly not human drivers. IMO the correct comparison is whether Teslas are on average more likely to hit firetrucks while on autopilot than normal vehicles while being driven by humans. I'm sure this data most likely won't be forthcoming, but I think it's important to keep in mind that we should be looking at accidents fleet-wide and not in isolation. Autonomously driven cars will have accidents. More importantly, their failure modes will be different than human drivers. However, as soon as they are even a small margin safer than human drivers, their adoption would save lives. Besides, the systems will continue to get better. Human drivers have proven stubbornly difficult to improve. We ought not reject autonomous driving technology only because it sometimes gets in accidents that appear to be strange or arbitrary to us. I consider the carnage perpetrated by human drivers to be arbitrary, but most people just take death on the roads as a fact of life.
KT-222 7y ago
Did the sun play a role in this? This picture from the report shows a low sun directly ahead. (Southbound 405 is going SSE in Culver City. 8:40AM accident) https://imgur.com/a/OaHNqdQ https://imgur.com/a/OaHNqdQ Assuming it was taken ~1 hour after accident, the sun would have been lower and to the left at time of the accident. The slightly askew firetruck might have been basically facing the sun, and its prominent rear features would have been in as much as shadow as possible. Obviously there were other factors involved (yes, ultimately driver's fault), but do Tesla's cameras have the dynamic range to be looking directly at a low sun and also see details in shadows? Originally I thought this was a high-speed accident and am surprised to learn it happened between 20-30 MPH.
aduitsis 7y ago
NTSB Report here: https://ntsb.gov/investigations/AccidentReports/Reports/HAB1907.pdf https://ntsb.gov/investigations/AccidentReports/Reports/HAB1... (found it here: https://www.theverge.com/2019/9/4/20849499/tesla-autopilot-crash-culver-city-2018-ntsb-report https://www.theverge.com/2019/9/4/20849499/tesla-autopilot-c...) Copying: When the last lead vehicle changed lanes—3 to 4 seconds before the crash—revealing the fire truck on the path of the Tesla, the system was unable to immediately detect the hazard and accelerated the Tesla toward the stationary truck. By the time the system detected the stationary vehicle and gave the driver a collision warning—0.49 second before impact —the collision was imminent and the warning was too late, particularly for an inattentive driver. The AEB system did not activate. Had the driver been attending to the driving task, he could have taken evasive action to avoid or mitigate the collision. So there you have it: "particularly for an inattentive driver". Not really an expert in the field, but I think a major problem with self-driving in general is handing over from an autopilot to a human. This is something that cannot be easily solved just by throwing smarter technology at the problem. Even if autopilot technology manages to be reliable 99.9% of the time, this won't be good enough, because people will invariably think they can rely on it 100% of the time and fall asleep or get distracted. So this 0.1% can be potentially catastrophic, because you cannot go from being asleep to full alertness in 0.5 seconds. Besides, even if one actually wants to maintain alertness, it may be harder to be alert and not doing anything than being alert and doing something (driving). So, unless autopilots can really really handle anything thrown at them, this will be a real issue. Although there is much willingness for people to convince others and be convinced that this kind of autopilot is very close to being a reality, I suspect it may be not.
- olau 7y ago
  There was an interview with George Hotz from comma.ai where he talked about this. His fix is a very attentive driver monitoring setup. So if you are not paying attention, it will barf at you. IIRC he was also talking about how this driver monitoring setup is worth something in itself for trucking companies, fewer damages.
quotemstr 7y ago
I worry much more about public overreaction to rare incidents slowing and overregulating self driving technology than I do about the rare crash here and there. Overall statistical safety is what matters, and statistics goes out the window when the public emotionally fixates on specific mishaps and demands regulation that slows and cripples overall-safe technology. Everyone believes, without reason, that he is a good driver and will always beat those janky autopilots. HN threads about autonomous vehicles are always full of unrepresentative anecdotes and angry moral denunciations of technology. It's sad.
- tim333 7y ago
  Overall statistical safety just now for Tesla autopilot is probably worse than plain human driving. You can reasonably argue though that allowing research now will save lives in the future.
joe_momma 7y ago
Hey Elon, why don't you build a car course and fill it with all the things your cars have failed to avoid. Why haven't you made some OTA for large obstacle non car thing?
jmpman 7y ago
If Tesla is unable to brake for a stationary fire truck with 3+ seconds of visibility, why should I be confident to pay $6k for “Full Self Driving”? Recognizing a stopped fire truck (effectively a wall) in front of me is a prerequisite for me to invest further. A part of me believes that the public is being intentionally misled on the capabilities, and Tesla’s lack of transparency, coupled with their financial incentive to obscure, makes me more cynical. What Tesla needs to do is to classify every known accident, and then recreate the accident (video please) followed by a recreation with their fixes. If the scenario can’t be prevented in software, it needs to be disclosed to the buyer. Would Tesla crash into that same fire truck again with the newest software? I suspect yes. I even suspect there are a bunch of engineers sitting around Tesla confident that crashing into that fire truck was the right thing to do. Heck, I think Tesla should be required to have an asterisk next to every self driving claim, with an asterisk for every known scenario where their product has failed, including links to the crash reports, including full video evidence. The public should have a right to know exactly where/when/why the system failed as long as those failures aren’t prevented in software from happening again.
- WhompingWindows 7y ago
  Why should anyone buy a feature for $6k that basically gets them nothing until a future point when they could buy it anyways?
  - grecy 7y ago
    It will be more expensive later.
    - klwejchkwejrhc 7y ago
      The thing is, it almost certainly still won't exist later.
- klwejchkwejrhc 7y ago
  >why should I be confident to pay $6k for “Full Self Driving” No one should be. Creating a fully self-driving car probably requires creating artificial general intelligence. It will almost certainly not exist for many more decades, if it ever exists. Elon Musk is a con artist.
navigatesol 7y ago
The Tesla damage control astro-turfing is out in full force. Between the surveillance state they've built and their complete disregard for safety, Silicon Valley will eventually be forced to atone for this from the public at large. Make sure you're on the right side of history.
- tim333 7y ago
  Historically the right side has generally been technology making peoples lives better. At least in material terms such as life expectancy, diseases cured and so on. They still moan though.

152 points by casefields 7y ago

$0.75 has been deposited into your Rivian Wallet™

The driver was found using his phone, over-relied on the autopilot system, and lied about what he was doing when being questioned. Sadly, we can't 100% trust these automated driving systems yet so folks need to stay attentive behind the wheel. The more accidents like this will cause lawmakers to create laws that can potentially slow down automated driving development.

birdyrooster 7y ago
If we want to be fair to consumers and bystanders, Tesla needs their product to safely stop the reckless behavior (not simply warn the user). If it can't do that, it needs to disable AutoPilot. AutoPilot was marketed in a way which led to Tesla selling their product to people with unwittingly dangerous uses in mind. It is unacceptable for Tesla to enable this reckless driving. Tesla has the ability to affect the scale of this problem most readily and relying on individual consumers is not in the best interests of public health.
- gpm 7y ago
  Why can't I say literally the exact same thing about all cars? Let's hold people responsible for their actions, not objects. It's one thing to say "autopilot makes it hard to focus", it's another thing to say "autopilot made me pick up the phone while driving, and lie about it".
  - birdyrooster 7y ago
    Tesla's Auto Pilot website makes claims that other auto manufacturers don't make like: "With the new Tesla Vision cameras, sensors and computing power, your Tesla will navigate tighter, more complex roads." There is a lot implied in the short phrasing of "navigate tighter, more complex roads". It also begs the question that Tesla's capabilities were sufficient in the first place. "All new Tesla cars have the hardware needed in the future for full self-driving in almost all circumstances. The system is designed to be able to conduct short and long distance trips with no action required by the person in the driver’s seat." Why make potentially misleading and untested future-looking statements when consumer safety should be the focus of a system like this? https://www.tesla.com/autopilot https://www.tesla.com/autopilot
- zaroth 7y ago
  This might sound like a good idea in principle — in effect, don’t allow people to use the system unless they are more attentive then even most drivers are normally. The problem is the natural outcome is that people will then simply just stop using the system, and end up less safe because of it, causing more accidents and possibly costing lives. In the interest of public health, the system has to be better than the average driver, and also not annoyingly unusable. So there is a balancing act. Tesla will already disable AP if you keep your hands off the wheel for too long. It even locks you out for the rest of the drive as a “penalty box” so you can’t just turn it back on. As I drive down the highway on AP, the number of cars drifting in their lanes, crossing the lines, changing lanes without signaling, not maintaining proper speed, driving aggressively, driving for an exit at the last minute, panic breaking, etc. is actually somewhat mind blowing. It’s amazing there aren’t more accidents. Versus the Model 3 which maintains speed and centers in its lane more precisely than I even do manually, for mile after mile. What’s also remarkable is the constant progression of new features, better feature detection, more complex navigation and route planning which has happened over just the last 12 months. Tesla AP now in 2019 is significantly improved from what it looked like even July 2018, and v10 will just continue the progression. Tesla safety stats are hard to compare to the general public, but what I can say definitely is a Tesla on AP on the highway is way more consistent and predictable of a driver than any human. It’s better to be in the Tesla in AP on the highway, but I imagine it’s also better to be driving behind or beside the Tesla in that same situation because it’s not driving distracted like almost everyone else on the road.
c3534l 7y ago
The best way to stay attentive behind the wheel is to actually be the one driving. There's virtually no way of being both attentive and passive for long periods of time. If a driver can't check their email or whatever on their phone while autopilot is on, then autopilot is not safe to put in cars. And while I'm okay with not-exactly-safe for most things people willingly consume or use, driving is not an area where it's okay to roll out a feature that may cause people to stop paying attention to the road.
- diminoten 7y ago
  It is certainly okay when the rates at which these things happen is lower than the rates at which non-self driving cars have accidents. A 0% accident rate is unrealistic and unfair.
  - macintux 7y ago
    Statistics makes for a tough sell to the survivors. “Yes, this autonomous vehicle killed your son who just happened to be in the wrong place at the wrong time, but someone else’s son would have died in an unrelated accident somewhere else were it not for autonomy.”
    - kemitche 7y ago
      Assume for a moment that vaccines could actually cause autism at a small rate. Your argument applies there. "Yes your son became autistic, but someone else's son somewhere else didn't get measles and die." I understand there are human emotions involved, but my opinion is that significantly reducing death count is still a better outcome. If we can't accept that noticeably better systems will still have faults, and that those faults will result in a different, but smaller number of deaths, then we're content asking for more deaths than necessary.
      - ipython 7y ago
        You don’t need to assume. With any medical procedure, there is risk involved. Including routine vaccination. That’s why there is a dedicated program to both report and compensate people who suffer adverse reactions from vaccines. See https://www.hrsa.gov/vaccine-compensation/index.html https://www.hrsa.gov/vaccine-compensation/index.html
vkou 7y ago
What is the point of a self-driving system where you need to stay attentive behind the wheel? The physical act of turning a wheel isn't the hard part of driving. Paying attention is.
- danhak 7y ago
  > The physical act of turning a wheel isn't the hard part of driving. Paying attention is. As an avid user of Tesla Autopilot I'll disagree with this. On a long road trip, monitoring my surroundings and making sure everything is generally ok is far less fatiguing than making constant microadjustments to steering, speed and following distance. Even while paying close attention to the ride, long trips (especially in stop-and-go traffic) become significantly less stressful and exhausting.
  - vkou 7y ago
    > is far less fatiguing than making constant microadjustments to steering, speed and following distance. Adaptive cruise control and lane assist, available in every modern car under the sun will make microadjustments to steering, speed, and following distance all day long. What exactly does autopilot bring to the table?
    - macintux 7y ago
      I also wonder whether “dumber” systems are easier to predict. I’d feel much more comfortable knowing what the car won’t do (change lanes on me, e.g.). (Update: to be clear, I’m not inherently opposed to a car changing lanes if I need it to, but it seems easier to deal with a potential emergency, and easier to persuade myself to pay more attention, if I know definitively it won’t do something that sophisticated.)
    - kinghajj 7y ago
      That's all Autopilot is--Tesla's branding for adaptive cruise and lane keeping. It's just as useful as competitor's versions.
  - olliej 7y ago
olliej 7y ago
If you create a system that by design will cause the driver to lose focus, you are responsible for the driver losing focus. Saying “we told the users that they must pay attention while they are not driving”, or “we told them that the system called ‘auto-pilot’ is not actually able to autopilot the car”, and “we advertised the car’s self driving ability but had documentation saying it doesn’t actually have that ability” then you are responsible for that system driver behaving exactly as you would predict. Their have been numerous studies that show that people stop paying attention when they are given menial or non-focus dependent tasks. Saying that the only safe way to use autopilot is by not being a human is fundamentally the same as saying “the brakes in the car only work if you press them continuously for 5s or apply 300lbs of force”.
- JeffL 7y ago
  My experience with autopilot isn't that I lose focus as much as I focus on something else. Instead of focusing on keeping in my lane, not going too fast, and if the car ahead of me is braking, I'm focusing on the road ahead -- is there construction coming up? Is there debris in the road? Is there a wall ahead that gets too close to the road? Is there anything weird up ahead that needs attention? You learn what AP can and can't handle and you keep an eye out for anything out of the ordinary, which on long road trips is significantly less fatiguing and probably safer.
  - munmaek 7y ago
    > as much as I focus on something else. Yes, so people ending up focusing on things like cellphones and email.
    - JeffL 7y ago
      There's obviously going to be some of that from people in every sort of car. The hope is that over all, people with AP are safer. I personally feel like I am.
Traster 7y ago
>Sadly, we can't 100% trust these automated driving systems yet so folks need to stay attentive behind the wheel. And companies selling the systems need to stop calling them "Autopilot". >The more accidents like this will cause lawmakers to create laws that can potentially slow down automated driving development. You'll actually hear this from people working on self-driving at most companies. That they have deliberately been conservative about their roll out and claims because they don't want to destroy the concept of self-driving before they can achieve it. Tesla has done a great job of damaging the reputation of self-driving cars for everyone.
- nocitrek 7y ago
  Realisticly, Tesla has done more for self-driving cars than any company in the world. It is almost synonym with self-driving car. Mistakes are inevitable no matter what you do, so I would be careful with your last statement.

hnburnsy 7y ago

Report... Highway Accident Brief: Rear-End Collision Between a Car Operating with Advanced Driver Assistance Systems and a Stationary Fire Truck https://www.ntsb.gov/investigations/AccidentReports/Pages/HAB1907.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HA...

c22 7y ago
Thanks for the link. Interestingly, the NTSB's page appears unable to scroll at all with javascript disabled, but I was still able to view all the content by resizing the viewport(?) in both Firefox and Chrome. A very odd design decision...

dreamcompiler 7y ago

Fire truck driver here: People crash into parked fire trucks and ambulances and cop cars all the damn time. That's why we park the Big Red Truck at an angle behind the accident -- so that when the 2-ton car crashes into the 30-ton truck, the car will bounce off and the people behind the truck (us) won't be injured. I don't disagree that Tesla's software is partly to blame here, but the null experiment also has a lousy track record.

smt88 7y ago
The null experiment is the human driver of that particular Tesla, not an average of all drivers
- schiffern 7y ago
  So if the crash didn't involve "the human driver of that particular Tesla" but instead was an identical incident with a different Tesla driver, we wouldn't be having this conversation? Because that seems highly improbable. This particular driver isn't what makes the story special or newsworthy. The nature of the incident is. Our choice of statistical cohort should reflect this data selection effect, ie "all Autopilot miles driven," not "all miles driven by <person's name>."
  - smt88 7y ago
    I disagree because we're comparing what this individual driver would have done without a feature advertised as Autopilot. I'm a partner in an auto insurance company, and most of our no-accident records will stay that way, statistically speaking. If this driver had a one in a billion miles chance of crashing and Autopilot (with an inattentve driver, as they all tend to be) had a one in a million miles chance, then Autopilot is decreasing safety. At a certain level of safety, Autopilot would make everyone safer, but it's definitely not there yet.
    - chingjun 7y ago
      How do you know that this individual driver has a one in a billion miles chance of crashing? By statistics from other human drivers I assume?

Animats 7y ago

Figure 7 depicts the movement of the Tesla and the two lead vehicles in the last 15 seconds before the crash. In the left panel—covering 15 to 8 seconds before impact—the Tesla is accelerating from 9 to 18 mph while following a lead vehicle (red car) at 30 to 46 feet.In the middle panel—covering 7 to 4 seconds before impact—the Tesla is traveling at a constant 21 mph while following a lead vehicle (green car) at a distance decreasing from 148 to 108 feet. In the right panel—covering the last 3 seconds before impact—the Tesla is accelerating from 21 to 31mph, with no lead vehicle and a forward collision warning half a second before impact. ... About 0.49 second (490 milliseconds) before the crash, the system detected a stationary object in the Tesla’s path. The forward collision warning activated, displaying a visual warning and sounding an auditory warning to the driver. By the moment of impact, the Tesla had accelerated to a speed of 30.9 mph. ... AEB did not activate during the event, and data show no driver-applied braking or steering before the crash. Tesla’s AEB is a radar/camera fusion system that is designed for front-to-rear collision mitigation or avoidance. According to the company, the system requires agreement from both the radar and the camera to initiate AEB; complex or unusual vehicle shapes can delay or prevent the system from classifying vehicles as targets or threats. [1] Notice the last line. The system still has to recognize an obstacle as being visually car-like before braking to avoid hitting it. If it's not recognized as a vehicle, the car will not stop. This was in January 2018, but it was a 2014 vehicle, which would mean the old Mobileye vision based collision system, right? We know Mobileye works that way, trying to draw boxes around car rear ends, because people have bought them and made videos showing what they see. Did that get fixed? Can it be fixed with a vision-based system? This is the big weakness of a no-LIDAR system. Here, the driver had 3 seconds to react, because the car ahead turned out of the lane to evade the fire truck well before reaching it. Compare this 2017 video of a Tesla hitting a construction barrier. The car ahead didn't turn until just before the barrier, only giving the driver under 1 second to react.[2] Same failure to detect a lane obstruction, but not enough time for the driver to do anything about it. [1] https://www.ntsb.gov/investigations/AccidentReports/Reports/HAB1907.pdf https://www.ntsb.gov/investigations/AccidentReports/Reports/... [2] https://www.youtube.com/watch?v=VTdcWnGnnJQ https://www.youtube.com/watch?v=VTdcWnGnnJQ

matt-attack 7y ago
It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding. What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how critical all of our human understanding is to driving.

jakeogh 7y ago

This shifting blame to the cloud is a bad trend. It's the driver's fault.

olliej 7y ago
... but the driver was the car...
- jakeogh 7y ago
  heh. You know it's a pre-programmed computer on wheels. It has no self.
- sjwright 7y ago
  Not legally. Has Autopilot got a driver’s license? The fact that software was in control at the time is no more relevant than the guy who asks his twelve year old son in the passenger seat to reach over and steer while the driver rifles through stuff on the back seat. Responsibility for an accident doesn’t ever rest with whatever entity is steering the car, it rests with the licensed driver. (Which could be software, when it gets good enough.) Anything else is madness.
  - wbl 7y ago
    This report is about determining what features are inherently dangerous. It is very relevant that autopilot can't detect vehicles.
    - jakeogh 7y ago
      A common marketing trick is to steal a word by naming your product it. Suddenly you have Autopilot when it's really "Autopilot". Or better "Tesla Autopilot".
      - sjwright 7y ago
        Yes, a respectable company like Chrysler wouldn't use such an inaccurate term. https://i2.wp.com/www.curbsideclassic.com/wp-content/uploads/2018/01/Chrysler-1958-Auto-Pilot-Brochure-03.jpg https://i2.wp.com/www.curbsideclassic.com/wp-content/uploads...

kazinator 7y ago

> The vehicle’s design “permitted the driver to disengage from the driving task” Which vehicle's design does not permit a driver to disengage from the driving task?

draw_down 7y ago
You guys are funny, I'll give you that!
olliej 7y ago
Autopilot. By design it encourages drivers to stop paying any attention - the fact that it has a “signal the driver” to try and maintain driver focus indicates that the core design produces this behavior. If the core design produces the same behavior, in a large enough portion of the user base to require such a feature, that’s a fairly good indication that the core design is causing a problem.
- kazinator 7y ago
  Every car has a measure of "autopilot" called "wheel alignment". The car was built before Alta Vista was eaten by Google, yet goes dead straight on the freeway while you rummage through the glove compartment for your second favorite CD. Maybe cars should have wacky alignment that pulls randomly left and right; that would teach drivers to keep their hands on the wheel and pay attention at all times.
- chrisseaton 7y ago
  So are you saying Autopilot does or doesn’t allow you to disengage? Because you said it didn’t then argued it did...
- m463 7y ago
  You are answering the opposite of the question. all cars allow you to stop paying attention. I think the idea behind autopilot is that it reduces your workload (like cruise control, or automatic wipers, or automatic headlights or any other labor-saving device)

alexanderh 7y ago

Even with Autopilot, everyone I know who owns one: Its their little baby... they don't even dare risk it, at all. These people doing stuff like this are either really stupid, or really rich... or both. Because I have never seen anyone I know dare drive recklessly with autopilot.

ModsCtrlideas 7y ago
I was looking to get a leaf or Cadillac in a few years when their autopilot is on used vehicles. Isn't the point that its... Autopilot?

StopWithThe405 7y ago

Saying "the <highway/interstate number>" has got to be the most disgusting thing to come out of CA aside from CA residents themselves.

torpfactory 7y ago

I think we should look at this probabilistically. Yes, autopilot should be improved to prevent this, but no system is perfect. Certainly not human drivers. IMO the correct comparison is whether Teslas are on average more likely to hit firetrucks while on autopilot than normal vehicles while being driven by humans. I'm sure this data most likely won't be forthcoming, but I think it's important to keep in mind that we should be looking at accidents fleet-wide and not in isolation. Autonomously driven cars will have accidents. More importantly, their failure modes will be different than human drivers. However, as soon as they are even a small margin safer than human drivers, their adoption would save lives. Besides, the systems will continue to get better. Human drivers have proven stubbornly difficult to improve. We ought not reject autonomous driving technology only because it sometimes gets in accidents that appear to be strange or arbitrary to us. I consider the carnage perpetrated by human drivers to be arbitrary, but most people just take death on the roads as a fact of life.

KT-222 7y ago

Did the sun play a role in this? This picture from the report shows a low sun directly ahead. (Southbound 405 is going SSE in Culver City. 8:40AM accident) https://imgur.com/a/OaHNqdQ https://imgur.com/a/OaHNqdQ Assuming it was taken ~1 hour after accident, the sun would have been lower and to the left at time of the accident. The slightly askew firetruck might have been basically facing the sun, and its prominent rear features would have been in as much as shadow as possible. Obviously there were other factors involved (yes, ultimately driver's fault), but do Tesla's cameras have the dynamic range to be looking directly at a low sun and also see details in shadows? Originally I thought this was a high-speed accident and am surprised to learn it happened between 20-30 MPH.

aduitsis 7y ago

NTSB Report here: https://ntsb.gov/investigations/AccidentReports/Reports/HAB1907.pdf https://ntsb.gov/investigations/AccidentReports/Reports/HAB1... (found it here: https://www.theverge.com/2019/9/4/20849499/tesla-autopilot-crash-culver-city-2018-ntsb-report https://www.theverge.com/2019/9/4/20849499/tesla-autopilot-c...) Copying: When the last lead vehicle changed lanes—3 to 4 seconds before the crash—revealing the fire truck on the path of the Tesla, the system was unable to immediately detect the hazard and accelerated the Tesla toward the stationary truck. By the time the system detected the stationary vehicle and gave the driver a collision warning—0.49 second before impact —the collision was imminent and the warning was too late, particularly for an inattentive driver. The AEB system did not activate. Had the driver been attending to the driving task, he could have taken evasive action to avoid or mitigate the collision. So there you have it: "particularly for an inattentive driver". Not really an expert in the field, but I think a major problem with self-driving in general is handing over from an autopilot to a human. This is something that cannot be easily solved just by throwing smarter technology at the problem. Even if autopilot technology manages to be reliable 99.9% of the time, this won't be good enough, because people will invariably think they can rely on it 100% of the time and fall asleep or get distracted. So this 0.1% can be potentially catastrophic, because you cannot go from being asleep to full alertness in 0.5 seconds. Besides, even if one actually wants to maintain alertness, it may be harder to be alert and not doing anything than being alert and doing something (driving). So, unless autopilots can really really handle anything thrown at them, this will be a real issue. Although there is much willingness for people to convince others and be convinced that this kind of autopilot is very close to being a reality, I suspect it may be not.

olau 7y ago
There was an interview with George Hotz from comma.ai where he talked about this. His fix is a very attentive driver monitoring setup. So if you are not paying attention, it will barf at you. IIRC he was also talking about how this driver monitoring setup is worth something in itself for trucking companies, fewer damages.

quotemstr 7y ago

I worry much more about public overreaction to rare incidents slowing and overregulating self driving technology than I do about the rare crash here and there. Overall statistical safety is what matters, and statistics goes out the window when the public emotionally fixates on specific mishaps and demands regulation that slows and cripples overall-safe technology. Everyone believes, without reason, that he is a good driver and will always beat those janky autopilots. HN threads about autonomous vehicles are always full of unrepresentative anecdotes and angry moral denunciations of technology. It's sad.

tim333 7y ago
Overall statistical safety just now for Tesla autopilot is probably worse than plain human driving. You can reasonably argue though that allowing research now will save lives in the future.

joe_momma 7y ago

Hey Elon, why don't you build a car course and fill it with all the things your cars have failed to avoid. Why haven't you made some OTA for large obstacle non car thing?

jmpman 7y ago

If Tesla is unable to brake for a stationary fire truck with 3+ seconds of visibility, why should I be confident to pay $6k for “Full Self Driving”? Recognizing a stopped fire truck (effectively a wall) in front of me is a prerequisite for me to invest further. A part of me believes that the public is being intentionally misled on the capabilities, and Tesla’s lack of transparency, coupled with their financial incentive to obscure, makes me more cynical. What Tesla needs to do is to classify every known accident, and then recreate the accident (video please) followed by a recreation with their fixes. If the scenario can’t be prevented in software, it needs to be disclosed to the buyer. Would Tesla crash into that same fire truck again with the newest software? I suspect yes. I even suspect there are a bunch of engineers sitting around Tesla confident that crashing into that fire truck was the right thing to do. Heck, I think Tesla should be required to have an asterisk next to every self driving claim, with an asterisk for every known scenario where their product has failed, including links to the crash reports, including full video evidence. The public should have a right to know exactly where/when/why the system failed as long as those failures aren’t prevented in software from happening again.

WhompingWindows 7y ago
Why should anyone buy a feature for $6k that basically gets them nothing until a future point when they could buy it anyways?
- grecy 7y ago
  It will be more expensive later.
  - klwejchkwejrhc 7y ago
    The thing is, it almost certainly still won't exist later.
klwejchkwejrhc 7y ago
>why should I be confident to pay $6k for “Full Self Driving” No one should be. Creating a fully self-driving car probably requires creating artificial general intelligence. It will almost certainly not exist for many more decades, if it ever exists. Elon Musk is a con artist.

navigatesol 7y ago

The Tesla damage control astro-turfing is out in full force. Between the surveillance state they've built and their complete disregard for safety, Silicon Valley will eventually be forced to atone for this from the public at large. Make sure you're on the right side of history.

tim333 7y ago
Historically the right side has generally been technology making peoples lives better. At least in material terms such as life expectancy, diseases cured and so on. They still moan though.