38 ms·

NHTSA’s full investigation into Tesla’s Autopilot shows 40% crash rate reduction

808 points by fmihaila 10y ago

snewman 10y ago
Tesla comes off extremely well in this report. For one thing, the 40% statistic cited in the headline appears to be well supported by the NHTSA report (section 5.4) and actually manages to frame the incident in a very positive light: ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to and after Autopilot installation. Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation. I had hoped to see more information about this specific incident. For instance, any data on whether the driver had his hands on the wheel, what steps the car had taken to prompt his attention, etc. But that doesn't seem to be included.
- Roritharr 10y ago
  I really would like to know the absolute numbers for this reduction. It's really questionable if this would hold up with millions of Teslas on the road in the hands of less wealthy(probably better educated) drivers.
  - dkonofalski 10y ago
    Wait... are you saying that people that are less wealthy are better educated? If so, I need to see some sources. That sounds preposterous.
    - jackmott 10y ago
      no. he needs more parens to make the order of operations clear. also, love the standard internet response to data. "the data in not perfect, or is incomplete, thus I will hold steadfast to my intuition" love you, internet.
      - GauntletWizard 10y ago
        It's very reasonable, when presented with data that doesn't pass a sniff check, to mentally bitbucket that data. Unfortunately, lots of people's noses are miscalibrated - And what's worse we're not sure which ones.
    - randomdata 10y ago
      Assuming by education you mean formal schooling: Consider that wealth is accumulated over time. While outliers certainly exist, the older you get, the more wealthy you are apt to be, at least to the point when you retire and start to use those reserves to live on. Also consider that in 1970s, the time when the boomers started coming of age, having 'high school or more' was reserved for just 50% of the population. Only a mere 25% for the oldest segment of the population. It has steadily climbed to more than 80% for the youngest adults. But they, statistically, haven't had time to amass a large amount of wealth yet.
    - avenoir 10y ago
      I also found that confusing. I think in this case "less wealthy" is within the context of Tesla owners, which is basically upper middle class(?), which sort of makes sense. At least that's how I read it.
    - Roritharr 10y ago
      No, i just took education as one possible factor that might be different between Tesla and Camry or Hummer drivers. Tesla Owners have a specific selection bias, its just hard to describe what that bias actually is, so i just took a stab in a direction to get the discussion going.
  - cryptoz 10y ago
    Probably as the technology scales up to millions of people using it, the numbers will actually be even better for safety. There is a network effect that compounds the safety factor - as more self-driving cars are on the road, there are fewer rogue human drivers. So with millions of Teslas driving around there will be that many fewer people causing accidents by human error.
  - Tepix 10y ago
    The rates dropped by 40%. The drivers remained the same. Why do you think a different driver demographic will not show a similar improvement?
    - wccrawford 10y ago
      Maybe other drivers are really good, or maybe this class of driver is really bad?
      - cryptoz 10y ago
        35,000 people die from car accidents every single year in the USA alone. Other drivers are not "really good".
        ebbv 10y ago
        Personally I believe most people are really terrible drivers. That being said, there were 210 million drivers in the US in 2009. There's probably a lot more today. That's just 0.01% of drivers. That fact makes me have to question just how bad people really are, despite my own personal opinion. Or maybe cars are just really safe. A better statistic might be number of accidents. In 2015 2.44 million people were injured in car accidents. That's a much more telling figure of just how bad most drivers are. Apparently our cars are just really safe.
        camiller 10y ago
        I had an econ professor assert that as cars got safer there were more accidents because people trusted the car to protect them more.... Logically, to reduce accidents due to carelessness we should put a 12" sharpened spike right in the middle of the steering wheel pointing at the drivers chest. If all accidents were deadly wouldn't drivers then exercise the utmost care?
        OrwellianChild 10y ago
        It's a great joke, but a poor strategy. Remember the end game: minimize harm. The universal solutions that remove human error have a better long-run potential to minimize harm, since they can exceed the effectiveness of even the most attentive and cautious human driver.
  - deleted 10y ago
    [deleted]
  - celticninja 10y ago
    Are you assuming the more wealthy owners have lower levels of education? Or have I misinterpreted your statement? I would have assumed tesla owners would fall into the more educated category. No idea which assumption is correct just wondering about your reasoning. For my assumption I assume that due to the higher cost of a Tesla better educated people are opting for them over the alternative because of the environmental benefits.
    - randomdata 10y ago
      I would have assumed due to the high cost of a Tesla that the owners would tend to be older, and thus a generation educated in much fewer numbers. Today's youth are much more educated, but they are still young, which hasn't given them much time to build wealth. I mean, who do you think is more likely to buy a Tesla: A 20 year old fresh college graduate, or his 'uneducated' father who has been saving and investing for 30 years and just sold his lifelong home in San Francisco? My bet is on the latter. The boomers in particular are considered the wealthiest generation ever, but not the most educated generation ever. I'm rather fascinated by how many are questioning the education thing here though. Coming from a farming community, where every (older) farmer I know is multimillions simply by virtue of having purchased farmland when they were young, it's difficult to see how education had any impact on that wealth. Some of them do have educations, some don't. It doesn't seem to make any difference.
      - celticninja 10y ago
        I evidently misunderstood the original comment, when OP said less wealthy (better educated) he meant that there would be fewer wealthy (and better educated) owners. I don't agree with the sentiment and think OP didn't really explain his position well either.
        randomdata 10y ago
        You don't agree that wealthy people tend to be older? Even though time is the easiest way to increase wealth (compound interest is your friend). Or you don't agree that older people are less educated? Even thought the data suggests otherwise.
  - atonalfreerider 10y ago
    Wealthy motorists drive more recklessly: http://www.pnas.org/content/109/11/4086.short http://www.pnas.org/content/109/11/4086.short
    - Roritharr 10y ago
      Makes sense, but its still interesting how the result would change when applied to the general population. Would VW Beetle drivers have the same reduction?
    - mozumder 10y ago
      I think you pointed to the wrong link. That's a study about ethics, not safety.
      - atonalfreerider 10y ago
        If you read the abstract you will see that the studies involve driver behavior. "Results Studies 1 and 2. Our first two studies were naturalistic field studies, and examined whether upper-class individuals behave more unethically than lower-class individuals while driving. In study 1, we investigated whether upper-class drivers were more likely to cut off other vehicles at a busy four-way intersection with stop signs on all sides. As vehicles are reliable indicators of a person's social rank and wealth (15), we used observers’ codes of vehicle status (make, age, and appearance) to index drivers’ social class. Observers stood near the intersection, coded the status of approaching vehicles, and recorded whether the driver cut off other vehicles by crossing the intersection before waiting their turn, a behavior that defies the California Vehicle Code. In the present study, 12.4% of drivers cut in front of other vehicles. A binary logistic regression indicated that upper-class drivers were the most likely to cut off other vehicles at the intersection, even when controlling for time of day, driver's perceived sex and age, and amount of traffic, b = 0.36, SE b = 0.18, P < 0.05"
        nitrogen 10y ago
        Did they account for bias in their observers?
  - onmobiletemp 10y ago
    They said it was a 40 percent drop after the roll out of auto-pilot features. That means it dropped 40 percent with the same driver base. Please read the article before posting comments.
  - Ar-Curunir 10y ago
    So "better" drivers saw 40% reduction; doesn't that indicate that worse drivers will see even greater reduction in accidents?
  - lamontcg 10y ago
    > wealthy (probably better educated) drivers. You mean drivers like the ones behind the wheel of all the BMWs and Audis that cut me off with no turn signals, speed, make aggressive lane changes, tailgate, and generally act like giant douches because a simple traffic ticket is nothing to their pocketbook while their time == money? Those people are going to be a hell of a lot safer behind the wheel of an autonomous (level 4) vehicle where they can be on the phone and their laptop as the vehicle obeys the speed limits and safe following distance.
    - deleted 10y ago
      [deleted]
    - celticninja 10y ago
      He said less wealthy, better educated drivers.
      - lamontcg 10y ago
        Pretty certain he meant it the way I quoted it.
        celticninja 10y ago
        Ah it now makes sense, I did ask him to clarify but I just parsed in my brain when reading your reply he meant fewer wealthy (better educated drivers) as opposed to less wealthy meaning not as wealthy. Thanks for your help.
        brokenmachine 10y ago
        I read it as less wealthy meaning not as wealthy as well. Funny how ambiguous some sentences can be.
        delecti 10y ago
        Well it's really that he used the wrong word. "Less wealthy drivers" can't correctly refer to the number of drivers.
- tehlike 10y ago
  I didnt read the report, but is this measurement done on a a/b testing way? There is usually some data skew/bias when they are done on different time scale.
  - vvanders 10y ago
    Same cars pre/post OTA. There was a ~6 month peroid where AP HW was out in the wild but not used.
    - pps43 10y ago
      Confounding factors. For example, crashes can be more likely after sunset or during inclement weather so seasonality can play a role. Or maybe the drivers who had Tesla longer got used to it.
      - celticninja 10y ago
        Could this be accounted for if the sample period was large enough, ie over 12 months to cover seasonality and a few months for time of day.
        tehlike 10y ago
        this particular one, maybe, but there might be others. like the user base pre-ota and post ota (number-wise) could be different. I think there is a good SaaS business with A/B testing robot/car software OTA. I would imagine some metrics would be: 1. car speed 2. battery consumption per mile. 3. user comfort metrics (if it's possible to measure, maybe like hearthrate, body temperature etc - i can imagine if robots were driving bad, it'd be visible in heart rate or something like that).4 The list probably goes on. It would be interesting to see how much battery consumption changed after OTA, for example.
        pps43 10y ago
        I suspect OTA update was applied simultaneously rather than staggered over the course of the year, so probably not. You can control for seasonality using some other data as a proxy, but the more adjustments you make, the more potential there is for introducing errors. Randomized test/control split is almost always better than before/after comparison.
- Vik1ng 10y ago
  Just funny that the 40% from Autosteer seem to exactly match with the general AEB safety improvement rate also mentioned in the report... IIHS research shows that AEB systems meeting the commitment would reduce rear-end crashes[emphasis added] by 40 percent. IIHS estimates that by 2025 – the earliest NHTSA believes it could realistically implement a regulatory requirement for AEB – the commitment will prevent 28,000 crashes and 12,000 injuries.
  - GrantS 10y ago
    Here is some of that IIHS research [1] from Jan 2016 that gives lots of raw crash numbers broken down by manufacturer, type of system (AEB/FCW), injuries involved, etc. Really informative. Some excerpts from "Effectiveness of Forward Collision Warning Systems with and without Autonomous Emergency Braking in Reducing Police-Reported Crash Rates", January 2016: "FCW alone and FCW with AEB reduced rear-end striking crash involvement rates by 23% and 39%, respectively. " "Among the 15,802 injury crash involvements in these states, the percentage of injury crash involvements that were rear-end striking crashes was larger among vehicles without front crash prevention (15%) than among vehicles with FCW alone (12%) or FCW with AEB (9%). Only 4% of rear- end injury crashes involved fatalities or serious (A-level) injuries." "Approximately 700,000 U.S. police-reported rear-end crashes in 2013 and 300,000 injuries in such crashes could have been prevented if all vehicles were equipped with FCW with AEB that performs similarly as it did for study vehicles." [1] http://orfe.princeton.edu/~alaink/SmartDrivingCars/Papers/IIHS-CicchinoEffectivenessOfCWS-Jan2016.pdf http://orfe.princeton.edu/~alaink/SmartDrivingCars/Papers/II...
  - yohui 10y ago
    For those unaware, "AEB" stands for "Autonomous Emergency Braking". See: https://en.wikipedia.org/wiki/Collision_avoidance_system https://en.wikipedia.org/wiki/Collision_avoidance_system Did Tesla cars have AEB prior to Autopilot installation? If not, then this suggests the 40% reduction in crashes may simply be due to the installation of an AEB system. What effect Autopilot's other features may have would remain uncertain.
    - celticninja 10y ago
- sfifs 10y ago
  Is the base size and confidence levels reported anywhere? I get professionally suspicious when reports publish percentage figure comparisons of rare events.
  - celticninja 10y ago
    What events do you consider rare? Car crashes?
    - usrusr 10y ago
      Crashes of cars that were hard enough to trigger the airbag, but still not so hard that the car would not get an autopilot update later in it's life. The "before autosteer" column does not include miles or crashes of Teslas that did not survive until OTA availability.
    - semi-extrinsic 10y ago
      Presumably they meant airbag deployment crashes in Teslas. We can estimate the absolute number from Tesla's press release in December stating that 1.3 billion miles have been driven on Autopilot, while the NHTSA report states the accidents they count occur 0.8 times per million miles. So in that case it's data from about 1000 crashes in the Autopilot case, and more in the non-Autopilot case. Which tells us that the statistics should be good enough (anything else would be a major scandal). However, that is not what they are comparing! As the NHTSA report states in footnote 21, they're not actually comparing Autopilot miles to normal driving. They are comparing crash rates in vehicles before and after Autopilot was installed in those vehicles. That muddies the waters quite a bit in my opinion, especially when they don't tell us how significant this decrease is. I'm not saying it's wrong, but I'm also not very confident that it is a causal effect, as opposed to just a correlation. Here is a PDF link to the NHTSA report, since TFA only links to a Scribd (thus unusable) version. https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF
      - jayjay71 10y ago
        So they're comparing miles driven manually before a certain date to miles driven manually and automatically after a certain date? It certainly implies that Autopilot is safer, but why not separate it properly (Autopilot vs manual) and give us far more accurate information? It just seems...odd.
- DonHopkins 10y ago
  The car should immediately pull over and shut down if it detects any activity on the driver's Twitter or Facebook feeds: Social Media Strike Out!
  - 6stringmerc 10y ago
    ...then fashion a start-up based on "Ownership" of the vehicle and code and sell devices to Tesla owners to over-ride any part of the system they don't like, then profit!
    - gravypod 10y ago
      Then sell the customers list to law enforcement to make side money! What a fantastic time to live in SV!
      - abakker 10y ago
        Sell the list of those using social media in the car to existing advertisers too. Extra high levels of engagement!
        nitrogen 10y ago
        Then post a joke thread about it on HN. Maximum Reddit! [Seriously, though. I don't want to downvote everyone in a chain but there's probably a more HN/SNR-friendly way to comment on potential abuses of automaker access to social media feeds.]
      - criley2 10y ago
        >Then sell the customers list to law enforcement to make side money! What a fantastic time to live in SV! implying that law enforcement won't work with homeland security to subpoena your data at cost under a gag order :)
  - celticninja 10y ago
    Would require you to link your social media accounts to your car so easily avoided.
- whyileft 10y ago
  > data supplied by Tesla
  - andruc 10y ago
    Where else could it possibly come from? Tesla is the only one collecting this data.
    - whyileft 10y ago
      bingo
- illumin8 10y ago
  They published that the driver took absolutely no action, and that he had 7 seconds to take some action (7 seconds from the time the truck started to turn across his path until his car struck it). The last action he had taken was to enable the cruise control at 74mph 2 minutes before the crash.
  - snewman 10y ago
    Yes, from previous reports it was already pretty clear that the driver was asleep at the wheel (literally or otherwise). The car is not supposed to allow this; it's supposed to check for hands-on-steering-wheel and other indicators of driver attention. It would have been nice to find out how that system broke down in this case.
    - mikeash 10y ago
      It doesn't require continuous engagement. You can drive with hands off the wheel for several minutes at a time in many situations.
      - xorxornop 10y ago
        Besides, it's perfectly possible to fall asleep but with your hands still touching (even gripping!) the steering wheel.
        gambiting 10y ago
        That's also a solved problem - Mercedes cars have "attention assist" which can figure out how attentive you are by measuring how you drive. If you are literally falling asleep at the wheel, the system will get very annoying and insist that you stop for some rest. Of course, whatever algorithm they are using won't work if the car is driving itself, you can't judge the reaction time if all the driver is doing is holding the wheel.
        Piskvorrr 10y ago
- greglindahl 10y ago
  A lot of people are asking what was in the October 2015 Autopilot 7.0 update. That was the introduction of autosteer. Automatic emergency braking (AEB) was initially released in 6.2, see https://www.tesla.com/sites/default/files/tesla_model_s_software_6_2.pdf https://www.tesla.com/sites/default/files/tesla_model_s_soft... 6.2 was announced in March, 2015.
xenadu02 10y ago
For those who don't want to signup to Scribd just to download a publicly available PDF: https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF
- sigmar 10y ago
  Thank you! I'm completely unwilling to register for that site just to download a pdf
sandworm101 10y ago
Great. There is no doubt that driver assists cut down on crashes. But what tesla has on the road is far from a total eyes-closed autopilot. That is an inflection point with this tech that nobody has dared to test on the public road. I remain unconvinced pending those trials. Also, still havent seen any autodrive handle off-road driving such as boarding a carferry or navigate a construction zone manned by an inattentive flag person.
- fmihaila 10y ago
  > But what tesla has on the road is far from a total eyes-closed autopilot. It doesn't matter that it is not full autopilot. What matters is that whatever they have right now, as currently deployed in the real world, produced the numbers behind NHTSA's conclusions.
- Tepix 10y ago
  What's your point?
  - sandworm101 10y ago
    The dream remains a dream. Being able to call your car like batman did the batmobile, a car able to navigate unsupervised is still a decade, perhaps decades, away.
    - wishinghand 10y ago
      Ok sure, but I don't care about that as much as I care about a drastic reduction in danger on the road. This is a big step forward for that.
    - laichzeit0 10y ago
      Oh jeez. So what if it's 10 years away? Even if it's 20 years away it's a piss drop in terms of time. Do you remember what the Internet was like in 1994? I do. It was shit. There wasn't even Google. Forget about looking things up on your mobile, there wasn't even mobile, nevermind Internet on mobile. It's barely 20 years and the world has _completely_ changed because of it. Self-driving cars are barely at the 1994 Internet stage. It's more like at the BBS stage with 9600 baud modems. Just sit back and chill. It's going to happen.
stale2002 10y ago
Oh, hey, will you look at that. The imperfect, incomplete, beta, level 2 self driving cars that were supposed to be the "dangerous" area of self driving are ALREADY better than human drivers. Can we stop the politics and deploy all the real self driving cars to the road immediately, since the government has proven that even the shitty variety is safer than humans?
- Shivetya 10y ago
  I think the primary issue many of us have is the name. the name implies far more capability than the system delivers. I have seen clear day demonstrations but am eagerly awaiting Tesla to show us a night time rainy video which I have seen references of testing having been done
  - extrapickles 10y ago
    Here is video of the automatic breaking feature triggering at night during light rain: https://www.youtube.com/watch?v=9X-5fKzmy38 https://www.youtube.com/watch?v=9X-5fKzmy38 The name is good for technical people, not so much for lay people who don't know that autopilots are more like cruse controls than something that completely flies the plane (they have been getting better though). The 40% reduction in air-bag activation seems to indicate the name is good enough while its probably not optimal.
  - hsod 10y ago
    Autopilot on planes still requires there to be human pilots for edge cases. I think everyone understands this, because everyone knows that there are still pilots on planes.
- kolbe 10y ago
  on the other hand, i am easily 3 times safer than the average driver, so self driving vehicles would endanger me.
  - robotresearcher 10y ago
    Even if that was true, the fact that all the other idiots are in self driving cars that are not crashing into you makes you safer.
dkonofalski 10y ago
I don't really know why this is surprising. Computers are already better than humans at most tasks that involve a limited set of behaviors and they have infinitely better response time than humans (and continue to get better). How could anyone think that a report like this was going to end up any differently?
- jay-anderson 10y ago
  It's okay to withhold judgement and to be skeptical until seeing actual data. Especially for rapidly developing technologies like self-driving cars.
  - dkonofalski 10y ago
    I didn't say it wasn't ok to be skeptical, just that I wasn't surprised by the results. Teslas have been on the road for over a year with AutoPilot and there haven't been any accidents caused by the system. Why anyone is surprised that AutoPilot is safer than a human driver is what I'm not understanding. Based on the votes, though, apparently computers are evil and shouldn't be trusted and we should all be amazed when they don't just kill us.
    - inimino 10y ago
      You said you didn't understand why it was surprising, which is a different thing than not being surprised. Computers are good at lots of things like playing chess, and still completely incapable at lots of things that involve common sense. Expecting autonomous cars to be better than human drivers eventually is one thing, but it is certainly not obvious that we are there yet.
- ams6110 10y ago
  You might want to read about Therac-25
- krschultz 10y ago
  The history of self driving cars shows that it is not an easy task. We've been working on this for over 15 years. I was on a self driving car team during 2008-2010. We won multiple intercollegiate competitions. At that time I was certain it would be 20+ years before we had something that could drive on the roads. The progress has been spectacular.
sxp 10y ago
The 40% number isn't very informative. The report has multiple notes about it: ODI analyzed data from crashes of Tesla Model S and Model X vehicles involving airbag deployments that occurred while operating in, or within 15 seconds of transitioning from, Autopilot mode. Some crashes involved impacts from other vehicles striking the Tesla from various directions with little to no warning to the Tesla driver. ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to[21] and after Autopilot installation.[22] Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation. 21 Approximately one-third of the subject vehicles accumulated mileage prior to Autopilot installation. 22 The crash rates are for all miles travelled before and after Autopilot installation and are not limited to actual Autopilot use. So the actual rates of crashes for Teslas using Autopilot vs Teslas not using Autopilot aren't reported.
- NoodleIncident 10y ago
  I'm not sure that it matters as much as you imply. Even if the only benefit of Tesla's autosteer is that it makes you more focused in the parts that are actually dangerous, a 40% improvement is still a 40% improvement.
- gwern 10y ago
  Reading it, I think it is essentially a 'before October 2015' vs 'after October 2015' comparison of overall accident rates, given the caveats there. So the -40% here should be a lower bound, since it includes a time period with 0% Autopilot usage (before October 2015) and a time period with an unknown but <100% Autopilot usage (after October 2015). If people after October 2015 used Autopilot for 50% of their mileage, then you'd expect the true effect of Autopilot to be more like -80%. This strikes me as suspiciously high: what fraction of airbag-worth accidents are supposed to happen on the highways where one can use Autopilot at all? So maybe there's some other trend going on which is driving this decrease.
huangc10 10y ago
Can anyone who is in the industry comment on how Autopilot performs in poor weather (ie. flash floods, thunderstorms, snowstorms etc...) All I can find from the article about weather was in section 3.1: > The manual includes several additional warnings related to system limitations, use near pedestrians and cyclists, and use on winding roads with sharp curves or with slippery surfaces or poor weather conditions. The system does not prevent operation on any road types.
- martin_bech 10y ago
  Havent tried it in heavy snow yet, but it performs pretty good in heavy rain. The software is constantly updated, and as a user, you really feel it getting better.
- NickM 10y ago
  It's not recommended, but does apparently work in a pinch: https://electrek.co/2016/12/28/tesla-autopilot-snow/ https://electrek.co/2016/12/28/tesla-autopilot-snow/
- problems 10y ago
  Not sure about Tesla's, but many others are completely disabled or say specifically not to use them in poor weather conditions. You have to look at what they're using for this, it's cameras with computer vision and radar. Radar has reflection problems like all hell at ground level and computer vision using cameras is generally piss poor (hence why you see videos of the Tesla hitting reflective objects). It's a miracle that they can do this well using it IMO, even without adverse weather.
- chrisseaton 10y ago
  How well does anyone drive through a flash flood?
- secabeen 10y ago
  It depends. The Tesla system is reliant on good contrast between the road stripes and the road surface. With grey skies and rain, that contrast can go down to a level where the lines cannot be reliably discerned, and AutoSteer is not available.
tn13 10y ago
40% figure is meaningless unless the absolute numbers are reported. How do we know if this difference is statistically significant ?
- elchief 10y ago
  was 1.3 crashes / million miles. Is now 0.8 crashes / million miles. Tesla's have driven >= 3 billion miles
  - danielharan 10y ago
    So 3000X the difference of 0.5, or 1500 crashes avoided. :O
cbr 10y ago
This is really good news. A major worry with driverless cars has been that companies would be harshly punished for accidents, even if there was a dramatic reduction in crashes overall.
- scotty79 10y ago
  They managed to weasel themselves out of similar problem when they rebranded vehicular manaslaughter to jaywalking shifting the blame to the victim. I'm sure there's no problem here that a bit of marketing and societal engineering can't solve.
bcaulfield 10y ago
So I'm far less likely to crash if I use this, and I have something to blame if I do. Everybody wins! (Except the engineers).
deleted 10y ago
[deleted]
randomstring 10y ago
Waiting for the headline: "Human Fails to Prevent Accident, Outraged Public Calls for Banning of all Human Drivers" The obsession with perfection in self-driving cars is misplaced, they just need to be demonstrably better than humans. This is obviously the future.
- Twisell 10y ago
  Or an alternate headline : "Telsa confirm they will always say its the customer fault!"
- NamTaf 10y ago
  At the very least in a legal standpoint that's not true, because with a human driver it's far easier to define fault (which in turn allows for the issue to be resolved) than if two autopilots happen to collide. Who pays for what then? From a safety point of view, I'm more inclined to agree however I'm not entirely convinced that we've passed the novelty stage of it yet where drivers will remain vigilant because it's a new and exciting feature and that mentally primes them to keep focused. It wouldn't suprise me that as it becomes more mainstream, drivers begin to be less vigilant on staying focused as society as a whole begins to trust it more. That's when everything could go bad. Nevertheless, you're absolutely right that this is the future no matter what so we need to design it as best we can to resolve all of the issues it may give rise to.
  - furyg3 10y ago
    There's all sorts of cases in real life now when defining fault is not at all easy. Children and animals are two examples of 'autonomous' individuals who's liability may lie with another party (parent/owner), buy also may not, depending on the circumstances. Additionally, who's at 'fault' really doesn't matter when in most cases the damages are fully outsourced to insurance providers. In the case of autonomous driving, this insurance burden is shifted up a level to the automaker, but this cost will be passed on to the car's owner in some way, either in the cost of the car or a subscription to autonomous driving. However if autonomous driving is safer than non-autonomous driving, premiums will go down, probably dramatically. Insurance companies can much more easily monitor the quality of autonomous driving for various automakers / cars either by the testing specification that they meet or the real-world statistics. Are autonomous Tesla's in less accidents than autonomous Volvo's? Cheaper premiums for Tesla. This is exactly the situation you want to be in. Premiums for individuals are blunt instrument... it's hard to change many individuals behavior, the rewards may be low for the individual (save $3 on your premium!), and the penalties can be pretty random (you sped that one time so you're a high risk driver... or are there just more police in your city?). It's way better to incentivize at a higher level, like the automaker. The only case where automakers would truly be at risk is something they should control, and that is if they pulled a VW and committed fraud on their autonomous driving tests, or were negligent in overlooking scenarios or providing updates.
ChuckMcM 10y ago
That is a pretty remarkable report. It essentially holds Tesla up as an exemplar of the standard other car makers will be expected to achieve.
zekevermillion 10y ago
Impressive. 2/5 reduction is a lot of lives saved.
mrtron 10y ago
What other car company can even recover the airbag deployment rate per mile?
- greglindahl 10y ago
  I believe that GM's OnStar also collects that data.
Animats 10y ago
It's interesting how vague this is. There's an NTSB investigation still pending into a specific Tesla crash.[1] The goals are different. NHTSA asks "do we need to do a recall?" NTSB asks "exactly what, in detail, happened here?" NTSB mostly does air crashes, but occasionally they do an auto crash with unusual properties. Here's the NTSB report for the Midland, TX crash between a train and a parade float.[2] That has detailed measurements of everything. They even brought in a train and a truck to reconstruct the accident positions. It took a combination of problems to cause that crash. The police lieutenant who had informed the railroad of the parade in previous years had retired, and his replacement didn't do it. The police marshalling the parade let it go through red lights. They were unaware that the traffic light near the railroad crossing was tied in to the crossing gates and signals. That's done to clear traffic from the tracks when a train is approaching before the gates go down. So ignoring the traffic signal took away 10 seconds of warning time. The driver thought the police had taken care of safety issues and was looking backwards at the trailer he was pulling, not sideways along the track. People at the parade were using air horns which sounded like a train horn, so the driver didn't notice the real train horn. That's what an NTSB investigation digs up. Those are worth reading to see how to analyze a failure. [1] https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY16FH018-preliminary.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW... [2] https://www.ntsb.gov/investigations/AccidentReports/Pages/HAR1302.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HA...
- pat_space 10y ago
  Very interesting, thanks for posting
- everlost 10y ago
  Yes, it does seem vague. Isn't the NHTSA investigation meant to be a non-biased root-cause-analysis about last year's trailer crash? Instead, this just seems to be a grading report on Tesla's autopilot compliance.
  - greglindahl 10y ago
    Section 5.3 pretty much sums up the root cause of this accident: the driver assist requires the driver to be alert, and the driver had 7 seconds to react and didn't. There's no requirement that driver assist software magically become self-driving software in that situation. Put another way: it's nice if the software can avoid a problem, but not required. The main remaining concern is that drivers not be confused about what they are required to do; there's discussion about that in this document.
battlebot 10y ago
I don't completely trust the NTSA and I'm skeptical about auto-piloting cars but accept that more and more of those will be on the roads. I will never ride in a vehicle that lacks an override mechanism. In general, I think we are moving way too fast towards these self-driving vehicles because certain factions want to try and replace long and short haul truckers with robotic systems that are cheaper and damn the consequences.
uncoder0 10y ago
After looking at the report it looks like Tesla ran into the same issue we did in the 2007 DARPA Urban Challenge. The trailer was higher than the front facing sensors. We and most other teams had all assumed 'Ground Based Obstacles' meant that any obstacles on the test track would make contact with the ground in the lane of travel. DARPA decided to put a railroad bar across the street and expected cars to back up and do a U-Turn when they encountered it. The bar was too high off the ground for our forward LIDAR to see it so we collided with the bar at nearly full speed.[1] The sad part about this is that when we were drinking after dropping out of the challenge our team leader said something along the lines of 'At least we know no one will ever die now from the mistake we just made.' [1] https://www.wired.com/2007/10/safety-last-for/ https://www.wired.com/2007/10/safety-last-for/
- mjevans 10y ago
  Predictions like that only work out if the lesson is sufficiently broadcast. Clearly, since this is still newsworthy here, that is not the case. (However this back channel is helping.)
  - maxander 10y ago
    If there isn't some compilation or review article along the lines of "all serious failure modes encountered in autonomous vehicles since 2005-ish," there should be.
    - taneq 10y ago
      That's an excellent idea and should be managed centrally (by the NHTSA or similar). Basically a communal regression test suite that all self-driving vehicles have to pass.
  - uncoder0 10y ago
    Agreed, this is a fantastic idea. All the DARPA documents from all three autonomous car related Grand Challenges should be availible. Not sure if they are all released on their website if not they're FOIA'able since most everything DARPA does is public.
themgt 10y ago
I don't always like Gladwell, but his piece on the Ford Pinto and the NHTSA philosophy towards auto safety more generally is quite worth the read [1]. I hadn't considered the intersection of this and self-driving car tech, but I wonder if NHTSA will basically take the position that as long as self-driving tech saves lives overall, a few "bugs" where the car kills the driver are an acceptable trade-off. [1] http://www.newyorker.com/magazine/2015/05/04/the-engineers-lament http://www.newyorker.com/magazine/2015/05/04/the-engineers-l...
brilliantcode 10y ago
For your reference, a level 4 automated car will look something like this: https://www.youtube.com/watch?v=jhUX8qWFGc4 https://www.youtube.com/watch?v=jhUX8qWFGc4 Imagine you are too fucked up to drive. Your car will be able to pick you up. Do you need Pepto Bismol too? Your car will pick it up from a drive through billed through your license plate. I'd give roughly 15~20 years for this to take place.
schraitle 10y ago
Does anybody know what the "Population" field indicates at the top of the report?
NamTaf 10y ago
I've railed on about the safety issues of autopilot before and how I'm not entirely comfortable with the pace they've developed compared to the considerations of human-machine interfaces and driver attentiveness, particularly given my (moderate) exposure to these sort of problems in other industries. Thus I'm particularly interested in that section of the article. What I found interesting is that figure 10 shows that as you jack up the independence of the machine, the level of driver distraction accordingly increases. Adaptave Cruise Control (ACC) shows a significantly higher percentage of shorter-duration off-road glances than Limited-Ability Autonomous Driving Systems (LAADS). Additionally, countermeasures help to alleviate some but not all of that increase in distraction. Importantly, this is coupled with the point that the duration in which drivers have to react to most impending is under 3 seconds. This may seem obvious but it's a critical set of data to help objectively demonstrate the risks involved with losing or even reducing alertness. It goes on to say that Tesla has addressed the risks of mode confusion, distraction, etc. and has implemented solutions to address this unreasonable risk, which they in turn define as abuse that is reasonably forseeable. In this, they're talking about the reasonably forseeable risk of eg: the driver not understanding if they're in autopilot or not. It goes on to mention that Tesla has also changed its driver monitoring strategy to promote driver attention, which I take to mean detecting hands on the steering wheel. Either way, Telsa's main approach to dealing with driver alertness by testing for hands on the steering wheel. My concern is that this doesn't consider the alertness of the driver to their surroundings, particularly other vehicles that may be approaching them or the process of anticipating hazards (approaching an intersecion where there's a blind corner and adjusting focus to pay more attention to what may come from it, for example). I don't see how Tesla's countermeasures address this. The physical act of manually driving causes drivers to maintain alertness not only to where they're going, but also the situational alertness of what's around their vehicle. Specifically, it's the process of random actions that requires an taking input, making a decision and executing the appropriate action that maintains this alertness. If the driver isn't having to make those random decisions and take action then their alertness drops. Autopilot, even with hands on the wheel, eliminates much of that random decision-making and reacting. When you drive, you mentally note the vehicle over your shoulder that is in the lane next to you, and subconsciously consider that they may do something insane. You consider those blind corners as you approach them and that vehicles may spontaneously appear out from them. You see a truck on the road which is approaching a bend and give it a wider berth because its centre throw may cause it to cut the corner into your lane slightly. These are all tasks that you do, that you may not do as well or at all when autopilot is steering, because you are not as engaged with the driving process. Critically, I don't see how ensuring hands are on the steering wheel causes these alertness tasks to continue as frequently as manual steering. The driver may be in the physical location to quickly take over, but they may not be in the mental location to do so. This is the major issue I have with the rapid autopilot development based on my experience in related areas where maintaining situational alertness proved to be very difficult when the person was engaged with only a limited scope of requirements to prove their conscious presence. I feel like the report doesn't really drill in to this as much as it needs to. It begins to touch on it around Figure 10 but sort of hand-waves it away saying 'Telsa considered it discharged their responsibility to make sure drivers stay focused by implementing countermeasures', but I believe it's more nuanced than that. It investigates the extent to which Tesla's system is good at ensuring drivers are physically present (that is, their hands aren't on the passenger seat making breakfast) but it doesn't really look at the mental presence that delivers situational alertness. That mental alertness is the major sticking point for me. I don't really have a solution beyond "drive manually" which isn't reasonable, because this technology is here to stay and will continue to grow, but it's why I've always been bearish about the rapid pace of rollout of these driverless technologies, particuarly when advertised as 'beta'. As I've said before, no amount of disclaimer and 'hey, you should do this' really changes how drivers behave once the equipment is placed in their hands.
ridiculous_fish 10y ago
This means that autopilot must be engaged at least 40% of the time (Amdahl's law!). Tesla owners, is that realistic?
lz400 10y ago
I think there's sometimes a lot of marketing and hand waving in this type of argument "crashes go down with autopilot". Most car accidents are caused by drunk or old people, and they drag the average up. if you tell me a Tesla autopilot beats a drunk guy it won't surprise anyone. Now as a non-drunk, young(ish) driver but experienced and careful, my statistics don't look anything like the average, they look at lot better. You have to convince this demographic, not beat the averages otherwise it's not rational for me to buy the feature. I'm guessing this goal post is a lot harder.
- phinnaeus 10y ago
  I'm confused. You think young(ish) people need to be convinced to buy semi-autonomous vehicles? If they can afford it, why wouldn't they buy it? People love things that make their life more convenient. In my experience, its the older people that are somehow scared of the concept of a self-driving car and would need to be convinced.
  - lz400 10y ago
    Because of security. I wouldn't buy an autonomous car that has an accident rate worse than my demographic. A car that has an accident rate better than average is not good enough. I hope that explains my point.
- mfgrimm 10y ago
  I suppose the best comparison would be to compare non-Tesla owners of the same demographic. Are accident rates correlated to SES? Do people who buy luxury sedans crash less? Just a guess: probably. Still, I know I drive stupidly sometimes, and autopilot might help.
  - lz400 10y ago
    Yes exactly, it would be interesting to see fair comparisons of people of the exact same profile (both driver and car) tesla vs others. I don't like that so many people seem happy enough to compare autopilot against the total average of accidents in the road and say "they save lives". When you are going to buy the car you want to know if it's going to be better than you.
em3rgent0rdr 10y ago
I only want to use open-source code for something that my life depends on. That way it can be open to any one to inspect some can independently determine if the code is behaving as desired.
brighton36 10y ago
The truck was visible for at least 7 seconds prior to the crash in the full report - another article here: http://arstechnica.com/cars/2017/01/after-fatal-tesla-crash-probe-us-regulators-conclude-theres-no-need-for-recall/ http://arstechnica.com/cars/2017/01/after-fatal-tesla-crash-... Strangely enough, giving more people autopilot would probably be better than letting people drive. I think Tesla's picked the right time to enable it, since the cross-over point between autopilots being better than humans in general use cases has been reached. Call it a beta if you want, but it's a pretty damn promising beta.

808 points by fmihaila 10y ago

Tesla comes off extremely well in this report. For one thing, the 40% statistic cited in the headline appears to be well supported by the NHTSA report (section 5.4) and actually manages to frame the incident in a very positive light: ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to and after Autopilot installation. Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation. I had hoped to see more information about this specific incident. For instance, any data on whether the driver had his hands on the wheel, what steps the car had taken to prompt his attention, etc. But that doesn't seem to be included.

Roritharr 10y ago
I really would like to know the absolute numbers for this reduction. It's really questionable if this would hold up with millions of Teslas on the road in the hands of less wealthy(probably better educated) drivers.
- dkonofalski 10y ago
  Wait... are you saying that people that are less wealthy are better educated? If so, I need to see some sources. That sounds preposterous.
  - jackmott 10y ago
    no. he needs more parens to make the order of operations clear. also, love the standard internet response to data. "the data in not perfect, or is incomplete, thus I will hold steadfast to my intuition" love you, internet.
    - GauntletWizard 10y ago
      It's very reasonable, when presented with data that doesn't pass a sniff check, to mentally bitbucket that data. Unfortunately, lots of people's noses are miscalibrated - And what's worse we're not sure which ones.
  - randomdata 10y ago
    Assuming by education you mean formal schooling: Consider that wealth is accumulated over time. While outliers certainly exist, the older you get, the more wealthy you are apt to be, at least to the point when you retire and start to use those reserves to live on. Also consider that in 1970s, the time when the boomers started coming of age, having 'high school or more' was reserved for just 50% of the population. Only a mere 25% for the oldest segment of the population. It has steadily climbed to more than 80% for the youngest adults. But they, statistically, haven't had time to amass a large amount of wealth yet.
  - avenoir 10y ago
    I also found that confusing. I think in this case "less wealthy" is within the context of Tesla owners, which is basically upper middle class(?), which sort of makes sense. At least that's how I read it.
  - Roritharr 10y ago
    No, i just took education as one possible factor that might be different between Tesla and Camry or Hummer drivers. Tesla Owners have a specific selection bias, its just hard to describe what that bias actually is, so i just took a stab in a direction to get the discussion going.
- cryptoz 10y ago
  Probably as the technology scales up to millions of people using it, the numbers will actually be even better for safety. There is a network effect that compounds the safety factor - as more self-driving cars are on the road, there are fewer rogue human drivers. So with millions of Teslas driving around there will be that many fewer people causing accidents by human error.
- Tepix 10y ago
  The rates dropped by 40%. The drivers remained the same. Why do you think a different driver demographic will not show a similar improvement?
  - wccrawford 10y ago
    Maybe other drivers are really good, or maybe this class of driver is really bad?
    - cryptoz 10y ago
      35,000 people die from car accidents every single year in the USA alone. Other drivers are not "really good".
      - ebbv 10y ago
        Personally I believe most people are really terrible drivers. That being said, there were 210 million drivers in the US in 2009. There's probably a lot more today. That's just 0.01% of drivers. That fact makes me have to question just how bad people really are, despite my own personal opinion. Or maybe cars are just really safe. A better statistic might be number of accidents. In 2015 2.44 million people were injured in car accidents. That's a much more telling figure of just how bad most drivers are. Apparently our cars are just really safe.
        camiller 10y ago
        I had an econ professor assert that as cars got safer there were more accidents because people trusted the car to protect them more.... Logically, to reduce accidents due to carelessness we should put a 12" sharpened spike right in the middle of the steering wheel pointing at the drivers chest. If all accidents were deadly wouldn't drivers then exercise the utmost care?
        OrwellianChild 10y ago
        It's a great joke, but a poor strategy. Remember the end game: minimize harm. The universal solutions that remove human error have a better long-run potential to minimize harm, since they can exceed the effectiveness of even the most attentive and cautious human driver.
- deleted 10y ago
  [deleted]
- celticninja 10y ago
  Are you assuming the more wealthy owners have lower levels of education? Or have I misinterpreted your statement? I would have assumed tesla owners would fall into the more educated category. No idea which assumption is correct just wondering about your reasoning. For my assumption I assume that due to the higher cost of a Tesla better educated people are opting for them over the alternative because of the environmental benefits.
  - randomdata 10y ago
    I would have assumed due to the high cost of a Tesla that the owners would tend to be older, and thus a generation educated in much fewer numbers. Today's youth are much more educated, but they are still young, which hasn't given them much time to build wealth. I mean, who do you think is more likely to buy a Tesla: A 20 year old fresh college graduate, or his 'uneducated' father who has been saving and investing for 30 years and just sold his lifelong home in San Francisco? My bet is on the latter. The boomers in particular are considered the wealthiest generation ever, but not the most educated generation ever. I'm rather fascinated by how many are questioning the education thing here though. Coming from a farming community, where every (older) farmer I know is multimillions simply by virtue of having purchased farmland when they were young, it's difficult to see how education had any impact on that wealth. Some of them do have educations, some don't. It doesn't seem to make any difference.
    - celticninja 10y ago
      I evidently misunderstood the original comment, when OP said less wealthy (better educated) he meant that there would be fewer wealthy (and better educated) owners. I don't agree with the sentiment and think OP didn't really explain his position well either.
      - randomdata 10y ago
        You don't agree that wealthy people tend to be older? Even though time is the easiest way to increase wealth (compound interest is your friend). Or you don't agree that older people are less educated? Even thought the data suggests otherwise.
- atonalfreerider 10y ago
  Wealthy motorists drive more recklessly: http://www.pnas.org/content/109/11/4086.short http://www.pnas.org/content/109/11/4086.short
  - Roritharr 10y ago
    Makes sense, but its still interesting how the result would change when applied to the general population. Would VW Beetle drivers have the same reduction?
  - mozumder 10y ago
    I think you pointed to the wrong link. That's a study about ethics, not safety.
    - atonalfreerider 10y ago
      If you read the abstract you will see that the studies involve driver behavior. "Results Studies 1 and 2. Our first two studies were naturalistic field studies, and examined whether upper-class individuals behave more unethically than lower-class individuals while driving. In study 1, we investigated whether upper-class drivers were more likely to cut off other vehicles at a busy four-way intersection with stop signs on all sides. As vehicles are reliable indicators of a person's social rank and wealth (15), we used observers’ codes of vehicle status (make, age, and appearance) to index drivers’ social class. Observers stood near the intersection, coded the status of approaching vehicles, and recorded whether the driver cut off other vehicles by crossing the intersection before waiting their turn, a behavior that defies the California Vehicle Code. In the present study, 12.4% of drivers cut in front of other vehicles. A binary logistic regression indicated that upper-class drivers were the most likely to cut off other vehicles at the intersection, even when controlling for time of day, driver's perceived sex and age, and amount of traffic, b = 0.36, SE b = 0.18, P < 0.05"
      - nitrogen 10y ago
        Did they account for bias in their observers?
- onmobiletemp 10y ago
  They said it was a 40 percent drop after the roll out of auto-pilot features. That means it dropped 40 percent with the same driver base. Please read the article before posting comments.
- Ar-Curunir 10y ago
  So "better" drivers saw 40% reduction; doesn't that indicate that worse drivers will see even greater reduction in accidents?
- lamontcg 10y ago
  > wealthy (probably better educated) drivers. You mean drivers like the ones behind the wheel of all the BMWs and Audis that cut me off with no turn signals, speed, make aggressive lane changes, tailgate, and generally act like giant douches because a simple traffic ticket is nothing to their pocketbook while their time == money? Those people are going to be a hell of a lot safer behind the wheel of an autonomous (level 4) vehicle where they can be on the phone and their laptop as the vehicle obeys the speed limits and safe following distance.
  - deleted 10y ago
    [deleted]
  - celticninja 10y ago
    He said less wealthy, better educated drivers.
    - lamontcg 10y ago
      Pretty certain he meant it the way I quoted it.
      - celticninja 10y ago
        Ah it now makes sense, I did ask him to clarify but I just parsed in my brain when reading your reply he meant fewer wealthy (better educated drivers) as opposed to less wealthy meaning not as wealthy. Thanks for your help.
        brokenmachine 10y ago
        I read it as less wealthy meaning not as wealthy as well. Funny how ambiguous some sentences can be.
        delecti 10y ago
        Well it's really that he used the wrong word. "Less wealthy drivers" can't correctly refer to the number of drivers.
tehlike 10y ago
I didnt read the report, but is this measurement done on a a/b testing way? There is usually some data skew/bias when they are done on different time scale.
- vvanders 10y ago
  Same cars pre/post OTA. There was a ~6 month peroid where AP HW was out in the wild but not used.
  - pps43 10y ago
    Confounding factors. For example, crashes can be more likely after sunset or during inclement weather so seasonality can play a role. Or maybe the drivers who had Tesla longer got used to it.
    - celticninja 10y ago
      Could this be accounted for if the sample period was large enough, ie over 12 months to cover seasonality and a few months for time of day.
      - tehlike 10y ago
        this particular one, maybe, but there might be others. like the user base pre-ota and post ota (number-wise) could be different. I think there is a good SaaS business with A/B testing robot/car software OTA. I would imagine some metrics would be: 1. car speed 2. battery consumption per mile. 3. user comfort metrics (if it's possible to measure, maybe like hearthrate, body temperature etc - i can imagine if robots were driving bad, it'd be visible in heart rate or something like that).4 The list probably goes on. It would be interesting to see how much battery consumption changed after OTA, for example.
      - pps43 10y ago
        I suspect OTA update was applied simultaneously rather than staggered over the course of the year, so probably not. You can control for seasonality using some other data as a proxy, but the more adjustments you make, the more potential there is for introducing errors. Randomized test/control split is almost always better than before/after comparison.
Vik1ng 10y ago
Just funny that the 40% from Autosteer seem to exactly match with the general AEB safety improvement rate also mentioned in the report... IIHS research shows that AEB systems meeting the commitment would reduce rear-end crashes[emphasis added] by 40 percent. IIHS estimates that by 2025 – the earliest NHTSA believes it could realistically implement a regulatory requirement for AEB – the commitment will prevent 28,000 crashes and 12,000 injuries.
- GrantS 10y ago
  Here is some of that IIHS research [1] from Jan 2016 that gives lots of raw crash numbers broken down by manufacturer, type of system (AEB/FCW), injuries involved, etc. Really informative. Some excerpts from "Effectiveness of Forward Collision Warning Systems with and without Autonomous Emergency Braking in Reducing Police-Reported Crash Rates", January 2016: "FCW alone and FCW with AEB reduced rear-end striking crash involvement rates by 23% and 39%, respectively. " "Among the 15,802 injury crash involvements in these states, the percentage of injury crash involvements that were rear-end striking crashes was larger among vehicles without front crash prevention (15%) than among vehicles with FCW alone (12%) or FCW with AEB (9%). Only 4% of rear- end injury crashes involved fatalities or serious (A-level) injuries." "Approximately 700,000 U.S. police-reported rear-end crashes in 2013 and 300,000 injuries in such crashes could have been prevented if all vehicles were equipped with FCW with AEB that performs similarly as it did for study vehicles." [1] http://orfe.princeton.edu/~alaink/SmartDrivingCars/Papers/IIHS-CicchinoEffectivenessOfCWS-Jan2016.pdf http://orfe.princeton.edu/~alaink/SmartDrivingCars/Papers/II...
- yohui 10y ago
  For those unaware, "AEB" stands for "Autonomous Emergency Braking". See: https://en.wikipedia.org/wiki/Collision_avoidance_system https://en.wikipedia.org/wiki/Collision_avoidance_system Did Tesla cars have AEB prior to Autopilot installation? If not, then this suggests the 40% reduction in crashes may simply be due to the installation of an AEB system. What effect Autopilot's other features may have would remain uncertain.
  - celticninja 10y ago
sfifs 10y ago
Is the base size and confidence levels reported anywhere? I get professionally suspicious when reports publish percentage figure comparisons of rare events.
- celticninja 10y ago
  What events do you consider rare? Car crashes?
  - usrusr 10y ago
    Crashes of cars that were hard enough to trigger the airbag, but still not so hard that the car would not get an autopilot update later in it's life. The "before autosteer" column does not include miles or crashes of Teslas that did not survive until OTA availability.
  - semi-extrinsic 10y ago
    Presumably they meant airbag deployment crashes in Teslas. We can estimate the absolute number from Tesla's press release in December stating that 1.3 billion miles have been driven on Autopilot, while the NHTSA report states the accidents they count occur 0.8 times per million miles. So in that case it's data from about 1000 crashes in the Autopilot case, and more in the non-Autopilot case. Which tells us that the statistics should be good enough (anything else would be a major scandal). However, that is not what they are comparing! As the NHTSA report states in footnote 21, they're not actually comparing Autopilot miles to normal driving. They are comparing crash rates in vehicles before and after Autopilot was installed in those vehicles. That muddies the waters quite a bit in my opinion, especially when they don't tell us how significant this decrease is. I'm not saying it's wrong, but I'm also not very confident that it is a causal effect, as opposed to just a correlation. Here is a PDF link to the NHTSA report, since TFA only links to a Scribd (thus unusable) version. https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF
    - jayjay71 10y ago
      So they're comparing miles driven manually before a certain date to miles driven manually and automatically after a certain date? It certainly implies that Autopilot is safer, but why not separate it properly (Autopilot vs manual) and give us far more accurate information? It just seems...odd.
DonHopkins 10y ago
The car should immediately pull over and shut down if it detects any activity on the driver's Twitter or Facebook feeds: Social Media Strike Out!
- 6stringmerc 10y ago
  ...then fashion a start-up based on "Ownership" of the vehicle and code and sell devices to Tesla owners to over-ride any part of the system they don't like, then profit!
  - gravypod 10y ago
    Then sell the customers list to law enforcement to make side money! What a fantastic time to live in SV!
    - abakker 10y ago
      Sell the list of those using social media in the car to existing advertisers too. Extra high levels of engagement!
      - nitrogen 10y ago
        Then post a joke thread about it on HN. Maximum Reddit! [Seriously, though. I don't want to downvote everyone in a chain but there's probably a more HN/SNR-friendly way to comment on potential abuses of automaker access to social media feeds.]
    - criley2 10y ago
      >Then sell the customers list to law enforcement to make side money! What a fantastic time to live in SV! implying that law enforcement won't work with homeland security to subpoena your data at cost under a gag order :)
- celticninja 10y ago
  Would require you to link your social media accounts to your car so easily avoided.
whyileft 10y ago
> data supplied by Tesla
- andruc 10y ago
  Where else could it possibly come from? Tesla is the only one collecting this data.
  - whyileft 10y ago
    bingo
illumin8 10y ago
They published that the driver took absolutely no action, and that he had 7 seconds to take some action (7 seconds from the time the truck started to turn across his path until his car struck it). The last action he had taken was to enable the cruise control at 74mph 2 minutes before the crash.
- snewman 10y ago
  Yes, from previous reports it was already pretty clear that the driver was asleep at the wheel (literally or otherwise). The car is not supposed to allow this; it's supposed to check for hands-on-steering-wheel and other indicators of driver attention. It would have been nice to find out how that system broke down in this case.
  - mikeash 10y ago
    It doesn't require continuous engagement. You can drive with hands off the wheel for several minutes at a time in many situations.
    - xorxornop 10y ago
      Besides, it's perfectly possible to fall asleep but with your hands still touching (even gripping!) the steering wheel.
      - gambiting 10y ago
        That's also a solved problem - Mercedes cars have "attention assist" which can figure out how attentive you are by measuring how you drive. If you are literally falling asleep at the wheel, the system will get very annoying and insist that you stop for some rest. Of course, whatever algorithm they are using won't work if the car is driving itself, you can't judge the reaction time if all the driver is doing is holding the wheel.
        Piskvorrr 10y ago
greglindahl 10y ago
A lot of people are asking what was in the October 2015 Autopilot 7.0 update. That was the introduction of autosteer. Automatic emergency braking (AEB) was initially released in 6.2, see https://www.tesla.com/sites/default/files/tesla_model_s_software_6_2.pdf https://www.tesla.com/sites/default/files/tesla_model_s_soft... 6.2 was announced in March, 2015.

xenadu02 10y ago

For those who don't want to signup to Scribd just to download a publicly available PDF: https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF

sigmar 10y ago
Thank you! I'm completely unwilling to register for that site just to download a pdf

sandworm101 10y ago

Great. There is no doubt that driver assists cut down on crashes. But what tesla has on the road is far from a total eyes-closed autopilot. That is an inflection point with this tech that nobody has dared to test on the public road. I remain unconvinced pending those trials. Also, still havent seen any autodrive handle off-road driving such as boarding a carferry or navigate a construction zone manned by an inattentive flag person.

fmihaila 10y ago
> But what tesla has on the road is far from a total eyes-closed autopilot. It doesn't matter that it is not full autopilot. What matters is that whatever they have right now, as currently deployed in the real world, produced the numbers behind NHTSA's conclusions.
Tepix 10y ago
What's your point?
- sandworm101 10y ago
  The dream remains a dream. Being able to call your car like batman did the batmobile, a car able to navigate unsupervised is still a decade, perhaps decades, away.
  - wishinghand 10y ago
    Ok sure, but I don't care about that as much as I care about a drastic reduction in danger on the road. This is a big step forward for that.
  - laichzeit0 10y ago
    Oh jeez. So what if it's 10 years away? Even if it's 20 years away it's a piss drop in terms of time. Do you remember what the Internet was like in 1994? I do. It was shit. There wasn't even Google. Forget about looking things up on your mobile, there wasn't even mobile, nevermind Internet on mobile. It's barely 20 years and the world has _completely_ changed because of it. Self-driving cars are barely at the 1994 Internet stage. It's more like at the BBS stage with 9600 baud modems. Just sit back and chill. It's going to happen.

stale2002 10y ago

Oh, hey, will you look at that. The imperfect, incomplete, beta, level 2 self driving cars that were supposed to be the "dangerous" area of self driving are ALREADY better than human drivers. Can we stop the politics and deploy all the real self driving cars to the road immediately, since the government has proven that even the shitty variety is safer than humans?

Shivetya 10y ago
I think the primary issue many of us have is the name. the name implies far more capability than the system delivers. I have seen clear day demonstrations but am eagerly awaiting Tesla to show us a night time rainy video which I have seen references of testing having been done
- extrapickles 10y ago
  Here is video of the automatic breaking feature triggering at night during light rain: https://www.youtube.com/watch?v=9X-5fKzmy38 https://www.youtube.com/watch?v=9X-5fKzmy38 The name is good for technical people, not so much for lay people who don't know that autopilots are more like cruse controls than something that completely flies the plane (they have been getting better though). The 40% reduction in air-bag activation seems to indicate the name is good enough while its probably not optimal.
- hsod 10y ago
  Autopilot on planes still requires there to be human pilots for edge cases. I think everyone understands this, because everyone knows that there are still pilots on planes.
kolbe 10y ago
on the other hand, i am easily 3 times safer than the average driver, so self driving vehicles would endanger me.
- robotresearcher 10y ago
  Even if that was true, the fact that all the other idiots are in self driving cars that are not crashing into you makes you safer.

dkonofalski 10y ago

I don't really know why this is surprising. Computers are already better than humans at most tasks that involve a limited set of behaviors and they have infinitely better response time than humans (and continue to get better). How could anyone think that a report like this was going to end up any differently?

jay-anderson 10y ago
It's okay to withhold judgement and to be skeptical until seeing actual data. Especially for rapidly developing technologies like self-driving cars.
- dkonofalski 10y ago
  I didn't say it wasn't ok to be skeptical, just that I wasn't surprised by the results. Teslas have been on the road for over a year with AutoPilot and there haven't been any accidents caused by the system. Why anyone is surprised that AutoPilot is safer than a human driver is what I'm not understanding. Based on the votes, though, apparently computers are evil and shouldn't be trusted and we should all be amazed when they don't just kill us.
  - inimino 10y ago
    You said you didn't understand why it was surprising, which is a different thing than not being surprised. Computers are good at lots of things like playing chess, and still completely incapable at lots of things that involve common sense. Expecting autonomous cars to be better than human drivers eventually is one thing, but it is certainly not obvious that we are there yet.
ams6110 10y ago
You might want to read about Therac-25
krschultz 10y ago
The history of self driving cars shows that it is not an easy task. We've been working on this for over 15 years. I was on a self driving car team during 2008-2010. We won multiple intercollegiate competitions. At that time I was certain it would be 20+ years before we had something that could drive on the roads. The progress has been spectacular.

sxp 10y ago

The 40% number isn't very informative. The report has multiple notes about it: ODI analyzed data from crashes of Tesla Model S and Model X vehicles involving airbag deployments that occurred while operating in, or within 15 seconds of transitioning from, Autopilot mode. Some crashes involved impacts from other vehicles striking the Tesla from various directions with little to no warning to the Tesla driver. ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to[21] and after Autopilot installation.[22] Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation. 21 Approximately one-third of the subject vehicles accumulated mileage prior to Autopilot installation. 22 The crash rates are for all miles travelled before and after Autopilot installation and are not limited to actual Autopilot use. So the actual rates of crashes for Teslas using Autopilot vs Teslas not using Autopilot aren't reported.

NoodleIncident 10y ago
I'm not sure that it matters as much as you imply. Even if the only benefit of Tesla's autosteer is that it makes you more focused in the parts that are actually dangerous, a 40% improvement is still a 40% improvement.
gwern 10y ago
Reading it, I think it is essentially a 'before October 2015' vs 'after October 2015' comparison of overall accident rates, given the caveats there. So the -40% here should be a lower bound, since it includes a time period with 0% Autopilot usage (before October 2015) and a time period with an unknown but <100% Autopilot usage (after October 2015). If people after October 2015 used Autopilot for 50% of their mileage, then you'd expect the true effect of Autopilot to be more like -80%. This strikes me as suspiciously high: what fraction of airbag-worth accidents are supposed to happen on the highways where one can use Autopilot at all? So maybe there's some other trend going on which is driving this decrease.

huangc10 10y ago

Can anyone who is in the industry comment on how Autopilot performs in poor weather (ie. flash floods, thunderstorms, snowstorms etc...) All I can find from the article about weather was in section 3.1: > The manual includes several additional warnings related to system limitations, use near pedestrians and cyclists, and use on winding roads with sharp curves or with slippery surfaces or poor weather conditions. The system does not prevent operation on any road types.

martin_bech 10y ago
Havent tried it in heavy snow yet, but it performs pretty good in heavy rain. The software is constantly updated, and as a user, you really feel it getting better.
NickM 10y ago
It's not recommended, but does apparently work in a pinch: https://electrek.co/2016/12/28/tesla-autopilot-snow/ https://electrek.co/2016/12/28/tesla-autopilot-snow/
problems 10y ago
Not sure about Tesla's, but many others are completely disabled or say specifically not to use them in poor weather conditions. You have to look at what they're using for this, it's cameras with computer vision and radar. Radar has reflection problems like all hell at ground level and computer vision using cameras is generally piss poor (hence why you see videos of the Tesla hitting reflective objects). It's a miracle that they can do this well using it IMO, even without adverse weather.
chrisseaton 10y ago
How well does anyone drive through a flash flood?
secabeen 10y ago
It depends. The Tesla system is reliant on good contrast between the road stripes and the road surface. With grey skies and rain, that contrast can go down to a level where the lines cannot be reliably discerned, and AutoSteer is not available.

tn13 10y ago

40% figure is meaningless unless the absolute numbers are reported. How do we know if this difference is statistically significant ?

elchief 10y ago
was 1.3 crashes / million miles. Is now 0.8 crashes / million miles. Tesla's have driven >= 3 billion miles
- danielharan 10y ago
  So 3000X the difference of 0.5, or 1500 crashes avoided. :O

cbr 10y ago

This is really good news. A major worry with driverless cars has been that companies would be harshly punished for accidents, even if there was a dramatic reduction in crashes overall.

scotty79 10y ago
They managed to weasel themselves out of similar problem when they rebranded vehicular manaslaughter to jaywalking shifting the blame to the victim. I'm sure there's no problem here that a bit of marketing and societal engineering can't solve.

bcaulfield 10y ago

So I'm far less likely to crash if I use this, and I have something to blame if I do. Everybody wins! (Except the engineers).

deleted 10y ago

[deleted]

randomstring 10y ago

Waiting for the headline: "Human Fails to Prevent Accident, Outraged Public Calls for Banning of all Human Drivers" The obsession with perfection in self-driving cars is misplaced, they just need to be demonstrably better than humans. This is obviously the future.

Twisell 10y ago
Or an alternate headline : "Telsa confirm they will always say its the customer fault!"
NamTaf 10y ago
At the very least in a legal standpoint that's not true, because with a human driver it's far easier to define fault (which in turn allows for the issue to be resolved) than if two autopilots happen to collide. Who pays for what then? From a safety point of view, I'm more inclined to agree however I'm not entirely convinced that we've passed the novelty stage of it yet where drivers will remain vigilant because it's a new and exciting feature and that mentally primes them to keep focused. It wouldn't suprise me that as it becomes more mainstream, drivers begin to be less vigilant on staying focused as society as a whole begins to trust it more. That's when everything could go bad. Nevertheless, you're absolutely right that this is the future no matter what so we need to design it as best we can to resolve all of the issues it may give rise to.
- furyg3 10y ago
  There's all sorts of cases in real life now when defining fault is not at all easy. Children and animals are two examples of 'autonomous' individuals who's liability may lie with another party (parent/owner), buy also may not, depending on the circumstances. Additionally, who's at 'fault' really doesn't matter when in most cases the damages are fully outsourced to insurance providers. In the case of autonomous driving, this insurance burden is shifted up a level to the automaker, but this cost will be passed on to the car's owner in some way, either in the cost of the car or a subscription to autonomous driving. However if autonomous driving is safer than non-autonomous driving, premiums will go down, probably dramatically. Insurance companies can much more easily monitor the quality of autonomous driving for various automakers / cars either by the testing specification that they meet or the real-world statistics. Are autonomous Tesla's in less accidents than autonomous Volvo's? Cheaper premiums for Tesla. This is exactly the situation you want to be in. Premiums for individuals are blunt instrument... it's hard to change many individuals behavior, the rewards may be low for the individual (save $3 on your premium!), and the penalties can be pretty random (you sped that one time so you're a high risk driver... or are there just more police in your city?). It's way better to incentivize at a higher level, like the automaker. The only case where automakers would truly be at risk is something they should control, and that is if they pulled a VW and committed fraud on their autonomous driving tests, or were negligent in overlooking scenarios or providing updates.

ChuckMcM 10y ago

That is a pretty remarkable report. It essentially holds Tesla up as an exemplar of the standard other car makers will be expected to achieve.

zekevermillion 10y ago

Impressive. 2/5 reduction is a lot of lives saved.

mrtron 10y ago

What other car company can even recover the airbag deployment rate per mile?

greglindahl 10y ago
I believe that GM's OnStar also collects that data.

Animats 10y ago

It's interesting how vague this is. There's an NTSB investigation still pending into a specific Tesla crash.[1] The goals are different. NHTSA asks "do we need to do a recall?" NTSB asks "exactly what, in detail, happened here?" NTSB mostly does air crashes, but occasionally they do an auto crash with unusual properties. Here's the NTSB report for the Midland, TX crash between a train and a parade float.[2] That has detailed measurements of everything. They even brought in a train and a truck to reconstruct the accident positions. It took a combination of problems to cause that crash. The police lieutenant who had informed the railroad of the parade in previous years had retired, and his replacement didn't do it. The police marshalling the parade let it go through red lights. They were unaware that the traffic light near the railroad crossing was tied in to the crossing gates and signals. That's done to clear traffic from the tracks when a train is approaching before the gates go down. So ignoring the traffic signal took away 10 seconds of warning time. The driver thought the police had taken care of safety issues and was looking backwards at the trailer he was pulling, not sideways along the track. People at the parade were using air horns which sounded like a train horn, so the driver didn't notice the real train horn. That's what an NTSB investigation digs up. Those are worth reading to see how to analyze a failure. [1] https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY16FH018-preliminary.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW... [2] https://www.ntsb.gov/investigations/AccidentReports/Pages/HAR1302.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HA...

pat_space 10y ago
Very interesting, thanks for posting
everlost 10y ago
Yes, it does seem vague. Isn't the NHTSA investigation meant to be a non-biased root-cause-analysis about last year's trailer crash? Instead, this just seems to be a grading report on Tesla's autopilot compliance.
- greglindahl 10y ago
  Section 5.3 pretty much sums up the root cause of this accident: the driver assist requires the driver to be alert, and the driver had 7 seconds to react and didn't. There's no requirement that driver assist software magically become self-driving software in that situation. Put another way: it's nice if the software can avoid a problem, but not required. The main remaining concern is that drivers not be confused about what they are required to do; there's discussion about that in this document.

battlebot 10y ago

I don't completely trust the NTSA and I'm skeptical about auto-piloting cars but accept that more and more of those will be on the roads. I will never ride in a vehicle that lacks an override mechanism. In general, I think we are moving way too fast towards these self-driving vehicles because certain factions want to try and replace long and short haul truckers with robotic systems that are cheaper and damn the consequences.

uncoder0 10y ago

After looking at the report it looks like Tesla ran into the same issue we did in the 2007 DARPA Urban Challenge. The trailer was higher than the front facing sensors. We and most other teams had all assumed 'Ground Based Obstacles' meant that any obstacles on the test track would make contact with the ground in the lane of travel. DARPA decided to put a railroad bar across the street and expected cars to back up and do a U-Turn when they encountered it. The bar was too high off the ground for our forward LIDAR to see it so we collided with the bar at nearly full speed.[1] The sad part about this is that when we were drinking after dropping out of the challenge our team leader said something along the lines of 'At least we know no one will ever die now from the mistake we just made.' [1] https://www.wired.com/2007/10/safety-last-for/ https://www.wired.com/2007/10/safety-last-for/

mjevans 10y ago
Predictions like that only work out if the lesson is sufficiently broadcast. Clearly, since this is still newsworthy here, that is not the case. (However this back channel is helping.)
- maxander 10y ago
  If there isn't some compilation or review article along the lines of "all serious failure modes encountered in autonomous vehicles since 2005-ish," there should be.
  - taneq 10y ago
    That's an excellent idea and should be managed centrally (by the NHTSA or similar). Basically a communal regression test suite that all self-driving vehicles have to pass.
- uncoder0 10y ago
  Agreed, this is a fantastic idea. All the DARPA documents from all three autonomous car related Grand Challenges should be availible. Not sure if they are all released on their website if not they're FOIA'able since most everything DARPA does is public.

themgt 10y ago

I don't always like Gladwell, but his piece on the Ford Pinto and the NHTSA philosophy towards auto safety more generally is quite worth the read [1]. I hadn't considered the intersection of this and self-driving car tech, but I wonder if NHTSA will basically take the position that as long as self-driving tech saves lives overall, a few "bugs" where the car kills the driver are an acceptable trade-off. [1] http://www.newyorker.com/magazine/2015/05/04/the-engineers-lament http://www.newyorker.com/magazine/2015/05/04/the-engineers-l...

brilliantcode 10y ago

For your reference, a level 4 automated car will look something like this: https://www.youtube.com/watch?v=jhUX8qWFGc4 https://www.youtube.com/watch?v=jhUX8qWFGc4 Imagine you are too fucked up to drive. Your car will be able to pick you up. Do you need Pepto Bismol too? Your car will pick it up from a drive through billed through your license plate. I'd give roughly 15~20 years for this to take place.

schraitle 10y ago

Does anybody know what the "Population" field indicates at the top of the report?

NamTaf 10y ago

I've railed on about the safety issues of autopilot before and how I'm not entirely comfortable with the pace they've developed compared to the considerations of human-machine interfaces and driver attentiveness, particularly given my (moderate) exposure to these sort of problems in other industries. Thus I'm particularly interested in that section of the article. What I found interesting is that figure 10 shows that as you jack up the independence of the machine, the level of driver distraction accordingly increases. Adaptave Cruise Control (ACC) shows a significantly higher percentage of shorter-duration off-road glances than Limited-Ability Autonomous Driving Systems (LAADS). Additionally, countermeasures help to alleviate some but not all of that increase in distraction. Importantly, this is coupled with the point that the duration in which drivers have to react to most impending is under 3 seconds. This may seem obvious but it's a critical set of data to help objectively demonstrate the risks involved with losing or even reducing alertness. It goes on to say that Tesla has addressed the risks of mode confusion, distraction, etc. and has implemented solutions to address this unreasonable risk, which they in turn define as abuse that is reasonably forseeable. In this, they're talking about the reasonably forseeable risk of eg: the driver not understanding if they're in autopilot or not. It goes on to mention that Tesla has also changed its driver monitoring strategy to promote driver attention, which I take to mean detecting hands on the steering wheel. Either way, Telsa's main approach to dealing with driver alertness by testing for hands on the steering wheel. My concern is that this doesn't consider the alertness of the driver to their surroundings, particularly other vehicles that may be approaching them or the process of anticipating hazards (approaching an intersecion where there's a blind corner and adjusting focus to pay more attention to what may come from it, for example). I don't see how Tesla's countermeasures address this. The physical act of manually driving causes drivers to maintain alertness not only to where they're going, but also the situational alertness of what's around their vehicle. Specifically, it's the process of random actions that requires an taking input, making a decision and executing the appropriate action that maintains this alertness. If the driver isn't having to make those random decisions and take action then their alertness drops. Autopilot, even with hands on the wheel, eliminates much of that random decision-making and reacting. When you drive, you mentally note the vehicle over your shoulder that is in the lane next to you, and subconsciously consider that they may do something insane. You consider those blind corners as you approach them and that vehicles may spontaneously appear out from them. You see a truck on the road which is approaching a bend and give it a wider berth because its centre throw may cause it to cut the corner into your lane slightly. These are all tasks that you do, that you may not do as well or at all when autopilot is steering, because you are not as engaged with the driving process. Critically, I don't see how ensuring hands are on the steering wheel causes these alertness tasks to continue as frequently as manual steering. The driver may be in the physical location to quickly take over, but they may not be in the mental location to do so. This is the major issue I have with the rapid autopilot development based on my experience in related areas where maintaining situational alertness proved to be very difficult when the person was engaged with only a limited scope of requirements to prove their conscious presence. I feel like the report doesn't really drill in to this as much as it needs to. It begins to touch on it around Figure 10 but sort of hand-waves it away saying 'Telsa considered it discharged their responsibility to make sure drivers stay focused by implementing countermeasures', but I believe it's more nuanced than that. It investigates the extent to which Tesla's system is good at ensuring drivers are physically present (that is, their hands aren't on the passenger seat making breakfast) but it doesn't really look at the mental presence that delivers situational alertness. That mental alertness is the major sticking point for me. I don't really have a solution beyond "drive manually" which isn't reasonable, because this technology is here to stay and will continue to grow, but it's why I've always been bearish about the rapid pace of rollout of these driverless technologies, particuarly when advertised as 'beta'. As I've said before, no amount of disclaimer and 'hey, you should do this' really changes how drivers behave once the equipment is placed in their hands.

ridiculous_fish 10y ago

This means that autopilot must be engaged at least 40% of the time (Amdahl's law!). Tesla owners, is that realistic?

lz400 10y ago

I think there's sometimes a lot of marketing and hand waving in this type of argument "crashes go down with autopilot". Most car accidents are caused by drunk or old people, and they drag the average up. if you tell me a Tesla autopilot beats a drunk guy it won't surprise anyone. Now as a non-drunk, young(ish) driver but experienced and careful, my statistics don't look anything like the average, they look at lot better. You have to convince this demographic, not beat the averages otherwise it's not rational for me to buy the feature. I'm guessing this goal post is a lot harder.

phinnaeus 10y ago
I'm confused. You think young(ish) people need to be convinced to buy semi-autonomous vehicles? If they can afford it, why wouldn't they buy it? People love things that make their life more convenient. In my experience, its the older people that are somehow scared of the concept of a self-driving car and would need to be convinced.
- lz400 10y ago
  Because of security. I wouldn't buy an autonomous car that has an accident rate worse than my demographic. A car that has an accident rate better than average is not good enough. I hope that explains my point.
mfgrimm 10y ago
I suppose the best comparison would be to compare non-Tesla owners of the same demographic. Are accident rates correlated to SES? Do people who buy luxury sedans crash less? Just a guess: probably. Still, I know I drive stupidly sometimes, and autopilot might help.
- lz400 10y ago
  Yes exactly, it would be interesting to see fair comparisons of people of the exact same profile (both driver and car) tesla vs others. I don't like that so many people seem happy enough to compare autopilot against the total average of accidents in the road and say "they save lives". When you are going to buy the car you want to know if it's going to be better than you.

em3rgent0rdr 10y ago

I only want to use open-source code for something that my life depends on. That way it can be open to any one to inspect some can independently determine if the code is behaving as desired.

brighton36 10y ago

The truck was visible for at least 7 seconds prior to the crash in the full report - another article here: http://arstechnica.com/cars/2017/01/after-fatal-tesla-crash-probe-us-regulators-conclude-theres-no-need-for-recall/ http://arstechnica.com/cars/2017/01/after-fatal-tesla-crash-... Strangely enough, giving more people autopilot would probably be better than letting people drive. I think Tesla's picked the right time to enable it, since the cross-over point between autopilots being better than humans in general use cases has been reached. Call it a beta if you want, but it's a pretty damn promising beta.