16 ms·

Tesla crash in September showed similarities to fatal Mountain View accident

303 points by jijojv 9y ago

jijojv 9y ago
Week after the fatal accident Musk continues to promote Autopilot as safe by retweeting stuff like https://twitter.com/Teslarati/status/980476745106231297 https://twitter.com/Teslarati/status/980476745106231297 Tesla Model S navigates one of Vienna’s ‘crappiest’ roads on Autopilot IIHS '17 study puts Autopilot at only reducing 40% accidents which is the same as their '16 study which had the same 40% reduction for any car with Auto Emergency Braking.
- otalp 9y ago
  What does Musk have to do for him to be investigated for misleading consumers or fraud?
  - aaron-lebo 9y ago
    He definitely has a penchant for exaggeration if not outright dishonesty. It bothers me that people respect someone who can say so many blatantly false things and that an engineer can be so loose with the truth. That's his shtick though, promise unbelievable things, then when you get halfway there, people don't care you were wrong and/or lying. He understands how to market himself and his ventures. My guess is he's got too many fans and money behind him. If things ever go truly south, that's when they'll really go after him, because people really care about their money.
    - buvanshak 9y ago
      >It bothers me that people respect someone who can say so many blatantly false things and that an engineer can be so loose with the truth. Why does it bother you though. "People" are like that, right? If you know your target demographics, and thus know what strings to pull, then you can make your target do anything. In this case, invest, financially and emotionally in what ever he is doing. Pretty basic advertising. Elon Musk's target demographics is the "tech" inclined (administrators and layman alike. America is the greatest country in the world!, Yay!), and what he is doing is to exactly cater to that target. What you see is just a manifestation of that. Again, pretty basic advertising/marketing tactics. But I think it is for the first time we are seeing PR tactics that are tailored to the nerds/techies. That it why this seems so effective.. In someway, I think Elon Musk is to tech what Christopher Nolan is to movies..
      - aaron-lebo 9y ago
        You're asking why exaggerated claims which have led to deaths are distasteful? It might be basic advertising, but that doesn't make it right nor effective. If Musk would just do what he has without all the exaggerated promises, why's that not good enough? The claims make him look worse, not better. Denying reality is great until it comes crashing down. Just a few examples. Forgive this first title. The important part is the quote: http://www.businessinsider.com/musk-epic-failure-2016-5?op=1 http://www.businessinsider.com/musk-epic-failure-2016-5?op=1 Tesla reported first-quarter earnings last week, and while they were better than Wall Street expected, the earthshaking news that emerged was that Musk is taking the carmaker in a new strategic direction. Tesla, which delivered about 50,000 vehicles last year, is aiming to deliver 500,000 in 2018. The production target isn't new, but the timeline is. Tesla had previously said it would build half a million cars a year by 2020. This did not happen, which led him to Tweet last July that they could deliver 20k cars per month by Dec. This did not happen either. Investors were relieved they managed to hit 2k per week this past week (this is down from the 2.5k they were targeting). https://twitter.com/elonmusk/status/881757617416056832 https://twitter.com/elonmusk/status/881757617416056832 https://www.marketwatch.com/story/tesla-stock-jumps-after-production-misses-but-liquidity-concerns-addressed-2018-04-03 https://www.marketwatch.com/story/tesla-stock-jumps-after-pr... He's got a shaky attachment to the truth. I don't know if he really believes himself (this is called delusion, usually) or he's a liar. Either way it's kind of odd for a guy who doesn't need to exaggerate to be impressive. Not sure what it has to do with Nolan. He's never promised a 20 hour movie.
  - qaq 9y ago
    Break the law? One surprising thing to me there is a ton of media attention on the SV right now (almost all negative) and that does not look like a coincidence. I disapprove of what FB is doing but on the scale of problems that Big Pharma, Health Insurance, big banks are causing thats not even a blip and yet most media attention and good amount of public outrage is fairly skillfully directed toward tech and SV.
    - Piskvorrr 9y ago
      Or perhaps that SV is vocal, excessively so: promises miracles, yesterday, at will, almost for free. In other words, massively overpromises, then underdelivers - that amounts to painting a huge sign "KICK ME" on its collective butt. While other enterprises might do that as well, even to a criminal extent, they do not brag about it. This might be all the explanation that's needed, no conspiracy necessary.
      - qaq 9y ago
        What miracles was FB promising? (Again not a fan of FB).
        Piskvorrr 9y ago
        I would guess "we'll give you social network, not a total-surveillance tool" would qualify. I don't think the second part would have been a reasonable expectation.
        qaq 9y ago
        Would you point at FB marketing that has this message?
        Piskvorrr 9y ago
        I don't see any (although "FB is free and always will be" has plenty of wiggle room). Also, I don't see any explicit mention that Tesla's Autopilot is actually an autonomous Level 5 system - and yet people assume that. Are you saying that people should only be angry about things they were promised explicitly? (If so, I would agree - but 1. people are not 100% rational, 2. expectations are important for PR, even if not legally binding. Going from "SV doesn't put this in big bold letters on the front page" to "it's a conspiracy against SV, not because they're an easier media target" is a bit too much of a leap for me.)
- oldgradstudent 9y ago
  Wasn't the IIHS study relevant to the old Mobileye-based Autopilot? Any reason to assume it applies to Tesla's in-house Autopilot?
  - jijojv 9y ago
    That may be correct as IIHS took few months and concluded in Jan'17. People are still waiting for in-house Autopilot aka AP2 to reach feature parity with mobileey AP1 if you follow https://forums.tesla.com/forum/forums/enhanced-ap-hw2-qa-firmware-2018104-81-photos-notes https://forums.tesla.com/forum/forums/enhanced-ap-hw2-qa-fir... so i'd suspect AP2 to be worse-off depending on when it gets tested.
- ilitirit 9y ago
  > at only reducing 40% accidents Unless I'm missing something, 40% is actually a good result (given what they're trying to measure). However, I think the more important statistic would be regarding how many injuries/fatalities does it reduce vs cause. Are there any studies on this?
bobsil1 9y ago
Gore point stripe faded: https://imgur.com/a/xHq57 https://imgur.com/a/xHq57 Street View: https://www.google.com/maps/@37.6346515,-122.104109,3a,75y,84.96h,99.37t/data=!3m6!1e1!3m4!1s4LqOilvG6oJ1oNApEwLi6A!2e0!7i13312!8i6656 https://www.google.com/maps/@37.6346515,-122.104109,3a,75y,8...
- fossuser 9y ago
  Can somebody just go out there and paint the road? There have been at least two other video near collisions of people driving on that section - it seems dangerous.
  - pen2l 9y ago
    Holy shit that’s a good idea, we should just fucking do it ourselves. This reminds me when I was living in the Midwest and there was this one spot where I had to do a u turn in my commute which was very dangerous because of some tree branches obstructing view of oncoming traffic. One day I almost got into an accident and since it shook me quite a lot, I wrote an email to the city that the tree branch needed to be trimmed so I could see better. Sure enough I got an email back a few days later from a woman saying they “understand my frustrations” or something like that. I thought I did my part and went on with my day. Fast forward about a month, and I almost get in an accident at the same spot, man I was fucking furious that my communication with city was just “ohh we understand” —- instead of employing a sea of men and women telling me that they “understand” why could they have just not just trimmed the tree and improved safety. If I was still living there I think I would have just gone with my chainsaw and done it myself.
    - tlrobinson 9y ago
      > Holy shit that’s a good idea, we should just fucking do it ourselves. Seems to work for Portland Anarchist Road Care: https://www.citylab.com/equity/2017/03/portland-anarchists-want-to-fix-your-streets-potholes/519588/ https://www.citylab.com/equity/2017/03/portland-anarchists-w...
      - 9y ago
brandmeyer 9y ago
There certainly do appear to be some similarities. But without the car's logs, it isn't possible for anyone reading this forum to know for sure. That's what the NTSB is for.
ChuckMcM 9y ago
I have now talked with two people who have autopilot in their model S's and both said the problem with autopilot is that it is "too good". Specifically it works completely correctly like 80 - 90% of the time and as a result, if you use it a lot, you start to expect it to be working and not fail. If it beeps then to tell you to take over, you have to mentally get back into the situational awareness of the road and then decide what to do about it. If that lag time is longer then the time the car has given you to respond, you would likely crash. Makes me wonder how this gets resolved without jumping through the gap to 'perfect' autopilot.
- doe88 9y ago
  This. I don't know I feel ambivalent about autopilot right now, as most on HN I always crave for new technologies, but the fact that something very dangerous can be very good most of the time and very bad occasionally make me weary of using it altogether. I know that's irrational because most often then not it probably saves you from your own mistakes. Also, if Tesla intends to shield itself behind a beta status for their autopilot system until it is perfect, I think this beta status will remain even longer then the time GMail was in beta. At least, in this case they should own this problem and somewhat hardcode a meaningful warning at such intersections or do something.
  - carlivar 9y ago
    It's not irrational at all. I drive our family 2018 Honda Odyssey, and it's already "saving me from my own mistakes" by beeping angrily when I signal for a lane change and there's someone in my blind spot. Works really well. It also has a notification light on each side of the car to indicate blind spot object presence, so I have to miss that first. The question seems to be which is better: 1) A car very intelligently helping a human drive better. 2) Or a car mostly driving well itself but needing humans to very rarely override fatal crashes. I prefer 1) I'm not sure there are any studies comparing these things, since "human driving but lots of extra tech help" is harder to quantify and doesn't have all the data going back to the mothership like Tesla.
    - 9y ago
brennankreiman 9y ago
Biggest thing I gathered was the public safety barrier was damaged from a previous crash, 12 days earlier, so the driver basically hit a hard wall. This has more implications for the US’s infrastructure than Tesla’s safety IMO.
- danso 9y ago
  via an earlier story: http://abc7news.com/automotive/exclusive-i-team-investigates-why-caltrans-didnt-fix-safety-barrier-before-tesla-driver-died-there/3280399/ http://abc7news.com/automotive/exclusive-i-team-investigates... > CalTrans finally sent ABC7 News a statement later Thursday"confirming" it's their policy to fix broken safety barriers within seven days, or five business days ... but storms delayed the work. As Dan Noyes reported Wednesday, the family told me they believe Walter Huang would have survived, if CalTrans fixed that safety barrier in a timely fashion.
  - kwhitefoot 9y ago
    Until the barrier is fixed that part of the road should have some sort of marking to indicate that there is an extra danger there, that's what cones were invented for.
    - dannyw 9y ago
      I'm sure a hitting series of traffic cones taking up the space where the safety barrier was could have alerted even a modestly distracted driver to brake.
- telmnstr 9y ago
  Plus the genius driver had a number of issues with autopilot in the same spot. He had complaints in with the Tesla dealership that his car tried to steer him into the barrier in that location, prior I believe. Something like 7 complaints filed? Maybe he was trying to get out of his loan but didn't know the impact absorption barrier had been removed.
11thEarlOfMar 9y ago
Vehicles with autonomous features will save lives. Fewer people will be injured or killed in cars that have these features. However, some people will still be injured or killed, and those people will comprise a different set than would otherwise have been harmed. They or their families will seek restitution and hence, the way forward will be determined by the courts.
- Declanomous 9y ago
  The thing is that most driving deaths don't occur on the highway with dry-well lit roads, and current vehicles can't drive for shit in bad weather. So they shouldn't have autopilot on at all, they should just have the collision avoidance features on.
abalone 9y ago
I wonder if this will be the case that really tests Tesla's blanket legal defense in court. (That accidents are due to less than "fully attentive" drivers.) The reaction time needed in that video to correct for the autopilot is downright scary. It's a situation where if you were piloting yourself you're a lot more likely to have mentally mapped that you continue in a straight line at that point, so listing to the left is going to feel WRONG even when suddenly blinded by the sun. But if autopilot takes you there, and you're "lizard brain level" paying attention but didn't have that mental navigation map in your head, and the sun's in your eyes, you're going to take a few seconds to try to figure out how to correct so as not to make things worse. That's enough to hit the gore point in that video.
oldgradstudent 9y ago
Tesla claimed[1] that: > Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2015 and roughly 20,000 times since just the beginning of the year, and there has never been an accident that we know of. There are over 200 successful Autopilot trips per day on this exact stretch of road. Which might be technically true, but conveniently ignores other, very similar crashes. [1] https://www.tesla.com/blog/what-we-know-about-last-weeks-accident https://www.tesla.com/blog/what-we-know-about-last-weeks-acc...
- Shivetya 9y ago
  my question which is the same as others, does Tesla register events where the driver overrides the system? If they register it how do they account for it? if they are not even considering that the driver overrode the system because it was going to crash then all their claims are suspect.
  - quotemstr 9y ago
    Why? The car isn't driving by itself. What matters is whether Autopilot is a net safety improvement, which it currently appears to be. Why would you add the weird requirement to the system that it be better without human input at all?
- dannyw 9y ago
  It took people a day to reproduce the crash and end up in a near-collision: https://www.youtube.com/watch?v=6QCF8tVqM3I https://www.youtube.com/watch?v=6QCF8tVqM3I
newnewpdro 9y ago
> Wednesdsay, a Tesla spokeswoman told the I-Team, "Autopilot is intended for use only with a fully attentive driver," and that it "does not prevent all accidents - such a standard would be impossible - but it makes them much less likely to occur." Actively veering towards stationary barriers in otherwise perfectly safe conditions is not what I would describe as reducing the likelihood of accidents.
- lsaferite 9y ago
  In this one specific case sure, the AP was confused about the lane markings and followed the wrong path. But in plenty of other cases the AP is responsible for preventing accidents (no, I have no link to back up my statement). Assuming that is the case, then that is what I would describe as reducing the likelihood of accidents. You cannot focus on a specific accident (or even a handful of accidents) and use that to refute the systemic benefits.
nodesocket 9y ago
Just my opinion, but I don't think the local bay area ABC I-TEAM news can be trusted as a realiabe primary source. Seems like they are "cashing in" on a national story.
- danso 9y ago
  Has anything in their past exclusives on this story been disputed? Victim who died in Tesla crash had complained about Autopilot: http://abc7news.com/automotive/i-team-exclusive-victim-who-died-in-tesla-crash-had-complained-about-autopilot/3275600/ http://abc7news.com/automotive/i-team-exclusive-victim-who-d... I-Team investigates why CalTrans didn't fix safety barrier before Tesla driver died there: http://abc7news.com/automotive/exclusive-i-team-investigates-why-caltrans-didnt-fix-safety-barrier-before-tesla-driver-died-there/3280399/ http://abc7news.com/automotive/exclusive-i-team-investigates...
- unityByFreedom 9y ago
  It's literally their job to report the news. What has been unreliable about their reporting?
TeeWEE 9y ago
When i look at the video, I would probably drive wrongly for the first few seconds: The road lining is not there anymore, causing you to make mistakes. But people can correct quickly when there is an anomaly. Weird that autopilot didnt see the divider.
danso 9y ago
Currently, the Tesla autopilot page has this as its top headline: > Full Self-Driving Hardware on All Cars Followed by this: > All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver. http://web.archive.org/web/20180323054727/https://www.tesla.com/autopilot http://web.archive.org/web/20180323054727/https://www.tesla.... Getting past the arguments of whether Tesla should be calling this "Autopilot", is its claims about hardware true? Is the failure to detect stationary objects like a firetruck or a gore point solely as a result of currently-inadequate software? Because it doesn't sound like the sensor suite is anywhere near the AVs of Waymo or Uber.
- Shank 9y ago
  > Getting past the arguments of whether Tesla should be calling this "Autopilot", is its claims about hardware true? Tesla actually hasn't called that Autopilot -- they refer to it as "full self driving" and not in the same category. But...is it true? It's vaporware right now, so we can't really say. Obviously, in a purely hypothetical landscape, even with cameras you should be able to emulate what humans can do and have 100% attentiveness. That is to say, humans don't have lidar and radar, yet humans do drive. So, if you can achieve human levels of perfection and have 100% attentiveness, it should be possible to beat out crash statistics based on the fact that many are caused by failure to respond on the human side accurately or fast enough. This is all hypothetical, though. All of it comes down to computer vision/software, which is really hard to do. It's possible, but not yet done.
  - miskin 9y ago
    "Autopilot" is the name of the page that danso referred to, is in the title of that page and is used on the page 9 times. How would you possibly not recognize this as name of the feature?
  - carlmr 9y ago
    >Obviously, in a purely hypothetical landscape, even with cameras you should be able to emulate what humans can do and have 100% attentiveness. That is to say, humans don't have lidar and radar, yet humans do drive. Which camera has the same dynamic range as the human eye?
jv22222 9y ago
I am a huge fan of Tesla. What I am noticing right now is that if I were able to afford a Tesla today, there is no way I would use Autopilot. I will still buy a Tesla if I can ever afford it, I'll just leave autopilot well alone. I don't know what gets me back from this mental place that seems to be firmly set in my mind now.
- JimmaDaRustla 9y ago
  Agreed - I might test using it, but I would ensure that I have 100% mental focus. Now I'm wondering, if I have to work hard to pay attention that a computer is operating correctly, what advantage am I gaining by using it? AutoPilot is a human assisted utility. I think many people here, and in general, expect too much of it and are being too critical when there is an incident. I do think Tesla needs to communicate better what AutoPilot is and isn't because we've now lost lives due to operator negligence, and by negligence, I mean the operator is still responsible for the vehicle despite relinquishing control to a computer.
- vasilipupkin 9y ago
  Just got a model S after driving my toyota camry for the last 10 years. Autopilot is a fun toy, but I can't understand for the life of me why people actually use it like its a real thing. I have it off on mine.
bra-ket 9y ago
this AI is dumb
spullara 9y ago
If you haven't driven a Tesla under autopilot please stop commenting about it. You really don't know what you are talking about.
bobsil1 9y ago
Btw linked accident was for a Model S with Tesla autopilot 1. Can tell because no camera on silver blinker assembly. They're now on 2.5.
- ino 9y ago
  Are you talking about the second video in the story, in which the car veers towards the divider?
  - bobsil1 9y ago
    Photos for the crash on 92/880 in September.
- josefresco 9y ago
  Can autopilot v1 cars be upgraded to v2.5? I'm assuming since you mentioned a camera it's not just software.
  - ohitsdom 9y ago
    No, Elon has tweeted a few times that it would be too extensive of an upgrade process. https://twitter.com/elonmusk/status/789007996038721536?lang=en https://twitter.com/elonmusk/status/789007996038721536?lang=...
Animats 9y ago
Come to think of it, whatever happened to Tesla's self-driving effort? They claimed to be shipping Model 3 cars with all the hardware needed for self driving (not including a LIDAR). They produced videos. Most of this was 1-2 years ago. But they never got to deployment, or even to the point of demoing it for car magazine writers.
- lozenge 9y ago
  They had a falling out with their software supplier Mobileye which set them back a year or more.
  - B1FF_PSUVM 9y ago
    I gathered that Mobileye found that the use of the software was "outside the box" they were willing to risk ...
  - BinaryIdiot 9y ago
    Was it really that much of a setback? It was my understanding they used some of their own software in combination and had already planned to take over the role Mobileye was serving. Maybe I bought too much into their PR spin but it sounded as if it was barely a delay.
    - stetrain 9y ago
      From reading owner discussions it seems like the general consensus is that they only just recently reached parity with (and slightly surpassed) their previous Mobileye implementation with their own software/sensor stack.
  - Piskvorrr 9y ago
    Did I just read "they found out that tech is not actually ready to fulfill what its marketing promised", or is that something else?
    - Animats 9y ago
kalal 9y ago
When the self driving revolution started, conservative people claimed that one accident would stop the whole business. Well there are many accidents already and people are still willing to risk. One reason may be our tendency to trust something which we do not understand. We feel that if the feature is already in the car, it must be safe. Well it is not. And even worse, it is driven more by marketing and money than by safety regulations.
antirez 9y ago
What is happening was clear to many since the start: Tesla incarnates the behavior of his founder, exaggerating the technology that is really available right now, and selling it as a product without the premise for it to be safe. A product that kinda drives your car but sometimes fails, and requires you to pay attention, is simply a crash that we are waiting to happen. And don't get trapped by the "data driven" analysis, like "yeah but it's a lot safer than humans" because there are at least three problems with this statement: 1. Ethical. One thing is that you do a stupid thing and die. Another is that a technology fails in trivial circumstances that in theory are avoidable. A doctor can make also errors, but a medical device is required to be very safe and reliable in what it does. 2. Of wrong comparisons: you should check the autopilot against a rested and concentrated driver that drives slowly and with a lot of care. Since in the statistics otherwise you do not account for the fact that when you drive, you are in control, and you can decide your level of safety. Such a diligent driver that goes to crash because of an immature software is terrible. 3. And lack of data: AFAIK there is not even enough data publicly available that tells us what is the crash rate of Teslas with auto-pilot enabled vs Teslas without auto pilot enabled, per kilometer in the same street conditions. That is, you need to compare only in the same streets where the auto pilot is able to drive. Autopilot will rule the world eventually and we will be free of using the time differently (even if this was possible 100 years ago already stressing on public transportation instead of spending the money to have a car each one... this is sad. Fortunately in central Europe this happened to a degree, check for instance North Italy, France, ...). But till it is not ready, to plug it as a feature in a so immature way just to have an advantage over competitors, is terrible business practice.
- kalal 9y ago
  4. This statistics must be conditioned. Not all drivers are the same. Some are careful, some are not. So in fact, the careful drivers will make their life miserable by using autopilot.
- sschueller 9y ago
  How can anyone take a company serios that sells features such as "Bio-Weapon Defense Mode". It almost sounds like snake oil and not far from something like the ADE 651. [1] https://en.wikipedia.org/wiki/ADE_651 https://en.wikipedia.org/wiki/ADE_651
hugh4life 9y ago
Self driving cars have absolutely no business being any road/lanes that is not purposely made/reserved for self driving cars. Period. Self driving cars need an ecosystem made for self driving cars. It's frankly madness to think otherwise.
- neolefty 9y ago
  What if they improve safety on shared roads but aren't perfect?
  - dannyw 9y ago
    If they are unable to avoid blatantly obvious yellow and black barriers / safety cushions and their algorithm is a simple "follow the whitest white line", then they should not be on shared roads.
    - neolefty 9y ago
      Again, they mostly do. Same with human drivers. By the same logic, should nobody drive?
- igammarays 9y ago
  Absolutely. It shouldn't even be that hard, with the billions that are being thrown at the tech. They should be spending time and money to upgrade specific highways and certify specific routes as "safe for automation", rather than trying to test their cars on the entire North American road network with millions of crazy edge cases. Over time, these certified routes will grow into their own network.
deleted 9y ago
[deleted]
josefresco 9y ago
Auto-pilot will put massive pressure on states and municipalities to improve road maintenance - specifically lane indicators and fixes to "peculiar" roads/intersections etc. This will increase budgets, which will in turn create massive opposition to self-driving autos by those not wanting to pay more taxes for other people's fancy cars. I travel/drive a good deal in the summer and for a few years now have silently (mostly) muttered to myself that if auto-pilot cars are going to use "painted lines" to navigate - they're "gonna have a bad time". Poorly painted lines aside, think of how many "wtf" moments you have on the roads around your town - just down the road from my house is a road that just... ends (due to poor planning years ago). It's obvious to human drivers what to do (you're supposed to exit the paved road and drive over a hump to the dirt road that continues), but for auto-pilot? Not so much unless they make use of "data sharing" that would reveal the human solution to this screwed up piece of road.
- deleted 9y ago
  [deleted]
riffic 9y ago
The Associated Press stylebook calls for reporters to avoid using the term "Accident": https://www.transalt.org/news/releases/9545 https://www.transalt.org/news/releases/9545
paul7986 9y ago
I'm going to wait five to ten years before I get my robot car. Let all those early adopters be the guinea pigs we need to reach that level of robot cars being safer then human drivers. It seems tons of upper middle class to wealthy people are lining up to pay dearly with their wallets and lives to be a billionaire's crash test dummy.
- awakeasleep 9y ago
  Aren't we all the crash test dummies if we're sharing the road with them?
  - WhompingWindows 9y ago
    We are nothing like crash test dummies. Crash test dummies are intentionally slammed into barriers at 60+ mph. Testing of vehicle safety features, and safety features of many products, is most often done in mass markets. Automatic safety features are being rolled out in essentially every single auto brand. Automatic emergency braking (AEB) is set to be standard in numerous makes in a couple years. Lane control, lane reminders, blindspot detectors, these features are all being rolled out on the roads, to varying degrees of success. This is a less-safe version of the way drugs are brought to market. First, small-scale tests with cells only, then mice, then healthy people, then target patients, and finally post-market surveillance. Cars indeed have a MUCH less regulated rollout for their safety features, yet to say we are dummies in their tests is not even close to reality. While drug testing may be more stringent, we also don't have 30-40k people a year dying from "not taking a drug" accidents, which is the situation on our roads today. Ultimately, it comes down to lives saved. Every time you delay a safety feature, you have to ask the question: is this delay going to lead to loss of life or saving a life? That's the moral quandary the automakers are in, and it seems they go with "release it" much more often than not. That will lead to some EXTREMELY well publicized accidents, and yet we will NOT hear of the countless cases where everything went nominally, and who's to say precisely how many accidents were prevented?
    - paul7986 9y ago
      So are you rushing out to be a Musk guinea pig or already are?
dannyw 9y ago
I want to ask: why is Tesla's advanced suite of cameras and ultrasonic sensors unable to detect a big, yellow and black stationary barrier? In combination with chevron markings that mean "never ever drive here". Either should be a globally overriding signal to emergency break or turn, not "keep following the white line". If the answer is "during morning light, it doesn't see it", then they need to take the feature offline until the car is fitted with an appropriate LIDAR; or other technology that can detect stationary obstacles.
- jonknee 9y ago
  I've seen a lot of Tesla apologizers talk about how it's such a bad design, but people seem to be able to dodge it successfully and so should Autopilot. It seems like this feature should require hands on the wheel until it has enough training data that it doesn't do things like smash you into an obvious obstacle in whatever lighting conditions are possible at any time.
  - pkulak 9y ago
    > this feature should require hands on the wheel It does.
    - jonknee 9y ago
      Yet, in this fatal crash: > "the driver’s hands were not detected on the wheel for six seconds prior to the collision" https://www.tesla.com/blog/update-last-week%E2%80%99s-accident https://www.tesla.com/blog/update-last-week%E2%80%99s-accide...
      - ricardobeat 9y ago
        What is the car supposed to do? Have robot arms that force your hands? It will actually slow down and stop if you continue not holding the wheel.
  - rasz 9y ago
    > but people seem to be able to dodge it successfully are they? so why was the barrier crushed in the first place before the crash? and why was it designed to be easily rebuild? its almost like someone knew people will have trouble and will eventually drive into it.
w_t_payne 9y ago
My former employer makes a camera-based lane departure warning system. Although I was not involved with its development, I do know that the amount of testing that they do on these systems is very extensive (hundreds of people involved in the test process). I would be very interested in seeing a side-by side test of the LDW systems of major manufacturers. This might give us an idea if the problem is fundamental to the technology or is simply a deficiency in the engineering approach of one (or some) manufacturer(s).
MPSimmons 9y ago
I have to believe that, at this point, given the markings on the road, that California Dept of Highways (or whoever is responsible for lane markings) is just trying to kill people.
YeGoblynQueenne 9y ago
>> The company tells Dan Noyes they have made it clear -- Autopilot is a driver assistance system that requires you to pay attention to the road at all times. The question is why it is Tesla, in particular, that has to remind its customers that they have to actually pay attention when they drive its cars. Is it possible the company somehow managed to convince people that its cars are equipped with self-driving software, rather than a "driver assistance system"?
StefanFrost 9y ago
So aside from the fact that I can not afford these cars I would also give these at least 10 years to mature and get further than (what really does seem) at least the beta. Right at the start I was very much wondering how these even allowed this feature on the road. Just from a regulatory point of view. It very much sounded and looked like a complete auto-pilot. That said, people go through a drivers test, get their license and then are allowed to drive any vehicle really. Can you just solve this by giving the driver assistant a test and seeing if it passes? This needs some kind of filter or regulator to make sure it is safe.
gfodor 9y ago
For all those who are clamoring for autopilot to go back into the lab consider this: I have autopilot, and I use it the way people are instructed to. I consider it a companion to my driving, a backup co-driver that has better reflexes. I am in control, and mentally steer the vehicle with my hands on the wheel with the machine. We are driving the car together and I overrule the machine when we disagree. Often the machine catches things before I am able to react. I am 100% convinced that my family is safer with "us" driving them then just me driving them solo. What do you say to people like me, who, if you took the feature away, would be forced to reduce their safety? It's easy to call for its removal when you only focus on the subset of people who will have their safety increased by removing it (ie, the people, often through no fault of their own, who come to rely upon it more than it is designed for.) What about everyone else that currently benefits from it? Will you force them to go back to driving "solo" to protect those people? It's not so simple as "Tesla is reckless by releasing this" -- this is providing real safety to people who are using it the way it is intended. Also, the release of autopilot as-is also makes it possible to accelerate the development of full autonomy, which will save lives that otherwise would be lost to accidents if it were to be created later in time. Any analysis which does not account for this dynamic is missing an important piece of the ethics of removing it. The main question that is worth asking is if there is more Tesla could do to ensure autopilot is used as a co-driver. The name is a poor choice to start.
- jakobegger 9y ago
  Why not the other way around? You do the driving, and the computer is the backup in case you mess up? Humans are extremely bad at monitoring monotonic things, even if we are trained to do them. That‘s why trains have elaborate dead man switches to ensure the operator is paying attention. From these accidents it‘s clear that Teslas aren‘t good enough at making sure the driver is paying attention. And it‘s also pretty clear that these autopilot victims weren‘t aware how distracted they really were. I‘m pretty sure they thought they had everything under control, right up to the point when their car did something stupid.
  - unityByFreedom 9y ago
    > Why not the other way around? You do the driving, and the computer is the backup in case you mess up? Unfortunately having the computer takeover has the same issues. If it thinks you're following the wrong lane it could choose to steer you into a barrier, for example. Chris Urmson's team at Google ran user tests at the start of their autonomous vehicle program and found that users misbehave [1] when behind the wheel of semi-autonomous vehicles, despite having been given instruction to pay attention. We're seeing that scenario play out in slow motion in Teslas and perhaps other driver-assistance programs that get less press. [1] https://youtu.be/Uj-rK8V-rik?t=14m7s https://youtu.be/Uj-rK8V-rik?t=14m7s
Jabbles 9y ago
To misquote Elon Musk: The advantage of getting somewhere [without having to steer] will be negatively affected if "but also, you might die" is on the ticket. https://www.reddit.com/r/space/comments/76e79c/i_am_elon_musk_ask_me_anything_about_bfr/dodes6v/ https://www.reddit.com/r/space/comments/76e79c/i_am_elon_mus...
pjkundert 9y ago
The root problem (as described by Taleb in "Skin In The Game", is: not all statistical probabilities are "ergodic" -- have no risk of Ruin or Extinction. The problem with saying that "our automatic driving system is much better than the average driver" and thus you should be good with allowing our system to drive, is that you are not exposed -- serially -- to the whims of some average driver. One "average driver" mistake and you're dead. No replays. Therefore, the compound "odds" are INCOMPUTABLE. Just like being paid $1,000,000 per trigger-pull to play Russian Roulette: you are not earning an average of $833,333.33 per try! On "average", over a pool of random 1-time players, sure. But you? You're guaranteed to be dead in a small number of plays. Non-ergodic and ergodic probabilities are incomparable. So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable. I live in a part of the world where even cruise control is unusable for 6 months of the year. Automatic driving systems? Not even thinkable, except for perhaps a few days of the year, on a few simple portions of a few roads...
- dragonwriter 9y ago
  > So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable. The risks from human drivers are also non-ergodic, so that suggests that if automatic driving systems are even minimally better than the average driver you should demand that every other driver be replaced with them without delay, because you will die sooner with human drivers on the road. Of course, you won't want to be replaced unless the automatic driver is better than you, and human drivers all think they are far above average. And that’s the real problem.
  - pjkundert 9y ago
    The real problem is that, yes, human drivers ARE better than computers — for some unknown subset of driving. It’s that small subset of lethal mistakes you avoid by not letting your computer drive that makes all the difference. The computer must not just be better on average — people must know that it’s better overall, across all possible weird events that a human just might throw a Hail-Mary and survive!

303 points by jijojv 9y ago

Week after the fatal accident Musk continues to promote Autopilot as safe by retweeting stuff like https://twitter.com/Teslarati/status/980476745106231297 https://twitter.com/Teslarati/status/980476745106231297 Tesla Model S navigates one of Vienna’s ‘crappiest’ roads on Autopilot IIHS '17 study puts Autopilot at only reducing 40% accidents which is the same as their '16 study which had the same 40% reduction for any car with Auto Emergency Braking.

otalp 9y ago
What does Musk have to do for him to be investigated for misleading consumers or fraud?
- aaron-lebo 9y ago
  He definitely has a penchant for exaggeration if not outright dishonesty. It bothers me that people respect someone who can say so many blatantly false things and that an engineer can be so loose with the truth. That's his shtick though, promise unbelievable things, then when you get halfway there, people don't care you were wrong and/or lying. He understands how to market himself and his ventures. My guess is he's got too many fans and money behind him. If things ever go truly south, that's when they'll really go after him, because people really care about their money.
  - buvanshak 9y ago
    >It bothers me that people respect someone who can say so many blatantly false things and that an engineer can be so loose with the truth. Why does it bother you though. "People" are like that, right? If you know your target demographics, and thus know what strings to pull, then you can make your target do anything. In this case, invest, financially and emotionally in what ever he is doing. Pretty basic advertising. Elon Musk's target demographics is the "tech" inclined (administrators and layman alike. America is the greatest country in the world!, Yay!), and what he is doing is to exactly cater to that target. What you see is just a manifestation of that. Again, pretty basic advertising/marketing tactics. But I think it is for the first time we are seeing PR tactics that are tailored to the nerds/techies. That it why this seems so effective.. In someway, I think Elon Musk is to tech what Christopher Nolan is to movies..
    - aaron-lebo 9y ago
      You're asking why exaggerated claims which have led to deaths are distasteful? It might be basic advertising, but that doesn't make it right nor effective. If Musk would just do what he has without all the exaggerated promises, why's that not good enough? The claims make him look worse, not better. Denying reality is great until it comes crashing down. Just a few examples. Forgive this first title. The important part is the quote: http://www.businessinsider.com/musk-epic-failure-2016-5?op=1 http://www.businessinsider.com/musk-epic-failure-2016-5?op=1 Tesla reported first-quarter earnings last week, and while they were better than Wall Street expected, the earthshaking news that emerged was that Musk is taking the carmaker in a new strategic direction. Tesla, which delivered about 50,000 vehicles last year, is aiming to deliver 500,000 in 2018. The production target isn't new, but the timeline is. Tesla had previously said it would build half a million cars a year by 2020. This did not happen, which led him to Tweet last July that they could deliver 20k cars per month by Dec. This did not happen either. Investors were relieved they managed to hit 2k per week this past week (this is down from the 2.5k they were targeting). https://twitter.com/elonmusk/status/881757617416056832 https://twitter.com/elonmusk/status/881757617416056832 https://www.marketwatch.com/story/tesla-stock-jumps-after-production-misses-but-liquidity-concerns-addressed-2018-04-03 https://www.marketwatch.com/story/tesla-stock-jumps-after-pr... He's got a shaky attachment to the truth. I don't know if he really believes himself (this is called delusion, usually) or he's a liar. Either way it's kind of odd for a guy who doesn't need to exaggerate to be impressive. Not sure what it has to do with Nolan. He's never promised a 20 hour movie.
- qaq 9y ago
  Break the law? One surprising thing to me there is a ton of media attention on the SV right now (almost all negative) and that does not look like a coincidence. I disapprove of what FB is doing but on the scale of problems that Big Pharma, Health Insurance, big banks are causing thats not even a blip and yet most media attention and good amount of public outrage is fairly skillfully directed toward tech and SV.
  - Piskvorrr 9y ago
    Or perhaps that SV is vocal, excessively so: promises miracles, yesterday, at will, almost for free. In other words, massively overpromises, then underdelivers - that amounts to painting a huge sign "KICK ME" on its collective butt. While other enterprises might do that as well, even to a criminal extent, they do not brag about it. This might be all the explanation that's needed, no conspiracy necessary.
    - qaq 9y ago
      What miracles was FB promising? (Again not a fan of FB).
      - Piskvorrr 9y ago
        I would guess "we'll give you social network, not a total-surveillance tool" would qualify. I don't think the second part would have been a reasonable expectation.
        qaq 9y ago
        Would you point at FB marketing that has this message?
        Piskvorrr 9y ago
        I don't see any (although "FB is free and always will be" has plenty of wiggle room). Also, I don't see any explicit mention that Tesla's Autopilot is actually an autonomous Level 5 system - and yet people assume that. Are you saying that people should only be angry about things they were promised explicitly? (If so, I would agree - but 1. people are not 100% rational, 2. expectations are important for PR, even if not legally binding. Going from "SV doesn't put this in big bold letters on the front page" to "it's a conspiracy against SV, not because they're an easier media target" is a bit too much of a leap for me.)
oldgradstudent 9y ago
Wasn't the IIHS study relevant to the old Mobileye-based Autopilot? Any reason to assume it applies to Tesla's in-house Autopilot?
- jijojv 9y ago
  That may be correct as IIHS took few months and concluded in Jan'17. People are still waiting for in-house Autopilot aka AP2 to reach feature parity with mobileey AP1 if you follow https://forums.tesla.com/forum/forums/enhanced-ap-hw2-qa-firmware-2018104-81-photos-notes https://forums.tesla.com/forum/forums/enhanced-ap-hw2-qa-fir... so i'd suspect AP2 to be worse-off depending on when it gets tested.
ilitirit 9y ago
> at only reducing 40% accidents Unless I'm missing something, 40% is actually a good result (given what they're trying to measure). However, I think the more important statistic would be regarding how many injuries/fatalities does it reduce vs cause. Are there any studies on this?

bobsil1 9y ago

Gore point stripe faded: https://imgur.com/a/xHq57 https://imgur.com/a/xHq57 Street View: https://www.google.com/maps/@37.6346515,-122.104109,3a,75y,84.96h,99.37t/data=!3m6!1e1!3m4!1s4LqOilvG6oJ1oNApEwLi6A!2e0!7i13312!8i6656 https://www.google.com/maps/@37.6346515,-122.104109,3a,75y,8...

fossuser 9y ago
Can somebody just go out there and paint the road? There have been at least two other video near collisions of people driving on that section - it seems dangerous.
- pen2l 9y ago
  Holy shit that’s a good idea, we should just fucking do it ourselves. This reminds me when I was living in the Midwest and there was this one spot where I had to do a u turn in my commute which was very dangerous because of some tree branches obstructing view of oncoming traffic. One day I almost got into an accident and since it shook me quite a lot, I wrote an email to the city that the tree branch needed to be trimmed so I could see better. Sure enough I got an email back a few days later from a woman saying they “understand my frustrations” or something like that. I thought I did my part and went on with my day. Fast forward about a month, and I almost get in an accident at the same spot, man I was fucking furious that my communication with city was just “ohh we understand” —- instead of employing a sea of men and women telling me that they “understand” why could they have just not just trimmed the tree and improved safety. If I was still living there I think I would have just gone with my chainsaw and done it myself.
  - tlrobinson 9y ago
    > Holy shit that’s a good idea, we should just fucking do it ourselves. Seems to work for Portland Anarchist Road Care: https://www.citylab.com/equity/2017/03/portland-anarchists-want-to-fix-your-streets-potholes/519588/ https://www.citylab.com/equity/2017/03/portland-anarchists-w...
    - 9y ago

brandmeyer 9y ago

There certainly do appear to be some similarities. But without the car's logs, it isn't possible for anyone reading this forum to know for sure. That's what the NTSB is for.

ChuckMcM 9y ago

I have now talked with two people who have autopilot in their model S's and both said the problem with autopilot is that it is "too good". Specifically it works completely correctly like 80 - 90% of the time and as a result, if you use it a lot, you start to expect it to be working and not fail. If it beeps then to tell you to take over, you have to mentally get back into the situational awareness of the road and then decide what to do about it. If that lag time is longer then the time the car has given you to respond, you would likely crash. Makes me wonder how this gets resolved without jumping through the gap to 'perfect' autopilot.

doe88 9y ago
This. I don't know I feel ambivalent about autopilot right now, as most on HN I always crave for new technologies, but the fact that something very dangerous can be very good most of the time and very bad occasionally make me weary of using it altogether. I know that's irrational because most often then not it probably saves you from your own mistakes. Also, if Tesla intends to shield itself behind a beta status for their autopilot system until it is perfect, I think this beta status will remain even longer then the time GMail was in beta. At least, in this case they should own this problem and somewhat hardcode a meaningful warning at such intersections or do something.
- carlivar 9y ago
  It's not irrational at all. I drive our family 2018 Honda Odyssey, and it's already "saving me from my own mistakes" by beeping angrily when I signal for a lane change and there's someone in my blind spot. Works really well. It also has a notification light on each side of the car to indicate blind spot object presence, so I have to miss that first. The question seems to be which is better: 1) A car very intelligently helping a human drive better. 2) Or a car mostly driving well itself but needing humans to very rarely override fatal crashes. I prefer 1) I'm not sure there are any studies comparing these things, since "human driving but lots of extra tech help" is harder to quantify and doesn't have all the data going back to the mothership like Tesla.
  - 9y ago

brennankreiman 9y ago

Biggest thing I gathered was the public safety barrier was damaged from a previous crash, 12 days earlier, so the driver basically hit a hard wall. This has more implications for the US’s infrastructure than Tesla’s safety IMO.

danso 9y ago
via an earlier story: http://abc7news.com/automotive/exclusive-i-team-investigates-why-caltrans-didnt-fix-safety-barrier-before-tesla-driver-died-there/3280399/ http://abc7news.com/automotive/exclusive-i-team-investigates... > CalTrans finally sent ABC7 News a statement later Thursday"confirming" it's their policy to fix broken safety barriers within seven days, or five business days ... but storms delayed the work. As Dan Noyes reported Wednesday, the family told me they believe Walter Huang would have survived, if CalTrans fixed that safety barrier in a timely fashion.
- kwhitefoot 9y ago
  Until the barrier is fixed that part of the road should have some sort of marking to indicate that there is an extra danger there, that's what cones were invented for.
  - dannyw 9y ago
    I'm sure a hitting series of traffic cones taking up the space where the safety barrier was could have alerted even a modestly distracted driver to brake.
telmnstr 9y ago
Plus the genius driver had a number of issues with autopilot in the same spot. He had complaints in with the Tesla dealership that his car tried to steer him into the barrier in that location, prior I believe. Something like 7 complaints filed? Maybe he was trying to get out of his loan but didn't know the impact absorption barrier had been removed.

11thEarlOfMar 9y ago

Vehicles with autonomous features will save lives. Fewer people will be injured or killed in cars that have these features. However, some people will still be injured or killed, and those people will comprise a different set than would otherwise have been harmed. They or their families will seek restitution and hence, the way forward will be determined by the courts.

Declanomous 9y ago
The thing is that most driving deaths don't occur on the highway with dry-well lit roads, and current vehicles can't drive for shit in bad weather. So they shouldn't have autopilot on at all, they should just have the collision avoidance features on.

abalone 9y ago

I wonder if this will be the case that really tests Tesla's blanket legal defense in court. (That accidents are due to less than "fully attentive" drivers.) The reaction time needed in that video to correct for the autopilot is downright scary. It's a situation where if you were piloting yourself you're a lot more likely to have mentally mapped that you continue in a straight line at that point, so listing to the left is going to feel WRONG even when suddenly blinded by the sun. But if autopilot takes you there, and you're "lizard brain level" paying attention but didn't have that mental navigation map in your head, and the sun's in your eyes, you're going to take a few seconds to try to figure out how to correct so as not to make things worse. That's enough to hit the gore point in that video.

oldgradstudent 9y ago

Tesla claimed[1] that: > Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2015 and roughly 20,000 times since just the beginning of the year, and there has never been an accident that we know of. There are over 200 successful Autopilot trips per day on this exact stretch of road. Which might be technically true, but conveniently ignores other, very similar crashes. [1] https://www.tesla.com/blog/what-we-know-about-last-weeks-accident https://www.tesla.com/blog/what-we-know-about-last-weeks-acc...

Shivetya 9y ago
my question which is the same as others, does Tesla register events where the driver overrides the system? If they register it how do they account for it? if they are not even considering that the driver overrode the system because it was going to crash then all their claims are suspect.
- quotemstr 9y ago
  Why? The car isn't driving by itself. What matters is whether Autopilot is a net safety improvement, which it currently appears to be. Why would you add the weird requirement to the system that it be better without human input at all?
dannyw 9y ago
It took people a day to reproduce the crash and end up in a near-collision: https://www.youtube.com/watch?v=6QCF8tVqM3I https://www.youtube.com/watch?v=6QCF8tVqM3I

newnewpdro 9y ago

> Wednesdsay, a Tesla spokeswoman told the I-Team, "Autopilot is intended for use only with a fully attentive driver," and that it "does not prevent all accidents - such a standard would be impossible - but it makes them much less likely to occur." Actively veering towards stationary barriers in otherwise perfectly safe conditions is not what I would describe as reducing the likelihood of accidents.

lsaferite 9y ago
In this one specific case sure, the AP was confused about the lane markings and followed the wrong path. But in plenty of other cases the AP is responsible for preventing accidents (no, I have no link to back up my statement). Assuming that is the case, then that is what I would describe as reducing the likelihood of accidents. You cannot focus on a specific accident (or even a handful of accidents) and use that to refute the systemic benefits.

nodesocket 9y ago

Just my opinion, but I don't think the local bay area ABC I-TEAM news can be trusted as a realiabe primary source. Seems like they are "cashing in" on a national story.

danso 9y ago
Has anything in their past exclusives on this story been disputed? Victim who died in Tesla crash had complained about Autopilot: http://abc7news.com/automotive/i-team-exclusive-victim-who-died-in-tesla-crash-had-complained-about-autopilot/3275600/ http://abc7news.com/automotive/i-team-exclusive-victim-who-d... I-Team investigates why CalTrans didn't fix safety barrier before Tesla driver died there: http://abc7news.com/automotive/exclusive-i-team-investigates-why-caltrans-didnt-fix-safety-barrier-before-tesla-driver-died-there/3280399/ http://abc7news.com/automotive/exclusive-i-team-investigates...
unityByFreedom 9y ago
It's literally their job to report the news. What has been unreliable about their reporting?

TeeWEE 9y ago

When i look at the video, I would probably drive wrongly for the first few seconds: The road lining is not there anymore, causing you to make mistakes. But people can correct quickly when there is an anomaly. Weird that autopilot didnt see the divider.

danso 9y ago

Currently, the Tesla autopilot page has this as its top headline: > Full Self-Driving Hardware on All Cars Followed by this: > All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver. http://web.archive.org/web/20180323054727/https://www.tesla.com/autopilot http://web.archive.org/web/20180323054727/https://www.tesla.... Getting past the arguments of whether Tesla should be calling this "Autopilot", is its claims about hardware true? Is the failure to detect stationary objects like a firetruck or a gore point solely as a result of currently-inadequate software? Because it doesn't sound like the sensor suite is anywhere near the AVs of Waymo or Uber.

Shank 9y ago
> Getting past the arguments of whether Tesla should be calling this "Autopilot", is its claims about hardware true? Tesla actually hasn't called that Autopilot -- they refer to it as "full self driving" and not in the same category. But...is it true? It's vaporware right now, so we can't really say. Obviously, in a purely hypothetical landscape, even with cameras you should be able to emulate what humans can do and have 100% attentiveness. That is to say, humans don't have lidar and radar, yet humans do drive. So, if you can achieve human levels of perfection and have 100% attentiveness, it should be possible to beat out crash statistics based on the fact that many are caused by failure to respond on the human side accurately or fast enough. This is all hypothetical, though. All of it comes down to computer vision/software, which is really hard to do. It's possible, but not yet done.
- miskin 9y ago
  "Autopilot" is the name of the page that danso referred to, is in the title of that page and is used on the page 9 times. How would you possibly not recognize this as name of the feature?
- carlmr 9y ago
  >Obviously, in a purely hypothetical landscape, even with cameras you should be able to emulate what humans can do and have 100% attentiveness. That is to say, humans don't have lidar and radar, yet humans do drive. Which camera has the same dynamic range as the human eye?

jv22222 9y ago

I am a huge fan of Tesla. What I am noticing right now is that if I were able to afford a Tesla today, there is no way I would use Autopilot. I will still buy a Tesla if I can ever afford it, I'll just leave autopilot well alone. I don't know what gets me back from this mental place that seems to be firmly set in my mind now.

JimmaDaRustla 9y ago
Agreed - I might test using it, but I would ensure that I have 100% mental focus. Now I'm wondering, if I have to work hard to pay attention that a computer is operating correctly, what advantage am I gaining by using it? AutoPilot is a human assisted utility. I think many people here, and in general, expect too much of it and are being too critical when there is an incident. I do think Tesla needs to communicate better what AutoPilot is and isn't because we've now lost lives due to operator negligence, and by negligence, I mean the operator is still responsible for the vehicle despite relinquishing control to a computer.
vasilipupkin 9y ago
Just got a model S after driving my toyota camry for the last 10 years. Autopilot is a fun toy, but I can't understand for the life of me why people actually use it like its a real thing. I have it off on mine.

bra-ket 9y ago

this AI is dumb

spullara 9y ago

If you haven't driven a Tesla under autopilot please stop commenting about it. You really don't know what you are talking about.

bobsil1 9y ago

Btw linked accident was for a Model S with Tesla autopilot 1. Can tell because no camera on silver blinker assembly. They're now on 2.5.

ino 9y ago
Are you talking about the second video in the story, in which the car veers towards the divider?
- bobsil1 9y ago
  Photos for the crash on 92/880 in September.
josefresco 9y ago
Can autopilot v1 cars be upgraded to v2.5? I'm assuming since you mentioned a camera it's not just software.
- ohitsdom 9y ago
  No, Elon has tweeted a few times that it would be too extensive of an upgrade process. https://twitter.com/elonmusk/status/789007996038721536?lang=en https://twitter.com/elonmusk/status/789007996038721536?lang=...

Animats 9y ago

Come to think of it, whatever happened to Tesla's self-driving effort? They claimed to be shipping Model 3 cars with all the hardware needed for self driving (not including a LIDAR). They produced videos. Most of this was 1-2 years ago. But they never got to deployment, or even to the point of demoing it for car magazine writers.

lozenge 9y ago
They had a falling out with their software supplier Mobileye which set them back a year or more.
- B1FF_PSUVM 9y ago
  I gathered that Mobileye found that the use of the software was "outside the box" they were willing to risk ...
- BinaryIdiot 9y ago
  Was it really that much of a setback? It was my understanding they used some of their own software in combination and had already planned to take over the role Mobileye was serving. Maybe I bought too much into their PR spin but it sounded as if it was barely a delay.
  - stetrain 9y ago
    From reading owner discussions it seems like the general consensus is that they only just recently reached parity with (and slightly surpassed) their previous Mobileye implementation with their own software/sensor stack.
- Piskvorrr 9y ago
  Did I just read "they found out that tech is not actually ready to fulfill what its marketing promised", or is that something else?
  - Animats 9y ago

kalal 9y ago

When the self driving revolution started, conservative people claimed that one accident would stop the whole business. Well there are many accidents already and people are still willing to risk. One reason may be our tendency to trust something which we do not understand. We feel that if the feature is already in the car, it must be safe. Well it is not. And even worse, it is driven more by marketing and money than by safety regulations.

antirez 9y ago

What is happening was clear to many since the start: Tesla incarnates the behavior of his founder, exaggerating the technology that is really available right now, and selling it as a product without the premise for it to be safe. A product that kinda drives your car but sometimes fails, and requires you to pay attention, is simply a crash that we are waiting to happen. And don't get trapped by the "data driven" analysis, like "yeah but it's a lot safer than humans" because there are at least three problems with this statement: 1. Ethical. One thing is that you do a stupid thing and die. Another is that a technology fails in trivial circumstances that in theory are avoidable. A doctor can make also errors, but a medical device is required to be very safe and reliable in what it does. 2. Of wrong comparisons: you should check the autopilot against a rested and concentrated driver that drives slowly and with a lot of care. Since in the statistics otherwise you do not account for the fact that when you drive, you are in control, and you can decide your level of safety. Such a diligent driver that goes to crash because of an immature software is terrible. 3. And lack of data: AFAIK there is not even enough data publicly available that tells us what is the crash rate of Teslas with auto-pilot enabled vs Teslas without auto pilot enabled, per kilometer in the same street conditions. That is, you need to compare only in the same streets where the auto pilot is able to drive. Autopilot will rule the world eventually and we will be free of using the time differently (even if this was possible 100 years ago already stressing on public transportation instead of spending the money to have a car each one... this is sad. Fortunately in central Europe this happened to a degree, check for instance North Italy, France, ...). But till it is not ready, to plug it as a feature in a so immature way just to have an advantage over competitors, is terrible business practice.

kalal 9y ago
4. This statistics must be conditioned. Not all drivers are the same. Some are careful, some are not. So in fact, the careful drivers will make their life miserable by using autopilot.
sschueller 9y ago
How can anyone take a company serios that sells features such as "Bio-Weapon Defense Mode". It almost sounds like snake oil and not far from something like the ADE 651. [1] https://en.wikipedia.org/wiki/ADE_651 https://en.wikipedia.org/wiki/ADE_651

hugh4life 9y ago

Self driving cars have absolutely no business being any road/lanes that is not purposely made/reserved for self driving cars. Period. Self driving cars need an ecosystem made for self driving cars. It's frankly madness to think otherwise.

neolefty 9y ago
What if they improve safety on shared roads but aren't perfect?
- dannyw 9y ago
  If they are unable to avoid blatantly obvious yellow and black barriers / safety cushions and their algorithm is a simple "follow the whitest white line", then they should not be on shared roads.
  - neolefty 9y ago
    Again, they mostly do. Same with human drivers. By the same logic, should nobody drive?
igammarays 9y ago
Absolutely. It shouldn't even be that hard, with the billions that are being thrown at the tech. They should be spending time and money to upgrade specific highways and certify specific routes as "safe for automation", rather than trying to test their cars on the entire North American road network with millions of crazy edge cases. Over time, these certified routes will grow into their own network.

deleted 9y ago

[deleted]

josefresco 9y ago

Auto-pilot will put massive pressure on states and municipalities to improve road maintenance - specifically lane indicators and fixes to "peculiar" roads/intersections etc. This will increase budgets, which will in turn create massive opposition to self-driving autos by those not wanting to pay more taxes for other people's fancy cars. I travel/drive a good deal in the summer and for a few years now have silently (mostly) muttered to myself that if auto-pilot cars are going to use "painted lines" to navigate - they're "gonna have a bad time". Poorly painted lines aside, think of how many "wtf" moments you have on the roads around your town - just down the road from my house is a road that just... ends (due to poor planning years ago). It's obvious to human drivers what to do (you're supposed to exit the paved road and drive over a hump to the dirt road that continues), but for auto-pilot? Not so much unless they make use of "data sharing" that would reveal the human solution to this screwed up piece of road.

deleted 9y ago
[deleted]

riffic 9y ago

The Associated Press stylebook calls for reporters to avoid using the term "Accident": https://www.transalt.org/news/releases/9545 https://www.transalt.org/news/releases/9545

paul7986 9y ago

I'm going to wait five to ten years before I get my robot car. Let all those early adopters be the guinea pigs we need to reach that level of robot cars being safer then human drivers. It seems tons of upper middle class to wealthy people are lining up to pay dearly with their wallets and lives to be a billionaire's crash test dummy.

awakeasleep 9y ago
Aren't we all the crash test dummies if we're sharing the road with them?
- WhompingWindows 9y ago
  We are nothing like crash test dummies. Crash test dummies are intentionally slammed into barriers at 60+ mph. Testing of vehicle safety features, and safety features of many products, is most often done in mass markets. Automatic safety features are being rolled out in essentially every single auto brand. Automatic emergency braking (AEB) is set to be standard in numerous makes in a couple years. Lane control, lane reminders, blindspot detectors, these features are all being rolled out on the roads, to varying degrees of success. This is a less-safe version of the way drugs are brought to market. First, small-scale tests with cells only, then mice, then healthy people, then target patients, and finally post-market surveillance. Cars indeed have a MUCH less regulated rollout for their safety features, yet to say we are dummies in their tests is not even close to reality. While drug testing may be more stringent, we also don't have 30-40k people a year dying from "not taking a drug" accidents, which is the situation on our roads today. Ultimately, it comes down to lives saved. Every time you delay a safety feature, you have to ask the question: is this delay going to lead to loss of life or saving a life? That's the moral quandary the automakers are in, and it seems they go with "release it" much more often than not. That will lead to some EXTREMELY well publicized accidents, and yet we will NOT hear of the countless cases where everything went nominally, and who's to say precisely how many accidents were prevented?
  - paul7986 9y ago
    So are you rushing out to be a Musk guinea pig or already are?

dannyw 9y ago

I want to ask: why is Tesla's advanced suite of cameras and ultrasonic sensors unable to detect a big, yellow and black stationary barrier? In combination with chevron markings that mean "never ever drive here". Either should be a globally overriding signal to emergency break or turn, not "keep following the white line". If the answer is "during morning light, it doesn't see it", then they need to take the feature offline until the car is fitted with an appropriate LIDAR; or other technology that can detect stationary obstacles.

jonknee 9y ago
I've seen a lot of Tesla apologizers talk about how it's such a bad design, but people seem to be able to dodge it successfully and so should Autopilot. It seems like this feature should require hands on the wheel until it has enough training data that it doesn't do things like smash you into an obvious obstacle in whatever lighting conditions are possible at any time.
- pkulak 9y ago
  > this feature should require hands on the wheel It does.
  - jonknee 9y ago
    Yet, in this fatal crash: > "the driver’s hands were not detected on the wheel for six seconds prior to the collision" https://www.tesla.com/blog/update-last-week%E2%80%99s-accident https://www.tesla.com/blog/update-last-week%E2%80%99s-accide...
    - ricardobeat 9y ago
      What is the car supposed to do? Have robot arms that force your hands? It will actually slow down and stop if you continue not holding the wheel.
- rasz 9y ago
  > but people seem to be able to dodge it successfully are they? so why was the barrier crushed in the first place before the crash? and why was it designed to be easily rebuild? its almost like someone knew people will have trouble and will eventually drive into it.

w_t_payne 9y ago

My former employer makes a camera-based lane departure warning system. Although I was not involved with its development, I do know that the amount of testing that they do on these systems is very extensive (hundreds of people involved in the test process). I would be very interested in seeing a side-by side test of the LDW systems of major manufacturers. This might give us an idea if the problem is fundamental to the technology or is simply a deficiency in the engineering approach of one (or some) manufacturer(s).

MPSimmons 9y ago

I have to believe that, at this point, given the markings on the road, that California Dept of Highways (or whoever is responsible for lane markings) is just trying to kill people.

YeGoblynQueenne 9y ago

>> The company tells Dan Noyes they have made it clear -- Autopilot is a driver assistance system that requires you to pay attention to the road at all times. The question is why it is Tesla, in particular, that has to remind its customers that they have to actually pay attention when they drive its cars. Is it possible the company somehow managed to convince people that its cars are equipped with self-driving software, rather than a "driver assistance system"?

StefanFrost 9y ago

So aside from the fact that I can not afford these cars I would also give these at least 10 years to mature and get further than (what really does seem) at least the beta. Right at the start I was very much wondering how these even allowed this feature on the road. Just from a regulatory point of view. It very much sounded and looked like a complete auto-pilot. That said, people go through a drivers test, get their license and then are allowed to drive any vehicle really. Can you just solve this by giving the driver assistant a test and seeing if it passes? This needs some kind of filter or regulator to make sure it is safe.

gfodor 9y ago

For all those who are clamoring for autopilot to go back into the lab consider this: I have autopilot, and I use it the way people are instructed to. I consider it a companion to my driving, a backup co-driver that has better reflexes. I am in control, and mentally steer the vehicle with my hands on the wheel with the machine. We are driving the car together and I overrule the machine when we disagree. Often the machine catches things before I am able to react. I am 100% convinced that my family is safer with "us" driving them then just me driving them solo. What do you say to people like me, who, if you took the feature away, would be forced to reduce their safety? It's easy to call for its removal when you only focus on the subset of people who will have their safety increased by removing it (ie, the people, often through no fault of their own, who come to rely upon it more than it is designed for.) What about everyone else that currently benefits from it? Will you force them to go back to driving "solo" to protect those people? It's not so simple as "Tesla is reckless by releasing this" -- this is providing real safety to people who are using it the way it is intended. Also, the release of autopilot as-is also makes it possible to accelerate the development of full autonomy, which will save lives that otherwise would be lost to accidents if it were to be created later in time. Any analysis which does not account for this dynamic is missing an important piece of the ethics of removing it. The main question that is worth asking is if there is more Tesla could do to ensure autopilot is used as a co-driver. The name is a poor choice to start.

jakobegger 9y ago
Why not the other way around? You do the driving, and the computer is the backup in case you mess up? Humans are extremely bad at monitoring monotonic things, even if we are trained to do them. That‘s why trains have elaborate dead man switches to ensure the operator is paying attention. From these accidents it‘s clear that Teslas aren‘t good enough at making sure the driver is paying attention. And it‘s also pretty clear that these autopilot victims weren‘t aware how distracted they really were. I‘m pretty sure they thought they had everything under control, right up to the point when their car did something stupid.
- unityByFreedom 9y ago
  > Why not the other way around? You do the driving, and the computer is the backup in case you mess up? Unfortunately having the computer takeover has the same issues. If it thinks you're following the wrong lane it could choose to steer you into a barrier, for example. Chris Urmson's team at Google ran user tests at the start of their autonomous vehicle program and found that users misbehave [1] when behind the wheel of semi-autonomous vehicles, despite having been given instruction to pay attention. We're seeing that scenario play out in slow motion in Teslas and perhaps other driver-assistance programs that get less press. [1] https://youtu.be/Uj-rK8V-rik?t=14m7s https://youtu.be/Uj-rK8V-rik?t=14m7s

Jabbles 9y ago

To misquote Elon Musk: The advantage of getting somewhere [without having to steer] will be negatively affected if "but also, you might die" is on the ticket. https://www.reddit.com/r/space/comments/76e79c/i_am_elon_musk_ask_me_anything_about_bfr/dodes6v/ https://www.reddit.com/r/space/comments/76e79c/i_am_elon_mus...

pjkundert 9y ago

The root problem (as described by Taleb in "Skin In The Game", is: not all statistical probabilities are "ergodic" -- have no risk of Ruin or Extinction. The problem with saying that "our automatic driving system is much better than the average driver" and thus you should be good with allowing our system to drive, is that you are not exposed -- serially -- to the whims of some average driver. One "average driver" mistake and you're dead. No replays. Therefore, the compound "odds" are INCOMPUTABLE. Just like being paid $1,000,000 per trigger-pull to play Russian Roulette: you are not earning an average of $833,333.33 per try! On "average", over a pool of random 1-time players, sure. But you? You're guaranteed to be dead in a small number of plays. Non-ergodic and ergodic probabilities are incomparable. So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable. I live in a part of the world where even cruise control is unusable for 6 months of the year. Automatic driving systems? Not even thinkable, except for perhaps a few days of the year, on a few simple portions of a few roads...

dragonwriter 9y ago
> So; since automatic driving systems are non-ergodic, they must be dramatically better (ie. orders of magnitude better) than the "average driver" to be considered viable. The risks from human drivers are also non-ergodic, so that suggests that if automatic driving systems are even minimally better than the average driver you should demand that every other driver be replaced with them without delay, because you will die sooner with human drivers on the road. Of course, you won't want to be replaced unless the automatic driver is better than you, and human drivers all think they are far above average. And that’s the real problem.
- pjkundert 9y ago
  The real problem is that, yes, human drivers ARE better than computers — for some unknown subset of driving. It’s that small subset of lethal mistakes you avoid by not letting your computer drive that makes all the difference. The computer must not just be better on average — people must know that it’s better overall, across all possible weird events that a human just might throw a Hail-Mary and survive!