11 ms·

I'm a huge Musk fan and wish him and Tesla all the success in the world. But releasing any kind of machine-assisted driving without a near infallible "don't hit

• zip1234 9y ago
  On the other hand, if you know that the technology is not completely infallible, but better than humans in a general sense and will save lives by releasing it, wouldn't you be morally obligated to do so? Wouldn't holding it back even though it would save lives be worse?
  • dlp211 9y ago
    It can't be marginally better than humans, it has to be an order of magnitude or more better. Humans are irrational, and incidents like this erode public trust.
    • zip1234 9y ago
      I agree, but there is also news about how Tesla's automatic braking features saved people from getting into collisions. I would think that might counteract some of the negative news. See this one from six months ago: https://www.youtube.com/watch?v=Ndeb1pMAsh4 https://www.youtube.com/watch?v=Ndeb1pMAsh4
      • mantas 9y ago
        Most manufacturers have automatic braking. But they do not pretend it's "autopilot" nor let user to rely on it. It's just there to save their ass if shit hits the fan.
      • makomk 9y ago
        It shouldn't counteract the negative news because every car manufacturer offers automatic emergency breaking as a feature so it's not a distinguishing benefit of Tesla, it works fine without Tesla's controversial and dangerous Autopilot mode enabled, the main example Elon Musk himself posted of it preventing a collision was in manual mode in conditions totally unsuitable for Autopilot, and the entire industry already agrees that it's a worthwhile safety improvement. I believe it's even going to become mandatory on new cars in the US at some point for safety reasons.
      • carlmr 9y ago
        Apart from this being a very basic feature in almost every (premium) car manufacturer offers, this is a very grave misunderstanding of human psychology. You can never cancel fear with good news. Otherwise we'd be happy about how safe we are today, not worrying about terrorism (in the US that is, there are countries with legitimate elevated terrorism risks that are scary and actually worrisome).
  • adanto6840 9y ago
    It's a tough thing to weigh however when it's perhaps "better than humans" in 98% of situations, but "on par with" or perhaps even "worse than humans" for the remaining 2%. That's where it gets sketchy to me, at least. That said, I'm casually in the market for a Tesla (ideally pre-owned) and one of the features I'd require is the latest auto-pilot hardware. I'd definitely make use of it right away too -- but I'd do so with the understanding that it is far from infallible, and that I still need to keep substantially all of my attention trained to the road just as I would in any other non-autopilot vehicle.
  • apeace 9y ago
    I think your point is void here, because in both of the cases I mentioned, the car was clearly worse than a human. Who drives into a concrete barrier or the middle of a tractor trailer truck? If the technology were better than humans I'd agree.
    • greendesk 9y ago
      >>Who drives into a concrete barrier or the middle of a tractor trailer truck? I have seen a drunk driver do exactly that. Someone who falls asleep might also do that. [edit] Come to think of it, I have almost caused an accident a decade ago. I was momentarily distracted as I turned to talk to my passenger. I was able to brake in time, as a car in front of me had suddenly stopped. Nevertheless, it was only a matter of maybe one second to avoid running into the other car. Me-a-decade ago might been in a similar accident.
      • chc 9y ago
        Also somebody who's been blinded by a bright light, or is suffering from a medical event, or is looking at their cell phone. There are lots of things that cause people to hit objects that should be easily avoidable.
      • losteric 9y ago
        I know one person that drove into a light post in the middle of an empty parking lot. Daylight, sober, not texting, not tired, not stunting - they were looking at the exit and simply didn't see the one light post between their car and the exit. "That light came out of nowhere!" Human car accidents are not news because roughly 100 Americans die in car accidents every day, out of ~15300 car accidents per day.
      • yahna 9y ago
        Neither of those people should be using autopilot. It wouldn't save them. I can also avoid being in either of those situations. If autopilot is comparable ot an exhausted or drunk driver why do I want to use it?
    • viraptor 9y ago
  • Dylan16807 9y ago
    The bar is not infallible in general, just infallible at driving directly into obstacles. Is it realistic to worry about a car that can't get that right, but is simultaneously safer than a human driver overall?
  • davrosthedalek 9y ago
    I think people will prefer risking being killed by their own mistakes than by the mistakes of an AI, even if that risk is somewhat higher, maybe even considerably higher.
    • sol_remmy 9y ago
      Disagree. I will have my family use whichever is statistically safer.
      • davrosthedalek 9y ago
        This might be harder to gauge than you think. Maybe a Tesla is much safer on the highway, but less safe in the city, or the other way around. Or less safe on ice, etc. The averaged risk (and we probably won't have anything else for a while) over the whole population might be quite different from that for your actual use.
    • semiel 9y ago
      Surely they can not turn on autopilot, if that's the case? For me, I prefer life over a false sense of control.
    • RandomBookmarks 9y ago
      Nope... I am actually happy to admit that I am a below average driver (as driving is sooo boring), and from what I read, the AI is already driving better than me - if used in the conditions it is designed for.
    • erobbins 9y ago
      I do. I have no interest in a self driving car. I hate being a passenger in a human driven car too. Give me control of my own destiny!
    • dkonofalski 9y ago
      Don't they already have that choice, though? No one is forcing anyone to use Autopilot. That's my main issue with the parent posts and the self-driving sentiments they're expressing - the person using it was grossly negligent and ignored all the warnings presented. You accept that the technology is in beta and that you accept the risks of that. The person ignoring those warnings is, more than likely, much less safer of driver than Autopilot ever could be. I just really don't see how this could be pinned as Tesla's fault. We're treading very closely to territory where we absolve people of personal responsibility simply because we have technology that we can blame, even if it's irrational or misdirected blame.
  • rohit2412 9y ago
    What if Tesla released autopilot as a completely passive accident prevention driving assist? Tesla seems to be allowing only two options - one with no driving assistance, and one where they can call it autonomous.
    • greglindahl 9y ago
      Emergency braking is still active even if you don't have autopilot turned on. This is also true for a lot of other cars with Level 2 autonomy.
      • rohit2412 9y ago
        The statistics being touted for reduced accident include emergency breaking.
        • greglindahl 9y ago
          Yes. I'd like to see AEB broken out of the accident number. Looks like this NTSB investigation did not release such a number.
• dlp211 9y ago
  I mostly agree and I wonder how long until incidents like these start eroding public trust. I fully believe that fully autonomous driving can and will bring a whole new paradigm to transportation in the US, but it won't happen if we try to rush something so critical.
  • balls187 9y ago
    Was it rushed though? Can you compare the number of hours driven with the autopilot and compare it to the statistics of human drivers, and determine which was safer? This was a fatal crash, but faulty airbags caused the death of 10 people. What selfdriving cars needs, rather than more time to bake, is a PR or lobby group, that can help spin (put in perspective) these types of events.
    • dlp211 9y ago
      I can only speculate, but the fact that engineers have left Tesla over this very issue is telling.
    • yahna 9y ago
      No, these are inexcusable deaths. This is literally a fatal flaw in the system.
      • dkonofalski 9y ago
        I'm sorry... I just don't buy that. Airbags have deployed and killed people by snapping their necks yet we include airbags in every single vehicle on the market without requiring that the drivers or passengers read and accept a warning about airbags. Autopilot requires manual consent from the user to even enable it and has so many overrides for the user that I feel like you're really stretching it by calling it a "fatal flaw". All it would have taken to prevent this accident was the person driving to hit the brakes in time. Even if they were just using cruise control in their vehicle, they would have died. This was completely about inattention and irresponsibility on the part of the driver. I can't see how anyone is making the leap to this being a "fatal flaw" in the system.
• ghgr 9y ago
  > [...] where a Tesla crashed into a concrete barrier[1] [...] Of course it should never have happened. Still, I'm impressed by how gracefully it handled the accident; emergency lights, keeping the lane and slowly braking the vehicle to a halt (again, such frontal collision was still unacceptable).
• mdekkers 9y ago
  their tech is not ready for release. Sounds like a typical software development effort. we always used to say that if we built bridges the way we write software, there wouldn't be many bridges. Well, now we build cars the way we build software, and guess what is happening... I firmly believe that this is all safety critical code, and we should look to NASA to ask how its done: https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Developing_Safety-Critical_Code https://en.wikipedia.org/wiki/The_Power_of_10:_Rules_for_Dev... and https://www.fastcompany.com/28121/they-write-right-stuff https://www.fastcompany.com/28121/they-write-right-stuff But hey, all that boring "safety" stuff is expensive, and takes too much time to complete.... EDIT: As has now become usual for HN, I am being downvoted instead of debated. This used to be a site for good debate. Too bad that's no longer the case.
  • mulmen 9y ago
    Move fast and break things turns out to be a bad way to build cars.
  • dkonofalski 9y ago
    I think you're being downvoted because you're making the assumption that Tesla and every other autonomous car manufacturer isn't putting any resources into "all that boring 'safety' stuff". That singular point really devalues the entirety of the rest of your post.
    • mdekkers 9y ago
      Interesting point, and not what I was assuming at all. I simply believe they don't put enough thought into it (and clearly the NTSB agrees). Downvotes should be about bad form, not about "I don't agree with you". HN used to be about the former, but now is much more the latter, and it is a poorer community because of it. HN isn't the first online tech-news community, and every single one of these communities that uses an up/downvote mechanism to self-regulate comment quality have gone the same way: as an increasing number of users gain voting privileges, voting replaces debate. When that happens, the community becomes a simple echo-chamber for the "established" truth of that community. This increasingly keeps away quality commenters, strengthening the effect. When voting, especially downvoting, is a frictionless, costless action, the quality of varied discourse will sooner or later die.
• mrdatawolf 9y ago
  "In 2013, 32,719 people died in motor vehicle collisions." (fro: https://www.millerandzois.com/car-accident-statistics.html https://www.millerandzois.com/car-accident-statistics.html) And you think it's a big deal that 2 crashes have happened? Also that truck driver broke the laws of the road and caused that mans death. That man broke the laws of the road and caused his own death. Pretending that the computers are to blame is silly. Put the blame on the people responsible not a computer. BTW same with the barrier incident. That was a driver failing to be vigilant. It was his fault for leaving autopilot on in a non-standard environment.
  • felippee 9y ago
    Obviously this truck may have broke down in that lane and could not be moved out, which removes the fault from the truck driver. In general cars have drivers, exactly because such things can happen and will happen without anyone else to blame. Neither does the blame matter if you are dead as a result. The 2 (in fact there were more) crashes are important because they show the state of the technology. Also keep in mind 99.9% drivers don't have an autopilot, 0.1% does. 2 crashes among the 0.1% are just as important as 32k crashes among the 99.9% of the rest.
• slg 9y ago
  I think Musk looks at it differently. There appear to be two fundamental truths about the current autopilot system. First off, the number of fatalities that have occurred with autopilot active compared to the amount of miles driven with autopilot active suggest that autopilot is safer than a human driver. Secondly, autopilot still is not flawless and it is possible for it to cause fatalities. Some people feel the latter should be enough to stop allowing drivers to turn on autopilot. I am betting Musk feels that the former serves as a moral obligation to get the safety features of autopilot in as many cars as possible in order to save as many lives as possible. He is throwing the switch on the trolley tracks [1]. [1] - https://en.wikipedia.org/wiki/Trolley_problem https://en.wikipedia.org/wiki/Trolley_problem
  • felippee 9y ago
    "the number of fatalities that have occurred with autopilot active compared to the amount of miles driven with autopilot active suggest that autopilot is safer than a human driver." Obviously you see how this deduction is subject to dozens of biases stemming from the fact that people turn on autopilot when the road conditions are safe to begin with. One would have to really carefully filter this data to avoid these biases, otherwise such conclusions are absolutely bogus.
    • toomuchtodo 9y ago
      The 40% reduction in accident rate was the result of the NTHSA investigation: https://techcrunch.com/2017/01/19/nhtsas-full-final-investigation-into-teslas-autopilot-shows-40-crash-rate-reduction/ https://techcrunch.com/2017/01/19/nhtsas-full-final-investig... "It’s essentially as good as result as Tesla can have hoped for from the U.S. traffic safety agency’s investigation, which took place over the last six months. Reuters reported earlier on Thursday that the investigation would not result in a recall of Tesla vehicles, but the full findings show that in fact, the federal regulatory body found plenty to praise while conducting its inquiry. The investigation does conclude with a minor admonition that Tesla could perhaps be more specific about its system limitations in its driver-assist features, but acknowledges that all required information is present and available to drivers and vehicle owners."
• Animats 9y ago
  Releasing any kind of machine-assisted driving without a near infallible "don't hit objects in front of the vehicle" function is simply irresponsible. Yes. There are four videos of Tesla cars crashing into objects which partly protruded into a lane. Three vehicles, one construction barrier [1], all on the left side. One fatal.[2] Relying on the Mobileye vision system, which is "recognize car/pedestrian, compute range to target" for anti-collision was grossly negligent of Tesla. Tesla has a radar, but it can't be very good if this is happening. Even the old Eaton VORAD radar, which we used on our 2005 DARPA Grand Challenge vehicle, would have avoided that. This is the simplest and most important part of automatic driving, and Tesla botched it. [1] https://www.youtube.com/watch?v=YIBTBxZ3NWw https://www.youtube.com/watch?v=YIBTBxZ3NWw [2] https://www.youtube.com/watch?v=fc0yYJ8-Dyo https://www.youtube.com/watch?v=fc0yYJ8-Dyo
  • chc 9y ago
    And yet, according to the NHTSA, cars are still much safer with it than without it. You seem to be smack-talking rather than offering a reasoned analysis of whether Autopilot is better than no-Autopilot.
    • jonathankoren 9y ago
      >Autopilot is better than no-Autopilot. If only there was something to put in the middle of this reasoning! Elon Musk fucked up as soon as he named it "autopilot". He set expectations at autonomous, and released it as a "beta" because that's what you do with some web app, and then when people started using it as an autonomous driving product (because that's what "autopilot" means), they unsurprisingly got killed. But shruggie! Move fast and break things! It's beta! What ya gonna do? On the other hand, if it was called "driver assist", or "super cruise", or some other thing, then drivers' expectations wouldn't be set at "autonomous driving", and then the'd be paying more attention, and probably wouldn't end up dead. I'm sorry, but this is all about setting expectations, and Elon fucked this up.
      • chc 9y ago
        I'll agree that the name "Autopilot" seems to mislead people, but that is a complete side track. Neither the person I replied to nor the person he replied to were talking about the name; they were talking about the technology and whether it was responsible to release it at all, with any name. (For one clear example, the GGP criticized "releasing any kind of machine-assisted driving" with Autopilot's limitations.)
• jacquesm 9y ago
  > Anyone who uses Tesla machine-assisted driving features today is putting themselves in grave danger That's their right to decide or not. What bugs me is that they also put me (and everybody else on the road) in danger.
• Karunamon 9y ago
  This crash, as well as the one where a Tesla crashed into a concrete barrier are evidence that their tech is not ready for release. But then you're forced to compare the autopilot accident rates against the regular old human accident rates. When you do that, the autopilot wins every time. Put another way, I'd be comfortable getting into one of these today. I trust the computer's bugs a lot more than I trust my own. I also trust its reaction time above my own, which is more likely to keep me safe if someone else does something stupid.
  • YCode 9y ago
    That's a major flaw in people's thinking about autopilot programs. An autopilot doesn't have to be 100% safe, it just needs to be significantly safer than a human pilot and in the long run it will save lives. I think the fallacy is thinking that it's okay for humans to be flawed but machines should be infallible.
    • rohit2412 9y ago
      Why are your options autopilot can unaided human? Wouldn't you compare autopilot va autopilot assisted human? That is, where the autopilot is a passive system that only comes into play to prevent accidents.
• agumonkey 9y ago
  I feared him rushing the feature long ago. Thought they raised the bar adequately since. He should stop spawning new projects and give more resources to this one instead.
• greglindahl 9y ago
  So, basically, you want to outlaw most current implementations of Level 2 autonomy, which are being shipped by a large number of carmakers. I was unaware that Waymo was planning on shipping a Level 2 system.
• manicdee 9y ago
  I use cruise control on my 2012 Honda all the time. Am I putting myself in danger?
• mgoetzke 9y ago
  BMW's auto cruise control and distance keeping + emergency braking etc isn't nearly good enough either as I know from experience. But it still helps.