7 ms·

Reading that initial report is terrifying. I am so glad the NTSB set the record straight that the driver had his hands on the wheel for the majority of the fin

Reading that initial report is terrifying. I am so glad the NTSB set the record straight that the driver had his hands on the wheel for the majority of the final minute of travel. Really makes me feel like Tesla was out to blame the driver from the get go. To be clear the driver is absolutely partially at fault, but my goodness autopilot sped up into the barrier in the final seconds — totally unexpected when the car has automatic emergency breaking. Emergency breaking feels not ready for prime time. I hope there are improvements there. Don’t want to see autopilot disabled as a result of this, would rather Tesla use this to double down and apply new learnings. Just so sad to hear about this guys death on his way to work - not the way I want to go. :(

testplzignore 8y ago
> Don’t want to see autopilot disabled as a result of this, would rather Tesla use this to double down and apply new learnings. I agree that Tesla obviously needs to be learning from this. But at this point in time, Autopilot needs to be turned off until Tesla can fix the fundamental flaws in the set of sensors on their cars. There is now strong evidence that Autopilot is dangerous, and there is still weak evidence that it is less dangerous than the alternative (i.e., a human driving). I expect that the NTSB will eventually issue a safety recommendation that Autopilot not be used in its current form. I'm a bit surprised they didn't issue an urgent recommendation in the preliminary report, or even before.
- TaylorAlexander 8y ago
  I have a hard time seeing it this way. Turned off until they can fix the fundamental flaws? Which flaws exactly? It’s a prototype technology that no one knows how to do perfectly. Should Waymo also be taken off the road? None of these technologies are provably safe, so we always have to accept some risk. How much risk is appropriate is hard to determine. Perhaps LIDAR would reduce the risk, but Uber showed you can kill people even on a LIDAR based vehicle if your software fails to classify the risk properly. And humans drive with just two eyeballs so it’s not unreasonable to think camera based systems should be in principle possible to make safe. So we could pull all the self driving cars off the road, but then we inhibit their ability to collect the wide range of real world data necessary to build the technology. Or we could pull the less successful tech off the road sooner, but that would cause those programs to wither and hand the market to one or two well funded players. I just don’t see how pulling the tech off the road because of a flaw that Tesla has likely already fixed would accomplish much. I do struggle to understand what we should do, but I’m not so quick to think the tech should be pulled.
falcolas 8y ago
> To be clear the driver is absolutely partially at fault I'm... not so certain. Why? The autopilot had likely exhibited proper behavior every time that the vehicle had passed that particular section of road prior, and if the driver was paying full attention to the behavior of the vehicle, he would only notice the problem around the 5 second mark. Five seconds, if you have no reason to be concerned about the vehicle's behavior, is not much time - especially if you consider that alert drivers are recommended to give themselves a minimum of 4 seconds of reaction time (i.e. follow a vehicle by at least 4 seconds).
- dralley 8y ago
  If I recall correctly (and I might be wrong), didn't the driver in question actually report prior to the accident that autopilot had anomalous behavior on this section of road?
- ericpauley 8y ago
  My vehicle (a Honda civic) exhibits this exact same functionality and behavior. (lane keeping, ACC, emergency braking for cars) They make it very clear the limitations of these systems. I'd say that in the 5 months I've owned it it's had this exact behavior (veering off exit ramps) 10 times. A simple jerk of the wheel puts it back on track, it's such a natural motion if you're paying even the slightest bit of attention. That being said Tesla fails to make their drivers aware of the limitations of autopilot, so I agree that this may not be in the driver.
  - freerobby 8y ago
    Tesla reminds you to pay attention and keep your hands on the wheel every time you engage Autopilot. It's one of very few legal disclaimers they show you all the time on the screen, and don't give you any way of turning it off.
    - jonhendry18 8y ago
      And we all know how assiduously people pay attention to messages that flash up on screen.
kevincox 8y ago
> Don’t want to see autopilot disabled as a result of this I would. I think any self-driving system that expects the "driver" (passenger I guess) to take over in tough situations is bound to fail again and again. If people feel like they don't need to pay attention they won't no matter how much they know that they should. I think that partial autonomy is a sort of uncanny valley, either side is much safer.
- prawn 8y ago
  I'd prevent them from using the AutoPilot term, penalise them for marketing imagery that includes people driving with their hands off the wheel, and require hands to be on the wheel far more often. Until it's significantly improved, it should be a backup safety system rather than a driving and safety system.
  - Robotbeat 8y ago
    >I'd prevent them from using the AutoPilot term I disagree. It's the same term as in aviation and does the same thing. > penalise them for marketing imagery that includes people driving with their hands off the wheel That's more reasonable.
    - HumanHater 8y ago
      Can this "Autopilot" control altitude of the vehicle? No. Therefor it's not the same thing, it is just something similar. What should we call car autopilot? There is more than one opinion. Some people believe in importance of some weird technical characteristics. For most people defining property of autopilot is its ability to control vehicle while pilot left the cockpit for a few minutes. So to them car autopilot must safely drive some roads and require human to take full control on the others. Taking over in case of emergency doesn't exactly fit this idea. What looks really weird in all this discussions is their fanatical uselessness. Who would be hurt if new law would prohibit to call autopilot anything less than L5? Well probably no one. Who would benefit? Most likely a lot of people. Why would anyone oppose regulation like this? Huge question.
    - 8y ago

testplzignore 8y ago

> Don’t want to see autopilot disabled as a result of this, would rather Tesla use this to double down and apply new learnings. I agree that Tesla obviously needs to be learning from this. But at this point in time, Autopilot needs to be turned off until Tesla can fix the fundamental flaws in the set of sensors on their cars. There is now strong evidence that Autopilot is dangerous, and there is still weak evidence that it is less dangerous than the alternative (i.e., a human driving). I expect that the NTSB will eventually issue a safety recommendation that Autopilot not be used in its current form. I'm a bit surprised they didn't issue an urgent recommendation in the preliminary report, or even before.

TaylorAlexander 8y ago
I have a hard time seeing it this way. Turned off until they can fix the fundamental flaws? Which flaws exactly? It’s a prototype technology that no one knows how to do perfectly. Should Waymo also be taken off the road? None of these technologies are provably safe, so we always have to accept some risk. How much risk is appropriate is hard to determine. Perhaps LIDAR would reduce the risk, but Uber showed you can kill people even on a LIDAR based vehicle if your software fails to classify the risk properly. And humans drive with just two eyeballs so it’s not unreasonable to think camera based systems should be in principle possible to make safe. So we could pull all the self driving cars off the road, but then we inhibit their ability to collect the wide range of real world data necessary to build the technology. Or we could pull the less successful tech off the road sooner, but that would cause those programs to wither and hand the market to one or two well funded players. I just don’t see how pulling the tech off the road because of a flaw that Tesla has likely already fixed would accomplish much. I do struggle to understand what we should do, but I’m not so quick to think the tech should be pulled.

falcolas 8y ago

> To be clear the driver is absolutely partially at fault I'm... not so certain. Why? The autopilot had likely exhibited proper behavior every time that the vehicle had passed that particular section of road prior, and if the driver was paying full attention to the behavior of the vehicle, he would only notice the problem around the 5 second mark. Five seconds, if you have no reason to be concerned about the vehicle's behavior, is not much time - especially if you consider that alert drivers are recommended to give themselves a minimum of 4 seconds of reaction time (i.e. follow a vehicle by at least 4 seconds).

dralley 8y ago
If I recall correctly (and I might be wrong), didn't the driver in question actually report prior to the accident that autopilot had anomalous behavior on this section of road?
ericpauley 8y ago
My vehicle (a Honda civic) exhibits this exact same functionality and behavior. (lane keeping, ACC, emergency braking for cars) They make it very clear the limitations of these systems. I'd say that in the 5 months I've owned it it's had this exact behavior (veering off exit ramps) 10 times. A simple jerk of the wheel puts it back on track, it's such a natural motion if you're paying even the slightest bit of attention. That being said Tesla fails to make their drivers aware of the limitations of autopilot, so I agree that this may not be in the driver.
- freerobby 8y ago
  Tesla reminds you to pay attention and keep your hands on the wheel every time you engage Autopilot. It's one of very few legal disclaimers they show you all the time on the screen, and don't give you any way of turning it off.
  - jonhendry18 8y ago
    And we all know how assiduously people pay attention to messages that flash up on screen.

kevincox 8y ago

> Don’t want to see autopilot disabled as a result of this I would. I think any self-driving system that expects the "driver" (passenger I guess) to take over in tough situations is bound to fail again and again. If people feel like they don't need to pay attention they won't no matter how much they know that they should. I think that partial autonomy is a sort of uncanny valley, either side is much safer.

prawn 8y ago
I'd prevent them from using the AutoPilot term, penalise them for marketing imagery that includes people driving with their hands off the wheel, and require hands to be on the wheel far more often. Until it's significantly improved, it should be a backup safety system rather than a driving and safety system.
- Robotbeat 8y ago
  >I'd prevent them from using the AutoPilot term I disagree. It's the same term as in aviation and does the same thing. > penalise them for marketing imagery that includes people driving with their hands off the wheel That's more reasonable.
  - HumanHater 8y ago
    Can this "Autopilot" control altitude of the vehicle? No. Therefor it's not the same thing, it is just something similar. What should we call car autopilot? There is more than one opinion. Some people believe in importance of some weird technical characteristics. For most people defining property of autopilot is its ability to control vehicle while pilot left the cockpit for a few minutes. So to them car autopilot must safely drive some roads and require human to take full control on the others. Taking over in case of emergency doesn't exactly fit this idea. What looks really weird in all this discussions is their fanatical uselessness. Who would be hurt if new law would prohibit to call autopilot anything less than L5? Well probably no one. Who would benefit? Most likely a lot of people. Why would anyone oppose regulation like this? Huge question.
  - 8y ago