6 ms·

This is why I don't understand the sad conclusion of the story in the article. The driver knew that AutoPilot had issues with that stretch of the freeway and wa

This is why I don't understand the sad conclusion of the story in the article. The driver knew that AutoPilot had issues with that stretch of the freeway and was able to repeat the issues on several occasions but still thought it best to be on his phone, while driving 71 mph, at the mercy of AutoPilot? Especially considering that he was an engineer who had expressed concern over this, it seems silly to me that he wouldn't just take over manually during that stretch... He didn't and now he's gone because of it. It's not worth the risk/reward.

mumblemumble 7y ago
We don't need to understand it. We just need to know that, even armed with all that knowledge, he still made the decision he did, and that we should not expect others with less knowledge of how software works to be any less human.
- dopamean 7y ago
  This has been my takeaway from the whole situation. It says something about the exactly how safe this stuff needs to be for it to be considered reliable.
  - heliodor 7y ago
    My impression is that it's already more reliable than humans. Not perfect, but it probably averages better than all the crashes humans are causing.
    - Reelin 7y ago
      > it's already more reliable than humans Only when comparing aggregate statistics. Not all humans are equally (un)safe drivers; insurance rates vary based on driving record for good reason.
- Drew_ 7y ago
  If anything I would expect less knowledgeable people to be more skeptical and cautious than somebody who has a vested interest and passion for technology. I can't imagine anybody other than a tech enthusiast who would maintain trust in a self driving car technology that previously attempted to veer them into a barrier.
  - riversflow 7y ago
    100% agree with this. Most people I’ve talked to about this incident think the guy is an idiot; if you had found a car’s feature to be unsafe on a portion of your commute before, why on earth would you trust it with your and other’s lives? If you had a new fancy belaying device slip in the gym, you wouldn’t use it on your next big wall climb. I also don’t get how everyone is forgiving him for being on his phone, in a construction zone no less. Reckless driving is reckless driving, being an Apple Engineer and Tesla owner doesn’t somehow negate that he was being a belligerent driver.
    - mumblemumble 7y ago
      It's not about forgiving him for being an idiot. It's more about recognizing that there's more than just the one idiot on the road. To an approximation, pretty much everyone who has ever had a driver's license has been guilty of making an idiotic decision or two while behind the wheel. And, Autopilot being a technology that, among other things, enables (or even encourages) idiotic behavior, there's real risk that placing too much blame on the driver's choices lulls us into an attitude that enables the next idiot to kill themselves and/or someone else.
      - riversflow 7y ago
        I agree that everyone makes bad choices from time to time on the road. But as a society we put 100% of the blame on the driver if they aren’t doing everything in there power to prevent an accident. “Everyone makes mistakes” isn’t a get out of jail free card, booze clouds people’s judgement to the point where after enough drinks they’ll think they are good to drive, but that doesn’t change anything if they get behind the wheel, its still a DUI. This guy was on his phone in a construction zone and crashed because of his lack of intervention, using a driving assist feature doesn’t somehow absolve him from being so preoccupied that his vehicle veered off the road and into a wall. Imagine if instead of him dying he had killed a construction worker; I have no doubt a jury would find him guilty of manslaughter. When you get into the driver’s seat of a car you are taking on responsibility for a death machine. I find it troubling that this conversation is happening at all, the blame should be put squarely on his shoulders. Iff the car suddenly slammed the wheel to the side causing him to lose control or became unresponsive to his inputs, that would be another matter. But this could have been prevented if he wasn’t being grossly negligent of the risk he was partaking in behind the wheel.
- StevenRayOrr 7y ago
  > We just need to know that, even armed with all that knowledge, he still made the decision he did, and that we should not expect others with less knowledge of how software works to be any less human. This is maybe one of the most important lessons of the 20th and 21st centuries (at least thus far): knowledge does not automatically prevent us from errors in judgments nor does it necessarily protect us from misfortune.
  - cmurf 7y ago
    Nor does it absolve others from liability.
  - phkahler 7y ago
    knowledge does not automatically prevent us from errors in judgments nor does it necessarily protect us from misfortune. Relevant xkcd https://xkcd.com/795/ https://xkcd.com/795/
    - mark-r 7y ago
      Not automatically, but it can help. I was in this very situation at the top of Pikes Peak. A storm moved in and the park rangers closed the place and sent everybody away. They knew the statistics of being hit by lightning in that very place.
  - ChipSkylark 7y ago
    My friends like to talk about how awesome it would be to time travel or know what the outcome of a future decision will be. Even when armed with knowledge from observed outcomes of near-identical setup scenarios, majority of the time they end up doing exactly what they would have normally done. Happens to me from time to time.
- troyvit 7y ago
  Maybe we should keep it the way and use it as a culling method.
ajross 7y ago
It was a mistake, people make them all the time. Mistakes in cars kill people, and you don't need to be driving a Tesla to see that. Frankly the whole point of automated control is to reduce this kind of mistake fatality, and... I mean, it's working. This was a tragedy for sure, but it was also fully two years ago. These events haven't been recurring, it's likely the specific proximate causes have been addressed, and by all reckoning these systems (while still not flawless!) are at or above parity with alert human drivers in terms of safety. Basically, I read the same facts you do and take the opposite conclusion. People make bad risk/reward decisions all the time, so we need to take them out of the loop to the extent practical.
dahdum 7y ago
His phone was using data, that’s not proof he was using it and distracted. Spotify streaming a new playlist could be responsible for that.
- dkonofalski 7y ago
  > Recovered phone logs show that a strategy game called “Three Kingdoms” was “active during the driver’s trip to work,” the NTSB said. Investigators said the log data “does not provide enough information to ascertain” whether Huang “was holding the phone or how interactive he was,” though it said “most players have both hands on the phone to support the device and manipulate game actions.” Huang’s data usage was “consistent” with online game activity “about the time of the crash,” according to the NTSB.
- pier25 7y ago
  Maybe he wasn't using his phone but he was distracted or maybe even asleep. Otherwise why let the car crash?
jhallenworld 7y ago
Maybe he was fulfilling Apple's demands: either attend a meeting on the phone while driving, or wake up earlier. (Definitely not singling out Apple here.. at IBM I had a coworker who was in an accident during a phone meeting- luckily non-fatal).
- jerkstate 7y ago
  I have coworkers who regularly take meetings or text chat/email while driving. It makes me uncomfortable. I have tried hinting that it's a bad idea, but don't feel comfortable telling them directly (don't think it would do anything except harm our relationship due to me questioning their judgement) or reporting to HR (fear of retaliation, some of the coworkers are senior to me in the management chain). On the other hand my discomfort kind of makes me feel like a busybody. 99 times out of 100 nothing will happen and the 100th time will probably just be a fenderbender. Not sure what to do except continue feeling uncomfortable with it.
  - dave5104 7y ago
    "Are you driving right now? We can reschedule to something more convenient for you? I don't want you to get pulled over."
    - deleted 7y ago
      [deleted]
  - copperx 7y ago
    You're way too cautious in your human interactions, which is terribly sad. But I understand it because of our business culture.
  - sitkack 7y ago
    We had this PM who ran our meetings, when people phoned in from the road she would say, "please take your calls while your aren't driving" and boot them from the meeting. Now I do the same thing and you should to.
- 7y ago
6gvONxR4sf7o 7y ago
> ...but still thought it best to be on his phone, while driving 71 mph, at the mercy of AutoPilot? > Records from an iPhone recovered from the crash site showed that Huang may have been using it before the accident. Records obtained from AT&T showed that data had been used while the vehicle was in motion, but the source of the transmissions couldn't be determined, the NTSB wrote. One transmission was less than one minute before the crash. For all we know, that could mean he had spotify on. Anyhow, "he was worried about it" is no reason to shift the blame to him.
- dkonofalski 7y ago
  I disagree. He was informed enough about the technology and aware enough of its limitations to write extensively about it. Tesla and all other assisted driving technologies all give the user a warning to stay alert and maintain control of the vehicle. He was clearly distracted somehow because he was relying on the technology. Also: > Recovered phone logs show that a strategy game called “Three Kingdoms” was “active during the driver’s trip to work,” the NTSB said. Investigators said the log data “does not provide enough information to ascertain” whether Huang “was holding the phone or how interactive he was,” though it said “most players have both hands on the phone to support the device and manipulate game actions.” Huang’s data usage was “consistent” with online game activity “about the time of the crash,” according to the NTSB.
yumraj 7y ago
Why was Autopilot driving above the speed limit? Or does Tesla allow you to set the speed?
- dkonofalski 7y ago
  It allows you to set the speed but only up to 8 mph above the posted speed limit (or something like that).
  - erulabs 7y ago
    As far as I know my model 3 at least doesn't limit the auto-pilot speed other than the max which is 90MPH. It's just a scroll wheel, you can set it however you like.
    - dkonofalski 7y ago
      I never used FSD but AP never let me go above 55mph in a 45mph zone when it was using traffic-aware driving.
Balanceinfinity 7y ago
He was an engineer - he probably wanted to try and "solve" the problem (isolate the car's behavior and see what inputs are causing the incorrect reaction) and engineer a solution. As an engineer, I can understand this drive. He probably felt he was on top of it because he knew of the bug.
- dkonofalski 7y ago
  I can understand that drive too. I cannot understand allowing a technology that you know to be unsafe take over completely so that I can play a game on my phone.
jm4 7y ago
That’s the part I don’t get either. The only thing I can come up with is he wasn’t paying attention to the drive up to that point and didn’t notice he was in a problem area. I have a Tesla and there are definitely problem areas. You learn them fairly quickly when you are taking the same route all the time and you’re trained to either turn off autopilot or at least be alert when going through those areas. Or maybe you test it out with your hands on the wheel ready to take over to see if they fixed the bug. There were a couple spots on my normal driving routes where the car would inexplicably swerve. It happened one time in each spot and that was enough. Both those spots have been fixed since, but there’s no way I’d be on my phone not paying attention driving through there. I’m still cautious. There are two more spots where the car will brake to 45 mph on the highway and then speed back up after a few hundred feet. I am always on high alert around there and usually won’t even use autopilot in those areas.
- microtherion 7y ago
  > The only thing I can come up with is he wasn’t paying attention to the drive up to that point and didn’t notice he was in a problem area. It's a well known phenomenon that the more you automate away routine tasks, the harder it is for the driver to stay alert and take over in non-routine situations. My understanding is that airplane pilots are specifically trained in strategies to avoid falling into that trap.
- harlanji 7y ago
  Tesla should pay for time spent in autopilot in these cases. It’s clearly not a finished product. That could incentivize attention paying as well.
- TheKarateKid 7y ago
  > Or maybe you test it out with your hands on the wheel ready to take over to see if they fixed the bug. Isn't holding the wheel always required now in Tesla's autopilot?
  - jm4 7y ago
    You have some flexibility to take your hands off the wheel for a bit. It nags you after a little bit and then gives you some more time to hold the wheel. If you keep your hands off for too long it will disengage and you can’t put autopilot back on until you put the car in park. You can satisfy the sensors by softly resting one hand on the wheel.

mumblemumble 7y ago

We don't need to understand it. We just need to know that, even armed with all that knowledge, he still made the decision he did, and that we should not expect others with less knowledge of how software works to be any less human.

dopamean 7y ago
This has been my takeaway from the whole situation. It says something about the exactly how safe this stuff needs to be for it to be considered reliable.
- heliodor 7y ago
  My impression is that it's already more reliable than humans. Not perfect, but it probably averages better than all the crashes humans are causing.
  - Reelin 7y ago
    > it's already more reliable than humans Only when comparing aggregate statistics. Not all humans are equally (un)safe drivers; insurance rates vary based on driving record for good reason.
Drew_ 7y ago
If anything I would expect less knowledgeable people to be more skeptical and cautious than somebody who has a vested interest and passion for technology. I can't imagine anybody other than a tech enthusiast who would maintain trust in a self driving car technology that previously attempted to veer them into a barrier.
- riversflow 7y ago
  100% agree with this. Most people I’ve talked to about this incident think the guy is an idiot; if you had found a car’s feature to be unsafe on a portion of your commute before, why on earth would you trust it with your and other’s lives? If you had a new fancy belaying device slip in the gym, you wouldn’t use it on your next big wall climb. I also don’t get how everyone is forgiving him for being on his phone, in a construction zone no less. Reckless driving is reckless driving, being an Apple Engineer and Tesla owner doesn’t somehow negate that he was being a belligerent driver.
  - mumblemumble 7y ago
    It's not about forgiving him for being an idiot. It's more about recognizing that there's more than just the one idiot on the road. To an approximation, pretty much everyone who has ever had a driver's license has been guilty of making an idiotic decision or two while behind the wheel. And, Autopilot being a technology that, among other things, enables (or even encourages) idiotic behavior, there's real risk that placing too much blame on the driver's choices lulls us into an attitude that enables the next idiot to kill themselves and/or someone else.
    - riversflow 7y ago
      I agree that everyone makes bad choices from time to time on the road. But as a society we put 100% of the blame on the driver if they aren’t doing everything in there power to prevent an accident. “Everyone makes mistakes” isn’t a get out of jail free card, booze clouds people’s judgement to the point where after enough drinks they’ll think they are good to drive, but that doesn’t change anything if they get behind the wheel, its still a DUI. This guy was on his phone in a construction zone and crashed because of his lack of intervention, using a driving assist feature doesn’t somehow absolve him from being so preoccupied that his vehicle veered off the road and into a wall. Imagine if instead of him dying he had killed a construction worker; I have no doubt a jury would find him guilty of manslaughter. When you get into the driver’s seat of a car you are taking on responsibility for a death machine. I find it troubling that this conversation is happening at all, the blame should be put squarely on his shoulders. Iff the car suddenly slammed the wheel to the side causing him to lose control or became unresponsive to his inputs, that would be another matter. But this could have been prevented if he wasn’t being grossly negligent of the risk he was partaking in behind the wheel.
StevenRayOrr 7y ago
> We just need to know that, even armed with all that knowledge, he still made the decision he did, and that we should not expect others with less knowledge of how software works to be any less human. This is maybe one of the most important lessons of the 20th and 21st centuries (at least thus far): knowledge does not automatically prevent us from errors in judgments nor does it necessarily protect us from misfortune.
- cmurf 7y ago
  Nor does it absolve others from liability.
- phkahler 7y ago
  knowledge does not automatically prevent us from errors in judgments nor does it necessarily protect us from misfortune. Relevant xkcd https://xkcd.com/795/ https://xkcd.com/795/
  - mark-r 7y ago
    Not automatically, but it can help. I was in this very situation at the top of Pikes Peak. A storm moved in and the park rangers closed the place and sent everybody away. They knew the statistics of being hit by lightning in that very place.
- ChipSkylark 7y ago
  My friends like to talk about how awesome it would be to time travel or know what the outcome of a future decision will be. Even when armed with knowledge from observed outcomes of near-identical setup scenarios, majority of the time they end up doing exactly what they would have normally done. Happens to me from time to time.
troyvit 7y ago
Maybe we should keep it the way and use it as a culling method.

ajross 7y ago

It was a mistake, people make them all the time. Mistakes in cars kill people, and you don't need to be driving a Tesla to see that. Frankly the whole point of automated control is to reduce this kind of mistake fatality, and... I mean, it's working. This was a tragedy for sure, but it was also fully two years ago. These events haven't been recurring, it's likely the specific proximate causes have been addressed, and by all reckoning these systems (while still not flawless!) are at or above parity with alert human drivers in terms of safety. Basically, I read the same facts you do and take the opposite conclusion. People make bad risk/reward decisions all the time, so we need to take them out of the loop to the extent practical.

dahdum 7y ago

His phone was using data, that’s not proof he was using it and distracted. Spotify streaming a new playlist could be responsible for that.

dkonofalski 7y ago
> Recovered phone logs show that a strategy game called “Three Kingdoms” was “active during the driver’s trip to work,” the NTSB said. Investigators said the log data “does not provide enough information to ascertain” whether Huang “was holding the phone or how interactive he was,” though it said “most players have both hands on the phone to support the device and manipulate game actions.” Huang’s data usage was “consistent” with online game activity “about the time of the crash,” according to the NTSB.
pier25 7y ago
Maybe he wasn't using his phone but he was distracted or maybe even asleep. Otherwise why let the car crash?

jhallenworld 7y ago

Maybe he was fulfilling Apple's demands: either attend a meeting on the phone while driving, or wake up earlier. (Definitely not singling out Apple here.. at IBM I had a coworker who was in an accident during a phone meeting- luckily non-fatal).

jerkstate 7y ago
I have coworkers who regularly take meetings or text chat/email while driving. It makes me uncomfortable. I have tried hinting that it's a bad idea, but don't feel comfortable telling them directly (don't think it would do anything except harm our relationship due to me questioning their judgement) or reporting to HR (fear of retaliation, some of the coworkers are senior to me in the management chain). On the other hand my discomfort kind of makes me feel like a busybody. 99 times out of 100 nothing will happen and the 100th time will probably just be a fenderbender. Not sure what to do except continue feeling uncomfortable with it.
- dave5104 7y ago
  "Are you driving right now? We can reschedule to something more convenient for you? I don't want you to get pulled over."
  - deleted 7y ago
    [deleted]
- copperx 7y ago
  You're way too cautious in your human interactions, which is terribly sad. But I understand it because of our business culture.
- sitkack 7y ago
  We had this PM who ran our meetings, when people phoned in from the road she would say, "please take your calls while your aren't driving" and boot them from the meeting. Now I do the same thing and you should to.
7y ago

6gvONxR4sf7o 7y ago

> ...but still thought it best to be on his phone, while driving 71 mph, at the mercy of AutoPilot? > Records from an iPhone recovered from the crash site showed that Huang may have been using it before the accident. Records obtained from AT&T showed that data had been used while the vehicle was in motion, but the source of the transmissions couldn't be determined, the NTSB wrote. One transmission was less than one minute before the crash. For all we know, that could mean he had spotify on. Anyhow, "he was worried about it" is no reason to shift the blame to him.

dkonofalski 7y ago
I disagree. He was informed enough about the technology and aware enough of its limitations to write extensively about it. Tesla and all other assisted driving technologies all give the user a warning to stay alert and maintain control of the vehicle. He was clearly distracted somehow because he was relying on the technology. Also: > Recovered phone logs show that a strategy game called “Three Kingdoms” was “active during the driver’s trip to work,” the NTSB said. Investigators said the log data “does not provide enough information to ascertain” whether Huang “was holding the phone or how interactive he was,” though it said “most players have both hands on the phone to support the device and manipulate game actions.” Huang’s data usage was “consistent” with online game activity “about the time of the crash,” according to the NTSB.

yumraj 7y ago

Why was Autopilot driving above the speed limit? Or does Tesla allow you to set the speed?

dkonofalski 7y ago
It allows you to set the speed but only up to 8 mph above the posted speed limit (or something like that).
- erulabs 7y ago
  As far as I know my model 3 at least doesn't limit the auto-pilot speed other than the max which is 90MPH. It's just a scroll wheel, you can set it however you like.
  - dkonofalski 7y ago
    I never used FSD but AP never let me go above 55mph in a 45mph zone when it was using traffic-aware driving.

Balanceinfinity 7y ago

He was an engineer - he probably wanted to try and "solve" the problem (isolate the car's behavior and see what inputs are causing the incorrect reaction) and engineer a solution. As an engineer, I can understand this drive. He probably felt he was on top of it because he knew of the bug.

dkonofalski 7y ago
I can understand that drive too. I cannot understand allowing a technology that you know to be unsafe take over completely so that I can play a game on my phone.

jm4 7y ago

That’s the part I don’t get either. The only thing I can come up with is he wasn’t paying attention to the drive up to that point and didn’t notice he was in a problem area. I have a Tesla and there are definitely problem areas. You learn them fairly quickly when you are taking the same route all the time and you’re trained to either turn off autopilot or at least be alert when going through those areas. Or maybe you test it out with your hands on the wheel ready to take over to see if they fixed the bug. There were a couple spots on my normal driving routes where the car would inexplicably swerve. It happened one time in each spot and that was enough. Both those spots have been fixed since, but there’s no way I’d be on my phone not paying attention driving through there. I’m still cautious. There are two more spots where the car will brake to 45 mph on the highway and then speed back up after a few hundred feet. I am always on high alert around there and usually won’t even use autopilot in those areas.

microtherion 7y ago
> The only thing I can come up with is he wasn’t paying attention to the drive up to that point and didn’t notice he was in a problem area. It's a well known phenomenon that the more you automate away routine tasks, the harder it is for the driver to stay alert and take over in non-routine situations. My understanding is that airplane pilots are specifically trained in strategies to avoid falling into that trap.
harlanji 7y ago
Tesla should pay for time spent in autopilot in these cases. It’s clearly not a finished product. That could incentivize attention paying as well.
TheKarateKid 7y ago
> Or maybe you test it out with your hands on the wheel ready to take over to see if they fixed the bug. Isn't holding the wheel always required now in Tesla's autopilot?
- jm4 7y ago
  You have some flexibility to take your hands off the wheel for a bit. It nags you after a little bit and then gives you some more time to hold the wheel. If you keep your hands off for too long it will disengage and you can’t put autopilot back on until you put the car in park. You can satisfy the sensors by softly resting one hand on the wheel.