6 ms·

Read the 'updated' version, it explains that the user was doing nearly everything they shouldn't regarding this feature. http://www.theverge.com/2016/5/11/11658

Read the 'updated' version, it explains that the user was doing nearly everything they shouldn't regarding this feature. http://www.theverge.com/2016/5/11/11658226/tesla-model-s-summon-autopilot-crash-letter http://www.theverge.com/2016/5/11/11658226/tesla-model-s-sum... That said, sure it should be able to stop on it's own, but I think they couldn't have been more clear that this is beta and the driver is still always responsible. In my view the driver is just as liable if the put on cruise control and don't pay attention. Is it the manufacturer's fault the car slammed into a vehicle in front of them while cruise control was on? No, I think any reasonable person will be saying it's the driver's fault.

ececconi 10y ago
I think the liability shifts a little bit because when you're in cruise control you're literally behind the steering wheel. In this case you're outside the vehicle. I think as more autonomous features get developed it's going to be complicated for some time regarding who is culpable for a given accident.
- mywittyname 10y ago
  My personal opinion is that it's your property and you're responsible for it no matter what. To me, it's not really any different than a person's dog biting another person or a tree falling on his neighbor's house. It's likely the owner did not intent for these events to happen, but they did and the owner should be held liable for it.
  - chc 10y ago
    > My personal opinion is that it's your property and you're responsible for it no matter what. I'm sure you don't mean that as absolutely as you're making it sound. For example, if I steal your car and run over somebody with it, surely that's my responsibility, not yours. Right?
- justina1 10y ago
  Even outside the vehicle you're still in control. By default, Summon can only be used with the mobile app and with a 'dead man switch': lift your finger off the button in the app and the car stops. The driver had to specifically disable that protection to use the feature the way he did, and now claims no responsibility. Also, pressing any button on the key stops the car. Seems like a whole host of bad decisions.
  - CydeWeys 10y ago
    > The driver had to specifically disable that protection to use the feature the way he did It shouldn't be possible to disable safety features. I can't press a button to disable the brakes on my car, for instance.
    - kamaal 10y ago
      >>It shouldn't be possible to disable safety features. Here really it needs to be defined what qualifies as safety. The act of driving(in certain circumstances) in itself is unsafe by many a definition, by relinquishing control to the driver you open all the risks that likely to occur. Its hard to draw the boundary as to what is safe and what isn't. Please tell me should cars carry mandatory breath analyzers? and disable ignition in case the analyzer finds traces of alcohol in the breath? Should there be a detector to check if the driver at the wheel has been well rested a night before? A detector to check if the driver isn't in rage? These scenarios only increase exponentially, but remember when you relinquish control to the driver you now open door for all risks equally. Therefore you can't handle all the thousands of situations, instead what you do is remind the user of the responsibility, double check the user decision and then relinquish the control to the user. Therefore in all seriousness, if you take control of the car regardless of the situation you are really responsible. Even if you control the car through your phone.
      - CydeWeys 10y ago
        There's a qualitative difference between allowing disabling of a safety feature (what I'm talking about) and adding additional ones (every example you just mentioned). In that light, do you have any response to my statement that it shouldn't be possible to disable safety features? Note that requiring the addition of other safety features is a separate issue.
sjm-lbm 10y ago
It's funny, I honestly had an entirely opposite reaction to that letter. It seems like a reasonable thing for the software developers to look at to confirm the systems were working as specified, but leaves me with a ton of questions about how this feature was designed overall. It sounds like the only two mistakes that the guy made were to make a double press of the park button (which, as far as mistakes go, isn't the most unreasonable thing to do), and to assume that a car told to be in park would, well, be in park. He ignored warnings, yes, but he was likely worried about getting out of the car and doing other things by that point, which isn't wildly unreasonable either.
- freehunter 10y ago
  Summon mode is not turned on by default. That's the damning thing here. He turned on the ability to use Summon mode by hand, purposefully. At that point he should know the responsibility that comes with that. That's directly akin to manually disabling traction control on a normal car. At that point, you can't blame the manufacturer for losing control. You held the button down for three seconds and it dinged and the dashboard light came on and you KNEW that it would disable traction control. He enabled Summon mode through the menu, then either accidentally or on purpose triggered Summon mode, then stepped out of the car and the car drove off. He shouldn't have assumed the car was in park when he knew that hitting the park button twice would activate this beta software that he had to manually enable in the first place. He used Summon mode on purpose and didn't pay attention to its limitations or the rules saying "only use this on private property". The only blame Tesla has is selling a car to an irresponsible driver.
  - mikeash 10y ago
    I'm pretty sure he had to turn on two separate settings, both with their own warnings. There's one setting to enable Summon, and then there's another setting to disable the requirement for "continuous press," since by default Summon only operates in a dead-man's-switch mode with the driver's finger on a button in the phone app. I do think that the double-click Park feature could use an extra confirmation step. It already pops up a window asking whether you want to go forward or backward, and all they need to do is make it so you have to actively select one, not default it to forward as they currently do.
ChuckMcM 10y ago
Interesting reading the follow up. This really really makes you wonder about the motives of the (non)driver. I agree with the various safety folks that the existing anti-collision features should take precedence over the summon feature so if nothing else I hope someone is back there re-ordering their subsumption behaviors to effect that. Tesla has a big target painted on its back moving as fast as it is, and people will take advantage of that. This smells of that sort of thing but one can never know without being there. It looks pretty clear this person isn't going to get any sympathy from Tesla.
- kamaal 10y ago
  >> I agree with the various safety folks that the existing anti-collision features should take precedence over the summon feature Well actually this is like executing something as a sudo user. The question of safety precedence doesn't arise because you have explicitly asked for it to be disabled. Now complaining that safety features should have still taken precedence is naive, its actually more like trying to dump the blame on somebody else's for what is very clearly your mistake.
sschueller 10y ago
I think it also depends on how you market and name features. Driver assist vs auto pilot, although the same feature they have different meanings to the general public and drivers may have the wrong expectations.
falcolas 10y ago
I'm reminded of the IT security talk (Ugly bags of water), where the first dozen or so slides were about how the automotive industry had to make a lot of changes to account for human's error proneness. I guess now that Software is sneaking back into automobiles, we're going to shift back into "blame the user" mode? "But he used the feature wrong!" - To which I say, why was he able to use the feature the wrong way in the first place? Why are there so many limitations (won't sense a bike; won't sense a partially opened garage door) on a feature designed and advertised as "hands off summoning of the vehicle"?
- sokoloff 10y ago
  Reference to the discussion page for that talk: https://news.ycombinator.com/item?id=11652940 https://news.ycombinator.com/item?id=11652940
serge2k 10y ago
> Read the 'updated' version, it explains that the user was doing nearly everything they shouldn't regarding this feature Should it matter? This should be a safety critical operation. User error should mean it fails safely.
taneq 10y ago
It sounds like Tesla are technically on the right side of their own usage instructions... but those instructions /stink/. It's like the user manual for a microwave oven saying it "must never be used unattended" and "the user is responsible for shutting off the oven if it fails to stop when the timer reaches zero".
- silverevlis 10y ago
  and "do not microwave metal".
  - taneq 10y ago
    Eeeh I dunno, that's more like someone trying to take their P90D up a 4WD track because it has four driven wheels. Now, if it had a "low range" setting on its gear selector, then that would be setting up an expectation that it'd handle it.
- gruez 10y ago
  The same could be said of stoves, but we seem to be totally okay with that
  - taneq 10y ago
    (Whoops, I meant to post this last night but apparently forgot.) Stoves don't advertise any degree of autonomy. You turn the tap, fire comes out. You put things in the fire, they get hot and/or they burn. It doesn't deliver any safeguards but it doesn't promise any. The other end of the scale (if we're talking about gas-burning apparatus) would be your instant gas hot water system. It's got a series of belt-and-suspenders safety devices in it to stop it from overheating, exploding, or gassing you out if the pilot light goes out. It absolutely does advertise that it's an automated system which will do its thing without your intervention - and if it fails to do its thing, the results can be disastrous. So it's designed to be safe. The problem lies when you have devices in the middle of the scale, with devices that are designed to have some degree of autonomy but lack the commitment to see it through. Take my drip coffee maker, for instance - the manual firmly admonishes you to "never run the machine unattended" on one page and then on the next, explains how the easy-to-use timer system allows you to have fresh coffee waiting for you when you wake up. The intent is clearly that the system be used without supervision, but instead of engineering in the appropriate protections (and tripling the price) the manufacturer just put some ass-covering lines in the manual in the hope that if there's a malfunction and a house burns down, that protects them from a lawsuit.

ececconi 10y ago

I think the liability shifts a little bit because when you're in cruise control you're literally behind the steering wheel. In this case you're outside the vehicle. I think as more autonomous features get developed it's going to be complicated for some time regarding who is culpable for a given accident.

mywittyname 10y ago
My personal opinion is that it's your property and you're responsible for it no matter what. To me, it's not really any different than a person's dog biting another person or a tree falling on his neighbor's house. It's likely the owner did not intent for these events to happen, but they did and the owner should be held liable for it.
- chc 10y ago
  > My personal opinion is that it's your property and you're responsible for it no matter what. I'm sure you don't mean that as absolutely as you're making it sound. For example, if I steal your car and run over somebody with it, surely that's my responsibility, not yours. Right?
justina1 10y ago
Even outside the vehicle you're still in control. By default, Summon can only be used with the mobile app and with a 'dead man switch': lift your finger off the button in the app and the car stops. The driver had to specifically disable that protection to use the feature the way he did, and now claims no responsibility. Also, pressing any button on the key stops the car. Seems like a whole host of bad decisions.
- CydeWeys 10y ago
  > The driver had to specifically disable that protection to use the feature the way he did It shouldn't be possible to disable safety features. I can't press a button to disable the brakes on my car, for instance.
  - kamaal 10y ago
    >>It shouldn't be possible to disable safety features. Here really it needs to be defined what qualifies as safety. The act of driving(in certain circumstances) in itself is unsafe by many a definition, by relinquishing control to the driver you open all the risks that likely to occur. Its hard to draw the boundary as to what is safe and what isn't. Please tell me should cars carry mandatory breath analyzers? and disable ignition in case the analyzer finds traces of alcohol in the breath? Should there be a detector to check if the driver at the wheel has been well rested a night before? A detector to check if the driver isn't in rage? These scenarios only increase exponentially, but remember when you relinquish control to the driver you now open door for all risks equally. Therefore you can't handle all the thousands of situations, instead what you do is remind the user of the responsibility, double check the user decision and then relinquish the control to the user. Therefore in all seriousness, if you take control of the car regardless of the situation you are really responsible. Even if you control the car through your phone.
    - CydeWeys 10y ago
      There's a qualitative difference between allowing disabling of a safety feature (what I'm talking about) and adding additional ones (every example you just mentioned). In that light, do you have any response to my statement that it shouldn't be possible to disable safety features? Note that requiring the addition of other safety features is a separate issue.

sjm-lbm 10y ago

It's funny, I honestly had an entirely opposite reaction to that letter. It seems like a reasonable thing for the software developers to look at to confirm the systems were working as specified, but leaves me with a ton of questions about how this feature was designed overall. It sounds like the only two mistakes that the guy made were to make a double press of the park button (which, as far as mistakes go, isn't the most unreasonable thing to do), and to assume that a car told to be in park would, well, be in park. He ignored warnings, yes, but he was likely worried about getting out of the car and doing other things by that point, which isn't wildly unreasonable either.

freehunter 10y ago
Summon mode is not turned on by default. That's the damning thing here. He turned on the ability to use Summon mode by hand, purposefully. At that point he should know the responsibility that comes with that. That's directly akin to manually disabling traction control on a normal car. At that point, you can't blame the manufacturer for losing control. You held the button down for three seconds and it dinged and the dashboard light came on and you KNEW that it would disable traction control. He enabled Summon mode through the menu, then either accidentally or on purpose triggered Summon mode, then stepped out of the car and the car drove off. He shouldn't have assumed the car was in park when he knew that hitting the park button twice would activate this beta software that he had to manually enable in the first place. He used Summon mode on purpose and didn't pay attention to its limitations or the rules saying "only use this on private property". The only blame Tesla has is selling a car to an irresponsible driver.
- mikeash 10y ago
  I'm pretty sure he had to turn on two separate settings, both with their own warnings. There's one setting to enable Summon, and then there's another setting to disable the requirement for "continuous press," since by default Summon only operates in a dead-man's-switch mode with the driver's finger on a button in the phone app. I do think that the double-click Park feature could use an extra confirmation step. It already pops up a window asking whether you want to go forward or backward, and all they need to do is make it so you have to actively select one, not default it to forward as they currently do.

ChuckMcM 10y ago

Interesting reading the follow up. This really really makes you wonder about the motives of the (non)driver. I agree with the various safety folks that the existing anti-collision features should take precedence over the summon feature so if nothing else I hope someone is back there re-ordering their subsumption behaviors to effect that. Tesla has a big target painted on its back moving as fast as it is, and people will take advantage of that. This smells of that sort of thing but one can never know without being there. It looks pretty clear this person isn't going to get any sympathy from Tesla.

kamaal 10y ago
>> I agree with the various safety folks that the existing anti-collision features should take precedence over the summon feature Well actually this is like executing something as a sudo user. The question of safety precedence doesn't arise because you have explicitly asked for it to be disabled. Now complaining that safety features should have still taken precedence is naive, its actually more like trying to dump the blame on somebody else's for what is very clearly your mistake.

sschueller 10y ago

I think it also depends on how you market and name features. Driver assist vs auto pilot, although the same feature they have different meanings to the general public and drivers may have the wrong expectations.

falcolas 10y ago

I'm reminded of the IT security talk (Ugly bags of water), where the first dozen or so slides were about how the automotive industry had to make a lot of changes to account for human's error proneness. I guess now that Software is sneaking back into automobiles, we're going to shift back into "blame the user" mode? "But he used the feature wrong!" - To which I say, why was he able to use the feature the wrong way in the first place? Why are there so many limitations (won't sense a bike; won't sense a partially opened garage door) on a feature designed and advertised as "hands off summoning of the vehicle"?

sokoloff 10y ago
Reference to the discussion page for that talk: https://news.ycombinator.com/item?id=11652940 https://news.ycombinator.com/item?id=11652940

serge2k 10y ago

> Read the 'updated' version, it explains that the user was doing nearly everything they shouldn't regarding this feature Should it matter? This should be a safety critical operation. User error should mean it fails safely.

taneq 10y ago

It sounds like Tesla are technically on the right side of their own usage instructions... but those instructions /stink/. It's like the user manual for a microwave oven saying it "must never be used unattended" and "the user is responsible for shutting off the oven if it fails to stop when the timer reaches zero".

silverevlis 10y ago
and "do not microwave metal".
- taneq 10y ago
  Eeeh I dunno, that's more like someone trying to take their P90D up a 4WD track because it has four driven wheels. Now, if it had a "low range" setting on its gear selector, then that would be setting up an expectation that it'd handle it.
gruez 10y ago
The same could be said of stoves, but we seem to be totally okay with that
- taneq 10y ago
  (Whoops, I meant to post this last night but apparently forgot.) Stoves don't advertise any degree of autonomy. You turn the tap, fire comes out. You put things in the fire, they get hot and/or they burn. It doesn't deliver any safeguards but it doesn't promise any. The other end of the scale (if we're talking about gas-burning apparatus) would be your instant gas hot water system. It's got a series of belt-and-suspenders safety devices in it to stop it from overheating, exploding, or gassing you out if the pilot light goes out. It absolutely does advertise that it's an automated system which will do its thing without your intervention - and if it fails to do its thing, the results can be disastrous. So it's designed to be safe. The problem lies when you have devices in the middle of the scale, with devices that are designed to have some degree of autonomy but lack the commitment to see it through. Take my drip coffee maker, for instance - the manual firmly admonishes you to "never run the machine unattended" on one page and then on the next, explains how the easy-to-use timer system allows you to have fresh coffee waiting for you when you wake up. The intent is clearly that the system be used without supervision, but instead of engineering in the appropriate protections (and tripling the price) the manufacturer just put some ass-covering lines in the manual in the hope that if there's a malfunction and a house burns down, that protects them from a lawsuit.