4 ms·

Eventually we need the NTSB to certify updates before they are pushed out over the air. Similar to what the FAA does. These OTA updates are not ok for large ma

Eventually we need the NTSB to certify updates before they are pushed out over the air. Similar to what the FAA does. These OTA updates are not ok for large machinery and endagers not only the Tesla driver but all others on the road.

x38iq84n 8y ago
Indeed. I have never understood the cheering for OTA of something that can literally kill you. Were they limited to infotainment and the useless easter eggs I would not care, however AP OTA is downright scary, especially pushed by vendor instead of being initiated by an informed user.
- 0815test 8y ago
  The AP can only kill you if you're distracted/asleep at the wheel and not paying attention. You're always in charge of driving the car appropriately, the "AP" can only provide tentative assistance of a very rough sort. It will try to guess at the control inputs you might want to provide (which does make things a bit more comfortable in the best case) but the actual control always rests with the human driver.
  - saagarjha 8y ago
    The issue is that the way the feature is designed and marketed makes it very easy for the driver to be distracted and not paying attention.
  - craftyguy 8y ago
    It shouldn't be marketed as 'Autopilot' then. Less informed people will expect it to do more than it can. It's (IMHO, criminally) misleading to market something as being capable of doing something it cannot do.
  - lisper 8y ago
    > The AP can only kill you if you're distracted/asleep at the wheel and not paying attention. That's not true. There are many situations where the AP could make a sudden turn and kill you long before you had time to react even if you were paying attention. And in fact, the situation in the video seem to be not so far away from that.
    - 0815test 8y ago
      Make a sudden turn? It's just lane following, so I really doubt that. And if your hands are on the wheel (as they should be) you'll very quickly become aware that the car is not staying in the lane as it's supposed to, and be in the best position to deliver the right input. Even looking at this video, the car didn't seem to "turn" suddenly and unexpectedly; if anything, it failed to physically take a turn even though it should have. Which is quite bad, but it's not startling to the point where a driver would be in trouble.
      - lisper 8y ago
        > Make a sudden turn? It's just lane following You're begging the question [1]. The autopilot has full control authority over the steering wheel, so if it fails, nothing constrains it from making a sudden turn. If it is "just lane following" then it hasn't failed (yet). [1] https://en.wikipedia.org/wiki/Begging_the_question https://en.wikipedia.org/wiki/Begging_the_question
        Karunamon 8y ago
        Your hands. If you're just resting them on the wheel, that's not enough to prevent the attention monitor from working, it actually has to measure some physical resistance. If you resist the automatic inputs too much, the AP cuts out. You can test this yourself in a tesla by engaging cruise control, then hitting a turn signal. This would normally initiate an automatic lane change - but keep your hands tightly on the wheel as if you wanted to stay in the lane you're in. The wheel will attempt to turn, fail as you're preventing it from turning - and the AP disables.
  - qball 8y ago
    > You're always in charge of driving the car appropriately, the "AP" can only provide tentative assistance of a very rough sort. It can "tentatively assist" you in killing yourself and potentially others, but we already know that from the disaster in Mountain View, where this exact same thing happened. The problem with behavior changes in autopilot is that the car needs to react the same way every time. If you're going around a corner on a road which you've driven on for years without issue, and the car all of a sudden does something unexpected, the panic and over-correction reaction that everyone instinctively has tends to cause more accidents than just holding your desired course does. It doesn't matter if it's a skid, or your Tesla trying to accelerate you into a wall at 70 miles an hour. If the car does something the driver is not expecting it to do for any reason (from fatally defective software to ice on the ground) the driver performs sub-optimally as a result. And since this is Tesla we're talking about (who bakes in features like your car not starting before you upgrade its software, which is another "feature" about these cars that's going to get someone killed sooner or later), I'm willing to bet that the car doesn't warn you that this might occur. It just works fine for 6 months, then an update gets pushed, the car tries to put you into a wall, and you cause an accident because you're trying to stop the car from killing you and didn't check your blind spot before that. That is a UI and UX failure of the highest magnitude, and it is completely unacceptable, no matter how well it otherwise tends to work.
- DarmokJalad1701 8y ago
  It is initiated by the user ... the update only happens after you give say it's okay to do so.
  - javagram 8y ago
    According to the posters on Reddit, this update’s release notes were about “Dog Mode” and “Sentry Mode”. Nothing warning that behavior of autopilot might change.
kitsunesoba 8y ago
I agree, but is there a way that could happen without slowing the process to a crawl? Depending on what’s involved it could easily push the gap between updates from months to years.
- toomuchtodo 8y ago
  This is a feature, not a bug, and should be expected in life critical systems. Would you want Boeing to push updates out as frequently as Tesla does with the same sparse release notes Tesla provides (“bug fixes”) when safety system functionality is modified? Disclaimer: I own a Model S
  - anth_anm 8y ago
    Imagine if Boeing got away with just pushing a software update to the 737 MAX8 and saying "it's fixed now".
    - CamperBob2 8y ago
      Well, it might have prevented the second crash if they had treated the matter with a bit more urgency. Depends on whether the two incidents really had a common cause, which is looking like the case. Of course it's also looking like they should have grounded the fleet after the first crash, given the history of the aircraft prior to its last flight.
      - deleted 8y ago
        [deleted]
  - CamperBob2 8y ago
    This argument would be a bit different if we were losing 30,000 people a year in airliner crashes, instead of approximately zero.
    - toomuchtodo 8y ago
Mizza 8y ago
Not eventually - now. This is putting everybody in danger, not just the owners of the cars.
quasse 8y ago
Yeah, it's a pretty crazy world where a game developer pushing an update to a game on Xbox live seems to have a harsher approval process to go through than an automaker pushing OTA updates that literally drive cars.
alistairSH 8y ago
OTA is fine. It's unregulated, unannounced OTA that's problematic. I'd love for my VW to get updates OTA without taking it to the dealer. But, I don't want to receive those updates without knowledge that the update has been tested sufficiently (and given I don't trust the vendor, I'd like NTSB or similar government body to do this on my behalf).
deleted 8y ago
[deleted]
deleted 8y ago
[deleted]
dragonwriter 8y ago
> Eventually we need the NTSB to certify updates before they are pushed out over the air. Similar to what the FAA does. You mean, similar to what FAA might do if it didn't allow manufucturer self-certification instead, right? Also, wouldn't NHSTA, which does safety regulation and standards for autos, not NTSB, that does accident investigation, be the natural agency?
smileysteve 8y ago
> Similar to what the FAA does. The timing of this is precarious (given the recent 737 Max allegations)
Traster 8y ago
Having the NTSB certify updates isn't going to increase safety. The accepted approach to safety as set in ISO 26262 largely focuses on making sure that the processes in place via which the software and hardware is designed, created and modified. The reason you wouldn't get regression if you were interested in FuSa is because you'd have a process in place to ensure that software can't be distributed without being tested and a process to ensure that bugs are included in the test suite. It's quite clear in this case that Tesla doesn't have an organisational structure that fulfills the requirements for functional safety.
- sangnoir 8y ago
  > Having the NTSB certify updates isn't going to increase safety. It will if they run it through regression tests that Tesla doesn't seem to have the discipline to.

x38iq84n 8y ago

Indeed. I have never understood the cheering for OTA of something that can literally kill you. Were they limited to infotainment and the useless easter eggs I would not care, however AP OTA is downright scary, especially pushed by vendor instead of being initiated by an informed user.

0815test 8y ago
The AP can only kill you if you're distracted/asleep at the wheel and not paying attention. You're always in charge of driving the car appropriately, the "AP" can only provide tentative assistance of a very rough sort. It will try to guess at the control inputs you might want to provide (which does make things a bit more comfortable in the best case) but the actual control always rests with the human driver.
- saagarjha 8y ago
  The issue is that the way the feature is designed and marketed makes it very easy for the driver to be distracted and not paying attention.
- craftyguy 8y ago
  It shouldn't be marketed as 'Autopilot' then. Less informed people will expect it to do more than it can. It's (IMHO, criminally) misleading to market something as being capable of doing something it cannot do.
- lisper 8y ago
  > The AP can only kill you if you're distracted/asleep at the wheel and not paying attention. That's not true. There are many situations where the AP could make a sudden turn and kill you long before you had time to react even if you were paying attention. And in fact, the situation in the video seem to be not so far away from that.
  - 0815test 8y ago
    Make a sudden turn? It's just lane following, so I really doubt that. And if your hands are on the wheel (as they should be) you'll very quickly become aware that the car is not staying in the lane as it's supposed to, and be in the best position to deliver the right input. Even looking at this video, the car didn't seem to "turn" suddenly and unexpectedly; if anything, it failed to physically take a turn even though it should have. Which is quite bad, but it's not startling to the point where a driver would be in trouble.
    - lisper 8y ago
      > Make a sudden turn? It's just lane following You're begging the question [1]. The autopilot has full control authority over the steering wheel, so if it fails, nothing constrains it from making a sudden turn. If it is "just lane following" then it hasn't failed (yet). [1] https://en.wikipedia.org/wiki/Begging_the_question https://en.wikipedia.org/wiki/Begging_the_question
      - Karunamon 8y ago
        Your hands. If you're just resting them on the wheel, that's not enough to prevent the attention monitor from working, it actually has to measure some physical resistance. If you resist the automatic inputs too much, the AP cuts out. You can test this yourself in a tesla by engaging cruise control, then hitting a turn signal. This would normally initiate an automatic lane change - but keep your hands tightly on the wheel as if you wanted to stay in the lane you're in. The wheel will attempt to turn, fail as you're preventing it from turning - and the AP disables.
- qball 8y ago
  > You're always in charge of driving the car appropriately, the "AP" can only provide tentative assistance of a very rough sort. It can "tentatively assist" you in killing yourself and potentially others, but we already know that from the disaster in Mountain View, where this exact same thing happened. The problem with behavior changes in autopilot is that the car needs to react the same way every time. If you're going around a corner on a road which you've driven on for years without issue, and the car all of a sudden does something unexpected, the panic and over-correction reaction that everyone instinctively has tends to cause more accidents than just holding your desired course does. It doesn't matter if it's a skid, or your Tesla trying to accelerate you into a wall at 70 miles an hour. If the car does something the driver is not expecting it to do for any reason (from fatally defective software to ice on the ground) the driver performs sub-optimally as a result. And since this is Tesla we're talking about (who bakes in features like your car not starting before you upgrade its software, which is another "feature" about these cars that's going to get someone killed sooner or later), I'm willing to bet that the car doesn't warn you that this might occur. It just works fine for 6 months, then an update gets pushed, the car tries to put you into a wall, and you cause an accident because you're trying to stop the car from killing you and didn't check your blind spot before that. That is a UI and UX failure of the highest magnitude, and it is completely unacceptable, no matter how well it otherwise tends to work.
DarmokJalad1701 8y ago
It is initiated by the user ... the update only happens after you give say it's okay to do so.
- javagram 8y ago
  According to the posters on Reddit, this update’s release notes were about “Dog Mode” and “Sentry Mode”. Nothing warning that behavior of autopilot might change.

kitsunesoba 8y ago

I agree, but is there a way that could happen without slowing the process to a crawl? Depending on what’s involved it could easily push the gap between updates from months to years.

toomuchtodo 8y ago
This is a feature, not a bug, and should be expected in life critical systems. Would you want Boeing to push updates out as frequently as Tesla does with the same sparse release notes Tesla provides (“bug fixes”) when safety system functionality is modified? Disclaimer: I own a Model S
- anth_anm 8y ago
  Imagine if Boeing got away with just pushing a software update to the 737 MAX8 and saying "it's fixed now".
  - CamperBob2 8y ago
    Well, it might have prevented the second crash if they had treated the matter with a bit more urgency. Depends on whether the two incidents really had a common cause, which is looking like the case. Of course it's also looking like they should have grounded the fleet after the first crash, given the history of the aircraft prior to its last flight.
    - deleted 8y ago
      [deleted]
- CamperBob2 8y ago
  This argument would be a bit different if we were losing 30,000 people a year in airliner crashes, instead of approximately zero.
  - toomuchtodo 8y ago

Mizza 8y ago

Not eventually - now. This is putting everybody in danger, not just the owners of the cars.

quasse 8y ago

Yeah, it's a pretty crazy world where a game developer pushing an update to a game on Xbox live seems to have a harsher approval process to go through than an automaker pushing OTA updates that literally drive cars.

alistairSH 8y ago

OTA is fine. It's unregulated, unannounced OTA that's problematic. I'd love for my VW to get updates OTA without taking it to the dealer. But, I don't want to receive those updates without knowledge that the update has been tested sufficiently (and given I don't trust the vendor, I'd like NTSB or similar government body to do this on my behalf).

deleted 8y ago

[deleted]

deleted 8y ago

[deleted]

dragonwriter 8y ago

> Eventually we need the NTSB to certify updates before they are pushed out over the air. Similar to what the FAA does. You mean, similar to what FAA might do if it didn't allow manufucturer self-certification instead, right? Also, wouldn't NHSTA, which does safety regulation and standards for autos, not NTSB, that does accident investigation, be the natural agency?

smileysteve 8y ago

> Similar to what the FAA does. The timing of this is precarious (given the recent 737 Max allegations)

Traster 8y ago

Having the NTSB certify updates isn't going to increase safety. The accepted approach to safety as set in ISO 26262 largely focuses on making sure that the processes in place via which the software and hardware is designed, created and modified. The reason you wouldn't get regression if you were interested in FuSa is because you'd have a process in place to ensure that software can't be distributed without being tested and a process to ensure that bugs are included in the test suite. It's quite clear in this case that Tesla doesn't have an organisational structure that fulfills the requirements for functional safety.

sangnoir 8y ago
> Having the NTSB certify updates isn't going to increase safety. It will if they run it through regression tests that Tesla doesn't seem to have the discipline to.