11 ms·

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable

And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal flying at 70mph with humans inside of it. The saying has been beat to death, but it bears repeating: Tesla is a prime case where the SV mindset of "move fast and break things" has resulted in "move fast and kill people". There's a reason that other vehicle manufacturers don't send out vehicle software updates willy-nilly, and it's not because they're technologically inferior.

slg 8y ago
This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and did a voluntary recall. That means many of those cars with buggy brake systems were on the road for years after a potential fix was available and were driven for billions of potentially unsafe miles.
- deleted 8y ago
  [deleted]
- txcwpalpha 8y ago
  >This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers understand a fault with such software could result in someone's death). Tesla seems to be the only major car manufacturer that has a problem with this. >So what is the right way to handle these updates? The way that other vehicle manufacturers (car, airplane, etc) have been doing it for decades is a pretty good way. >You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. There is no evidence that said OTA update made Tesla cars any safer. There is evidence that similar OTA updates have made Tesla cars more unsafe. The brake OTA that you mentioned has actually potentially done more harm than good. Tesla owners have been reporting that the same update made unexpected changes to the way their cars handle/accelerate in addition to the change in braking distance. These were forced, unpredictable changes that were introduced without warning. When you're driving a 2 ton vehicle at 70mph, being able to know exactly how your car will react in all situations, including how fast it accelerates, how well it handles, how fast it brakes, and how the autopilot will act is crucial to maintaining safety. Tesla messing with those parameters without warning is a detriment to safety, not an advantage.
  - slg 8y ago
    I didn't say no car has ever relied on software. I said cars are becoming more reliant on software. I don't think that is a controversial statement. I also don't think it is controversial to say that other automakers also occasionally ship buggy code. The Toyota brake issue I mentioned in the previous post is one example. Additionally, the argument that we should continue to handle updates this way simply because we have done it this way for decades is the laziest possible reasoning. It is frankly surprising to see that argument on HN of all places. As for the evidence that OTA updates can make things safer, this is from Consumer Reports: >Consumer Reports now recommends the Tesla Model 3, after our testers found that a recent over-the-air (OTA) update improved the car’s braking distance by almost 20 feet. [1] That update going out immediately OTA is going to save lives compared to if Tesla waited for the cars to be serviced like other manufacturers. I don't think you can legitimately argue against that fact. [1] - https://www.consumerreports.org/car-safety/tesla-model-3-gets-cr-recommendation-after-braking-update/ https://www.consumerreports.org/car-safety/tesla-model-3-get...
    - txcwpalpha 8y ago
      > That update going out immediately OTA is going to save lives compared to if Tesla waited for the cars to be serviced like other manufacturers. I don't think you can legitimately argue against that fact. There is again no evidence to support this fact. There is evidence that Tesla's OTA software updates have introduced safety issues with Tesla cars. That's a fact. Better braking distance is of course a good thing but if anything, the fact that Teslas were on the road for so long with a sub-par braking distance is more evidence of a problem with Tesla than it is evidence of a benefit of OTA updates. The other factor in that brake story is that it took mere days for Tesla to release an update to "fix" the brakes. This isn't a good thing. The fact that it was accomplished so quickly means that the OTA update was likely not tested very well. It also means that the issue was easy to fix, which calls into question why it wasn't fixed before. It also highlights the fact that Tesla, for some reason, failed to do the most basic testing on their own cars for braking distance. Comparing the braking distance of their cars should have been one of the very first things they did before even selling the cars, but apparently it took a third party to do that before Tesla was even aware of the issue. This doesn't inspire confidence in Tesla cars at all.
- WiseWeasel 8y ago
  Certainly communication of any changes to all drivers inexperienced with the latest version; ideally user interaction required for the update to be applied, and potentially even the ability to reverse them if they are unhappy with the changes.
- mannykannot 8y ago
  The first thing about doing it right is to make sure it has been developed in an appropriate manner for safety-critical systems, which includes, but is by no means limited to, adequate testing. The second thing is to require the owners to take some action as part of the installation procedure, so that it is hard for them to overlook the fact that it has happened. The third thing is that changes with safety implications should not be bundled with 'convenience/usability' upgrades (including those that are more of a convenience for the manufacturer than for the user.) To be fair, I am not aware of Tesla doing that, but it is a common enough practice in the software business to justify being mentioned. And it has to be done securely. Again, I am not aware of Tesla getting this wrong.
- amelius 8y ago
  > So what is the right way to handle these updates? Require updates to be sent to a government entity, which will test the code for X miles of real traffic, and then releases the updates to the cars. Of course, costs of this are to be paid by the company.
  - ekianjo 8y ago
    Yeah, because we know governments are really good at giving certifications and doing tests that mean sonething. Lets put every design decision in the hand of governements then! or better, nationalize car companies! Problem solved?
    - amelius 8y ago
      In any case, let a third party control the actual updating, so that we know when and how often cars are updated. Require at least X months of time between code submission and deployment to cars. We don't want a culture of "quick fixes".
      - stickfigure 8y ago
        This is a popular idea: Just put someone in charge! It ignores the incentives for those gatekeepers, who are now part of the system. In practice I don't think you're going to get better updates, you're going to get "almost no updates". See also: The FDA and drug approvals.
        amelius 8y ago
        So you want to do away with the FDA? Put BigPharma in charge and let them regulate themselves? That sounds a lot less appealing to me.
        wooter 8y ago
        fdareview.org - the fda kills more people in late/non approvals than it saves through denials. this is why compassionate use has been on the rise
- lttlrck 8y ago
  Great that they fixed the brakes OTA. But how exactly did the inferior braking algorithm get on the Model 3 in the first place? And what are the chances of a regression?
- LoSboccacc 8y ago
  At the very _least_ when you introduce a change in behavior have it to be enabled from the user through the dashboard. This creates at least one touch point for user education.
- pishpash 8y ago
  Give me a dumb car. The software is not ready. Get them off the road.
  - hfdgiutdryg 8y ago
    Keep them on the road. They're still safer than a huge percentage of the awful drivers that I see every day.
    - salawat 8y ago
      They really aren't. At least you can rely on 99% of humans to try to act according to self-preservation instinct MOST of the time. I see a Tesla, and I try to get away from them as soon as possible.
      - hfdgiutdryg 8y ago
        At least you can rely on 99% of humans to try to act according to self-preservation instinct MOST of the time. Nope. I see tremendous numbers of distracted drivers who don't even realize there's a threat. I also see many utterly incompetent drivers who will not take any evasive action, including braking, because they simply don't understand basic vehicle dynamics or that one needs to react to unexpected circumstances.
- Bluestrike2 8y ago
  While I like Tesla, I find the praise for Tesla's fast OTA update for its braking problem to be freaking terrifying. A problem with variable stopping distances is the sort of thing that should be blindingly obvious in the telemetry data from your testing procedures. Brake systems, and ABS controls in particular, are normally rigorously tested over the course of 12-18 months in different environments and conditions.[0] That Tesla completely missed something like that suggests either their testing procedures are drastically flawed (missing something that CR was able to easily and quickly verify in different cars), that their software development process isn't meshed up with their hardware testing and validation, or a combination of the two. Neither option is a good one. The fact that Tesla was able to shave 19 feet of their braking distances is horrifying. After months of testing different variations and changes to refine your braking systems, shaving off an extra 19 feet should be impossible. There shouldn't be any room to gain extra inches without making tradeoffs in performance in other conditions that you've already ruled out making. If there's an extra 19 feet to be found for free after a few days of dev time, you did something drastically wrong. And that's completely ignoring physical testing before pushing your new update. Code tests aren't sufficient; you're changing physical real-world behavior, and there's always a tradeoff when you're dealing with braking and traction. Tesla is being praised by consumers and the media because, hey, who doesn't like the idea that problems can be fixed a couple days after being identified? That's great. In this case, Tesla literally made people's cars better than they were just a few days before. But it trivializes a problem with very real consequences, and I hope that trivialization doesn't extend to Tesla's engineers. Instead of talking about a brake problem, people are talking about how great the fast OTA update for the problem is. Consumers find that comforting, as the OTA updates can makes what's otherwisea pain in the ass (recalls and dealer visits for software updates) effortless. Hell, I'm a believer in release early, release often for software. Users benefit, as do developers. At the same time, the knowledge that you can quickly fix a bug and push out an update can be a bit insidious. It's a bit of a double-edged sword in that it gives you a sense of comfort that can bite you in the ass as it trivializes the consequences of a bug. And when bug reports for your product can literally come in the form of coroner's reports, that comfort isn't a good thing for developers. 0. https://www.forbes.com/sites/samabuelsamid/2018/05/30/rapid-tesla-software-updates-are-great-but-hide-more-fundamental-problems/#776cd4e757de https://www.forbes.com/sites/samabuelsamid/2018/05/30/rapid-...
- TeMPOraL 8y ago
  > So what is the right way to handle these updates? Avoid doing them in the first place? It's not like bit rot is - or should be - a problem for cars. It's a problem specific to the Internet-connected software ecosystem, which a car shouldn't be a part of. So basically: develop software, test the shit out of it, then release. If you happen to find some critical problem later on that is fixable with software, by all means fix it, again test the shit out of it, and only then update. If OTA updates on cars are frequent, it means someone preferred to get to market quickly instead of building the product right. Which, again, is fine for bullshit social apps, but not fine for life-critical systems.
  - JimmyAustin 8y ago
    Tesla does test the shit out of it before they release a patch. The problem is that users expectations of the systems performance suddenly get out of sync with what the car is going to do. Part of me wonders if there should be a very quick, unskipable, animated, easy to understand explanation of the patch notes before you can drive when they make material changes to core driving functionality.
    - TeMPOraL 8y ago
      I'd vote in favor of such explanation, though this alone may not be enough to cancel out possibly thousands of hours of experience with the previous system behavior.
    - letsgetphysITal 8y ago
      While using Autopilot (Big A), there should be a loud klaxon every 30 seconds followed by a notification "CHECK ROAD CONDITIONS" and "REMAIN ENGAGED WITH DRIVING" in the same urgent tone of an aircraft autopilot (small a) warning system. Tesla did make a mistake calling it Autopilot, but only because regular folk don't understand that aircraft autopilot is literally a heading, altitude, and speed, and will not make any correction for fault. Aircraft autopilot will fly you straight into a mountain if one happens to be in the way.
      - 8y ago
- syrrim 8y ago
  Updates should fix problems not create new ones. The tried and true method for silicon valley bug fixing is to ship it to the users and let them report any issues. This is wholly insufficient for car software. Car software should seldom have bugs in the first place, but OTA updates should never bar never introduce new bugs to replace the old.
Pfhreak 8y ago
This seems testable. IANAAE (I Am Not An Automotive Engineer), but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Like, if in one case the new code would want you to move left, and the old code goes straight, one of those behaviors is probably wrong. If the driver corrects, then the new code is probably correct, but if the driver does not, then the new code is doing something probably incorrect. At the very least, you should be able to get some sort of magnitude/fuzzy understanding of how frequently the new code is disagreeing, and you can figure out where and go check out those conditions.
- jakecopp 8y ago
  A/B testing self-driving car software to find bugs? Would probably work great, but that is also terrifying! But you're right, if you're really going to do a full roll out, may as well test it on a subsegment first - I'd hate for it to be used as a debugging tool though.
- akira2501 8y ago
  > but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Because there are no closed facilities that you can use to actually perform any meaningful test. You could test "in-situ", but you would need an absolutely _huge_ testing area in order to accurately test and check all the different roadway configurations the vehicle is likely to encounter. You'll probably want more than one pass, some with pedestrians, some without, some in high light and some in low, etc.. It's worth noting that American's drive more than 260 billion miles each _month_. It's just an enormous problem.
  - dchichkov 8y ago
    This particular case might have been testable "in-situ": [ABS News Reporter] Dan Noyes also spoke and texted with Walter Huang's brother, Will, today. He confirmed Walter was on the way to work at Apple when he died. He also makes a startling claim — that before the crash, Walter complained "seven-to-10 times the car would swivel toward that same exact barrier during auto-pilot. Walter took it into dealership addressing the issue, but they couldn't duplicate it there." It is very believable that the car would swivel toward the same exact barrier on auto-pilot. BTW - I'm running a nonprofit/public dataset project aimed at increasing safety of autonomous vehicles. If anyone here wants to contribute (with suggestions / pull requests / following it on twitter / etc) - you'd be most welcome. Its: https://www.safe-av.org/ https://www.safe-av.org/
erikpukinskis 8y ago
More accurately: move fast and save some lives and kill people.
InternetOfStuff 8y ago
> And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal I recently got downvoted for that exact line of reasoning. Looks like some people don't like to hear that :-)
- bertil 8y ago
  I think you got down-voted because you misrepresented what Telsa is actually doing, which is a difficult arbitrage between: - known preventable deaths from, say, not staying in the lane aggressively enough; - possible surprises and subsequent deaths. There is an ethical conundrum (that is quite different from the trolley problem) between a known fix and an unknown possibility. If both are clearly establish and you dismiss the former to have a simplistic take, yes, you would be down-voted because you are making the debate less informed. Without falling to solutionism, in that case, the remaining issue seems rather isolated to a handful of areas that look like lanes and could be either painted properly, or that Telsa cars could train to avoid. The later fix would have to be sent rapidly and could have surprising consequences -- although it seems decreasingly likely. That learning pattern (resolving unintended surprises as they happen decreasingly often) is common in software. This explains why this community prefers (to an extend) Telsa to other manufacturers. Others have preferred the surprise-free and PR-friendly option of not saving the dozens of thousand of lives dying on the road at the moment. There are ethical backgrounds to non-interventionism. As the victim of car violence, I happen to think that their position is unethical. I’m actually at what is considered an extreme position, of being in favour of Telsa (and Waymo) taking more risk than necessary and temporarily increasing the number of accidents on the road because they have a far better track record of learning from those accidents (and the subsequent press coverage) and that would lower the overall number of deaths faster. As it happens, they don’t need to: even with less than half of accidents from their counterparts, they still get spectacular learning rate.
  - InternetOfStuff 8y ago

slg 8y ago

This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. So what is the right way to handle these updates? You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. Toyota had a similar problem a few years ago and did a voluntary recall. That means many of those cars with buggy brake systems were on the road for years after a potential fix was available and were driven for billions of potentially unsafe miles.

deleted 8y ago
[deleted]
txcwpalpha 8y ago
>This isn't an issue specific to Tesla as all automakers are now making cars that are more and more dependent on software. Cars have been dependent on software for a long time (literally decades). This isn't something new. Even combustion engine cars have had software inside of them that controls the operation of the engine, and this software is vigorously tested for safety issues (because most car manufacturers understand a fault with such software could result in someone's death). Tesla seems to be the only major car manufacturer that has a problem with this. >So what is the right way to handle these updates? The way that other vehicle manufacturers (car, airplane, etc) have been doing it for decades is a pretty good way. >You mentioned a clear flaw with OTA updates, but there are also numerous advantages. For example, the recent Tesla brake software issue was fixed with an OTA update. That immediately made cars safer. There is no evidence that said OTA update made Tesla cars any safer. There is evidence that similar OTA updates have made Tesla cars more unsafe. The brake OTA that you mentioned has actually potentially done more harm than good. Tesla owners have been reporting that the same update made unexpected changes to the way their cars handle/accelerate in addition to the change in braking distance. These were forced, unpredictable changes that were introduced without warning. When you're driving a 2 ton vehicle at 70mph, being able to know exactly how your car will react in all situations, including how fast it accelerates, how well it handles, how fast it brakes, and how the autopilot will act is crucial to maintaining safety. Tesla messing with those parameters without warning is a detriment to safety, not an advantage.
- slg 8y ago
  I didn't say no car has ever relied on software. I said cars are becoming more reliant on software. I don't think that is a controversial statement. I also don't think it is controversial to say that other automakers also occasionally ship buggy code. The Toyota brake issue I mentioned in the previous post is one example. Additionally, the argument that we should continue to handle updates this way simply because we have done it this way for decades is the laziest possible reasoning. It is frankly surprising to see that argument on HN of all places. As for the evidence that OTA updates can make things safer, this is from Consumer Reports: >Consumer Reports now recommends the Tesla Model 3, after our testers found that a recent over-the-air (OTA) update improved the car’s braking distance by almost 20 feet. [1] That update going out immediately OTA is going to save lives compared to if Tesla waited for the cars to be serviced like other manufacturers. I don't think you can legitimately argue against that fact. [1] - https://www.consumerreports.org/car-safety/tesla-model-3-gets-cr-recommendation-after-braking-update/ https://www.consumerreports.org/car-safety/tesla-model-3-get...
  - txcwpalpha 8y ago
    > That update going out immediately OTA is going to save lives compared to if Tesla waited for the cars to be serviced like other manufacturers. I don't think you can legitimately argue against that fact. There is again no evidence to support this fact. There is evidence that Tesla's OTA software updates have introduced safety issues with Tesla cars. That's a fact. Better braking distance is of course a good thing but if anything, the fact that Teslas were on the road for so long with a sub-par braking distance is more evidence of a problem with Tesla than it is evidence of a benefit of OTA updates. The other factor in that brake story is that it took mere days for Tesla to release an update to "fix" the brakes. This isn't a good thing. The fact that it was accomplished so quickly means that the OTA update was likely not tested very well. It also means that the issue was easy to fix, which calls into question why it wasn't fixed before. It also highlights the fact that Tesla, for some reason, failed to do the most basic testing on their own cars for braking distance. Comparing the braking distance of their cars should have been one of the very first things they did before even selling the cars, but apparently it took a third party to do that before Tesla was even aware of the issue. This doesn't inspire confidence in Tesla cars at all.
WiseWeasel 8y ago
Certainly communication of any changes to all drivers inexperienced with the latest version; ideally user interaction required for the update to be applied, and potentially even the ability to reverse them if they are unhappy with the changes.
mannykannot 8y ago
The first thing about doing it right is to make sure it has been developed in an appropriate manner for safety-critical systems, which includes, but is by no means limited to, adequate testing. The second thing is to require the owners to take some action as part of the installation procedure, so that it is hard for them to overlook the fact that it has happened. The third thing is that changes with safety implications should not be bundled with 'convenience/usability' upgrades (including those that are more of a convenience for the manufacturer than for the user.) To be fair, I am not aware of Tesla doing that, but it is a common enough practice in the software business to justify being mentioned. And it has to be done securely. Again, I am not aware of Tesla getting this wrong.
amelius 8y ago
> So what is the right way to handle these updates? Require updates to be sent to a government entity, which will test the code for X miles of real traffic, and then releases the updates to the cars. Of course, costs of this are to be paid by the company.
- ekianjo 8y ago
  Yeah, because we know governments are really good at giving certifications and doing tests that mean sonething. Lets put every design decision in the hand of governements then! or better, nationalize car companies! Problem solved?
  - amelius 8y ago
    In any case, let a third party control the actual updating, so that we know when and how often cars are updated. Require at least X months of time between code submission and deployment to cars. We don't want a culture of "quick fixes".
    - stickfigure 8y ago
      This is a popular idea: Just put someone in charge! It ignores the incentives for those gatekeepers, who are now part of the system. In practice I don't think you're going to get better updates, you're going to get "almost no updates". See also: The FDA and drug approvals.
      - amelius 8y ago
        So you want to do away with the FDA? Put BigPharma in charge and let them regulate themselves? That sounds a lot less appealing to me.
        wooter 8y ago
        fdareview.org - the fda kills more people in late/non approvals than it saves through denials. this is why compassionate use has been on the rise
lttlrck 8y ago
Great that they fixed the brakes OTA. But how exactly did the inferior braking algorithm get on the Model 3 in the first place? And what are the chances of a regression?
LoSboccacc 8y ago
At the very _least_ when you introduce a change in behavior have it to be enabled from the user through the dashboard. This creates at least one touch point for user education.
pishpash 8y ago
Give me a dumb car. The software is not ready. Get them off the road.
- hfdgiutdryg 8y ago
  Keep them on the road. They're still safer than a huge percentage of the awful drivers that I see every day.
  - salawat 8y ago
    They really aren't. At least you can rely on 99% of humans to try to act according to self-preservation instinct MOST of the time. I see a Tesla, and I try to get away from them as soon as possible.
    - hfdgiutdryg 8y ago
      At least you can rely on 99% of humans to try to act according to self-preservation instinct MOST of the time. Nope. I see tremendous numbers of distracted drivers who don't even realize there's a threat. I also see many utterly incompetent drivers who will not take any evasive action, including braking, because they simply don't understand basic vehicle dynamics or that one needs to react to unexpected circumstances.
Bluestrike2 8y ago
While I like Tesla, I find the praise for Tesla's fast OTA update for its braking problem to be freaking terrifying. A problem with variable stopping distances is the sort of thing that should be blindingly obvious in the telemetry data from your testing procedures. Brake systems, and ABS controls in particular, are normally rigorously tested over the course of 12-18 months in different environments and conditions.[0] That Tesla completely missed something like that suggests either their testing procedures are drastically flawed (missing something that CR was able to easily and quickly verify in different cars), that their software development process isn't meshed up with their hardware testing and validation, or a combination of the two. Neither option is a good one. The fact that Tesla was able to shave 19 feet of their braking distances is horrifying. After months of testing different variations and changes to refine your braking systems, shaving off an extra 19 feet should be impossible. There shouldn't be any room to gain extra inches without making tradeoffs in performance in other conditions that you've already ruled out making. If there's an extra 19 feet to be found for free after a few days of dev time, you did something drastically wrong. And that's completely ignoring physical testing before pushing your new update. Code tests aren't sufficient; you're changing physical real-world behavior, and there's always a tradeoff when you're dealing with braking and traction. Tesla is being praised by consumers and the media because, hey, who doesn't like the idea that problems can be fixed a couple days after being identified? That's great. In this case, Tesla literally made people's cars better than they were just a few days before. But it trivializes a problem with very real consequences, and I hope that trivialization doesn't extend to Tesla's engineers. Instead of talking about a brake problem, people are talking about how great the fast OTA update for the problem is. Consumers find that comforting, as the OTA updates can makes what's otherwisea pain in the ass (recalls and dealer visits for software updates) effortless. Hell, I'm a believer in release early, release often for software. Users benefit, as do developers. At the same time, the knowledge that you can quickly fix a bug and push out an update can be a bit insidious. It's a bit of a double-edged sword in that it gives you a sense of comfort that can bite you in the ass as it trivializes the consequences of a bug. And when bug reports for your product can literally come in the form of coroner's reports, that comfort isn't a good thing for developers. 0. https://www.forbes.com/sites/samabuelsamid/2018/05/30/rapid-tesla-software-updates-are-great-but-hide-more-fundamental-problems/#776cd4e757de https://www.forbes.com/sites/samabuelsamid/2018/05/30/rapid-...
TeMPOraL 8y ago
> So what is the right way to handle these updates? Avoid doing them in the first place? It's not like bit rot is - or should be - a problem for cars. It's a problem specific to the Internet-connected software ecosystem, which a car shouldn't be a part of. So basically: develop software, test the shit out of it, then release. If you happen to find some critical problem later on that is fixable with software, by all means fix it, again test the shit out of it, and only then update. If OTA updates on cars are frequent, it means someone preferred to get to market quickly instead of building the product right. Which, again, is fine for bullshit social apps, but not fine for life-critical systems.
- JimmyAustin 8y ago
  Tesla does test the shit out of it before they release a patch. The problem is that users expectations of the systems performance suddenly get out of sync with what the car is going to do. Part of me wonders if there should be a very quick, unskipable, animated, easy to understand explanation of the patch notes before you can drive when they make material changes to core driving functionality.
  - TeMPOraL 8y ago
    I'd vote in favor of such explanation, though this alone may not be enough to cancel out possibly thousands of hours of experience with the previous system behavior.
  - letsgetphysITal 8y ago
    While using Autopilot (Big A), there should be a loud klaxon every 30 seconds followed by a notification "CHECK ROAD CONDITIONS" and "REMAIN ENGAGED WITH DRIVING" in the same urgent tone of an aircraft autopilot (small a) warning system. Tesla did make a mistake calling it Autopilot, but only because regular folk don't understand that aircraft autopilot is literally a heading, altitude, and speed, and will not make any correction for fault. Aircraft autopilot will fly you straight into a mountain if one happens to be in the way.
    - 8y ago
syrrim 8y ago
Updates should fix problems not create new ones. The tried and true method for silicon valley bug fixing is to ship it to the users and let them report any issues. This is wholly insufficient for car software. Car software should seldom have bugs in the first place, but OTA updates should never bar never introduce new bugs to replace the old.

Pfhreak 8y ago

This seems testable. IANAAE (I Am Not An Automotive Engineer), but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Like, if in one case the new code would want you to move left, and the old code goes straight, one of those behaviors is probably wrong. If the driver corrects, then the new code is probably correct, but if the driver does not, then the new code is doing something probably incorrect. At the very least, you should be able to get some sort of magnitude/fuzzy understanding of how frequently the new code is disagreeing, and you can figure out where and go check out those conditions.

jakecopp 8y ago
A/B testing self-driving car software to find bugs? Would probably work great, but that is also terrifying! But you're right, if you're really going to do a full roll out, may as well test it on a subsegment first - I'd hate for it to be used as a debugging tool though.
akira2501 8y ago
> but why can't you run both the new and old code side by side and if the actions they take are materially different investigate further? Because there are no closed facilities that you can use to actually perform any meaningful test. You could test "in-situ", but you would need an absolutely _huge_ testing area in order to accurately test and check all the different roadway configurations the vehicle is likely to encounter. You'll probably want more than one pass, some with pedestrians, some without, some in high light and some in low, etc.. It's worth noting that American's drive more than 260 billion miles each _month_. It's just an enormous problem.
- dchichkov 8y ago
  This particular case might have been testable "in-situ": [ABS News Reporter] Dan Noyes also spoke and texted with Walter Huang's brother, Will, today. He confirmed Walter was on the way to work at Apple when he died. He also makes a startling claim — that before the crash, Walter complained "seven-to-10 times the car would swivel toward that same exact barrier during auto-pilot. Walter took it into dealership addressing the issue, but they couldn't duplicate it there." It is very believable that the car would swivel toward the same exact barrier on auto-pilot. BTW - I'm running a nonprofit/public dataset project aimed at increasing safety of autonomous vehicles. If anyone here wants to contribute (with suggestions / pull requests / following it on twitter / etc) - you'd be most welcome. Its: https://www.safe-av.org/ https://www.safe-av.org/

erikpukinskis 8y ago

More accurately: move fast and save some lives and kill people.

InternetOfStuff 8y ago

> And this is exactly why all of these articles recently about how "great" it is that Tesla sends out frequent OTA updates are ridiculous. Frequent, unpredictable updates with changelogs that just read "Improvements and bug fixes" is fine when we're talking about a social media app, but is entirely unacceptable when we're talking about the software that controls a 2 ton hunk of metal I recently got downvoted for that exact line of reasoning. Looks like some people don't like to hear that :-)

bertil 8y ago
I think you got down-voted because you misrepresented what Telsa is actually doing, which is a difficult arbitrage between: - known preventable deaths from, say, not staying in the lane aggressively enough; - possible surprises and subsequent deaths. There is an ethical conundrum (that is quite different from the trolley problem) between a known fix and an unknown possibility. If both are clearly establish and you dismiss the former to have a simplistic take, yes, you would be down-voted because you are making the debate less informed. Without falling to solutionism, in that case, the remaining issue seems rather isolated to a handful of areas that look like lanes and could be either painted properly, or that Telsa cars could train to avoid. The later fix would have to be sent rapidly and could have surprising consequences -- although it seems decreasingly likely. That learning pattern (resolving unintended surprises as they happen decreasingly often) is common in software. This explains why this community prefers (to an extend) Telsa to other manufacturers. Others have preferred the surprise-free and PR-friendly option of not saving the dozens of thousand of lives dying on the road at the moment. There are ethical backgrounds to non-interventionism. As the victim of car violence, I happen to think that their position is unethical. I’m actually at what is considered an extreme position, of being in favour of Telsa (and Waymo) taking more risk than necessary and temporarily increasing the number of accidents on the road because they have a far better track record of learning from those accidents (and the subsequent press coverage) and that would lower the overall number of deaths faster. As it happens, they don’t need to: even with less than half of accidents from their counterparts, they still get spectacular learning rate.
- InternetOfStuff 8y ago