4 ms·

Every time this article comes up on hacker news there are a few comments that show up talking about how no one was able to prove that the bugs definitely caused

Every time this article comes up on hacker news there are a few comments that show up talking about how no one was able to prove that the bugs definitely caused the crashes, and thus the criticisms of Toyota are overblown. I'm left wondering how on earth to reconcile those viewpoints with general software engineering (which is a huge part of what this community is about), where we regularly see articles about new concurrency frameworks, fault-tolerant programming, formal methods to prove correctness of code, etc. If it's important enough for Amazon to use TLA+ to prove correctness of DyanmoDB then shouldn't it be considered outrageous that Toyota doesn't do the same thing for something that is potentially an accelerating death machine?

Gibbon1 11y ago
The reason generally given is that in at least standard cars the brakes can apply far more torque than the engine can. It's telling how naive people are, they think the problem is 'unintended acceleration' A mechanical engineer thinks the problem describe is 'unintended brake failure'
- jessaustin 11y ago
  And if the brakes won't stop the car, try the key switch! An engine that is not running will not cause acceleration. (Sure there are probably some oddball push-button ignitions that can't be turned off while in gear, but most vehicles can be.)
  - fr0sty 11y ago
    Turning off the ignition may deactivate your airbags* which could make a bad situation worse.You would be much better off attempting to shift your car into Neutral first. * This was what made the GM Ingnition switch problem so deadly. The engine shut off (taking power steering and vaccuum-assist braking with it) and the airbags deactivated making any subsequent crash less survivable.
    - jessaustin 11y ago
      You're right that shifting is better. That is a pretty terrible failure mode for airbags, however. How much would it cost to put a big reliable capacitor in the airbag circuit, to keep the power on for 30 seconds? A dollar?
- yaakov34 11y ago
  Absolutely right. After reading any number of stories about these "runaway death machines", what becomes clear is that the only realistic way for a car to accelerate at full power against the wishes of the driver is for the driver to be flooring the gas while thinking he is on the brakes. We should be adding smarter automatic systems to the cars to assist drivers, rather than e.g. throwing out drive-by-wire linkages and replacing them with mechanics, which are generally less reliable. There are already cars which will slam on the brakes in any collision which is strong enough to deploy airbags - it stops the car from careening around too much. It should be possible for cars to detect an impending collision and slam the brakes 1/2 second early to prevent or reduce it.
yaakov34 11y ago
Sure, they should improve or clean up the code on general principles. However, there are 2 issues here: 1) A lot of these articles imply that some expert has demonstrated how a coding error will cause unintended acceleration. Then when you look at the actual source, it becomes clear that the "demonstration" involves changing the internal state of the controller in all sorts of arbitrary ways, and sometimes rewiring its sensors in an invalid way as well. In other words, this is not a recipe along the lines of "blip the throttle while changing from N to D, press the brake within 0.2 seconds, the throttle will now be wide open". The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild. Implying that this code is killing people somewhere out there is misleading. 2) Yes, it has been overblown. We know that throttles can stick open, usually due to jammed or sticking linkages and pedals. That's not a huge problem, since brakes are powerful enough to stop cars in this state. I don't think it makes a huge amount of sense to rewrite software to stop software-induced unintended acceleration, which probably doesn't even happen, instead of writing software for lane departure/collision warning/assisted emergency braking and other safety systems, which we know can help drivers avoid accidents. I guess a third issue here is the American litigation system, which can turn companies into villains without the slightest indication that their products cause any more issues than anybody else's - all it takes is a non-zero probability of failure (true for most products) and a media + legal frenzy.
- duaneb 11y ago
  > The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild. I don't agree with this. Analysis of the source code, especially with global variables reducing the value of analyzing a unit, is not going to imply that it will discover anything about the (presumably millions of) users who exercise the code in live situations daily. Just the inability to CLEAR the source code from blame is a failing in responsibility on behalf of the brand. The american litigation system has its issues, but I would argue that class action lawsuits are not among them. It allows social change through clear legal decisions when people are otherwise disagreeing on matters much like this.
lambdaelite 11y ago
The HN community at large isn't experienced with software engineering. It seems like there are a handful of commenters who are, but they are a very small minority. Personally, I'm surprised that the majority of the HN community overlooks or ignores the reports from NASA and Exponent (Toyota's outside expert witness) in favor of the testimony from Barr, the witness for the plaintiffs in the Oklahoma case. (I'm honestly amazed that Barr was allowed as an expert witness) Barr did show that the code "smelled" and that it was possible to inject specific faults to induce an uncommanded acceleration, but he did not show that his proposed failure mode occurred nor that it was probable to occur. In fact, part of his failure mode was that it left no record of a DTC. It's the perfect kind of failure mode for a sympathetic plaintiff to wield against an arrogant defendant with deep pockets: possible that it occurred, possible to demonstrate, understandable for a jury (who hasn't had a BSOD?), and impossible to disprove. You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist. Proof assistants, which today remain challenging to use and integrate into an SLDC, were even more arcane. In fact, your example of TLA+ wouldn't fly even today since TLA+ doesn't do code generation, and that would be a necessary component of the verification process of a regulated SLDC. I don't think TLA+ existed at the time either. Things certainly have changed!
- noir_lord 11y ago
  > You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist. DO-178B was created in 1989, Ada was created in the 70's, standards for mission and safety critical systems have been around since the 70's, I was reading about Ada and safety critical programming as a kid in the 80's (I was a weird kid). http://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf http://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_s... has a very good overview. Whether you accept that multiple independent experts in safety critical programming are right or not is up to you however the stuff that came out of the trial painted a very clear picture to me of just how bad this software is. A cynical person might consider that if Toyota is this poor how many other manufacturers are also as bad. EDIT: I'm not saying that the UA was caused by this, without the source code and some proof I can't say that however the general setting is pretty terrifying.

Gibbon1 11y ago

The reason generally given is that in at least standard cars the brakes can apply far more torque than the engine can. It's telling how naive people are, they think the problem is 'unintended acceleration' A mechanical engineer thinks the problem describe is 'unintended brake failure'

jessaustin 11y ago
And if the brakes won't stop the car, try the key switch! An engine that is not running will not cause acceleration. (Sure there are probably some oddball push-button ignitions that can't be turned off while in gear, but most vehicles can be.)
- fr0sty 11y ago
  Turning off the ignition may deactivate your airbags* which could make a bad situation worse.You would be much better off attempting to shift your car into Neutral first. * This was what made the GM Ingnition switch problem so deadly. The engine shut off (taking power steering and vaccuum-assist braking with it) and the airbags deactivated making any subsequent crash less survivable.
  - jessaustin 11y ago
    You're right that shifting is better. That is a pretty terrible failure mode for airbags, however. How much would it cost to put a big reliable capacitor in the airbag circuit, to keep the power on for 30 seconds? A dollar?
yaakov34 11y ago
Absolutely right. After reading any number of stories about these "runaway death machines", what becomes clear is that the only realistic way for a car to accelerate at full power against the wishes of the driver is for the driver to be flooring the gas while thinking he is on the brakes. We should be adding smarter automatic systems to the cars to assist drivers, rather than e.g. throwing out drive-by-wire linkages and replacing them with mechanics, which are generally less reliable. There are already cars which will slam on the brakes in any collision which is strong enough to deploy airbags - it stops the car from careening around too much. It should be possible for cars to detect an impending collision and slam the brakes 1/2 second early to prevent or reduce it.

yaakov34 11y ago

Sure, they should improve or clean up the code on general principles. However, there are 2 issues here: 1) A lot of these articles imply that some expert has demonstrated how a coding error will cause unintended acceleration. Then when you look at the actual source, it becomes clear that the "demonstration" involves changing the internal state of the controller in all sorts of arbitrary ways, and sometimes rewiring its sensors in an invalid way as well. In other words, this is not a recipe along the lines of "blip the throttle while changing from N to D, press the brake within 0.2 seconds, the throttle will now be wide open". The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild. Implying that this code is killing people somewhere out there is misleading. 2) Yes, it has been overblown. We know that throttles can stick open, usually due to jammed or sticking linkages and pedals. That's not a huge problem, since brakes are powerful enough to stop cars in this state. I don't think it makes a huge amount of sense to rewrite software to stop software-induced unintended acceleration, which probably doesn't even happen, instead of writing software for lane departure/collision warning/assisted emergency braking and other safety systems, which we know can help drivers avoid accidents. I guess a third issue here is the American litigation system, which can turn companies into villains without the slightest indication that their products cause any more issues than anybody else's - all it takes is a non-zero probability of failure (true for most products) and a media + legal frenzy.

duaneb 11y ago
> The fact that no such recipe has been found, despite many millions of dollars spent on expert analysis, suggests that it doesn't exist in the wild. I don't agree with this. Analysis of the source code, especially with global variables reducing the value of analyzing a unit, is not going to imply that it will discover anything about the (presumably millions of) users who exercise the code in live situations daily. Just the inability to CLEAR the source code from blame is a failing in responsibility on behalf of the brand. The american litigation system has its issues, but I would argue that class action lawsuits are not among them. It allows social change through clear legal decisions when people are otherwise disagreeing on matters much like this.

lambdaelite 11y ago

The HN community at large isn't experienced with software engineering. It seems like there are a handful of commenters who are, but they are a very small minority. Personally, I'm surprised that the majority of the HN community overlooks or ignores the reports from NASA and Exponent (Toyota's outside expert witness) in favor of the testimony from Barr, the witness for the plaintiffs in the Oklahoma case. (I'm honestly amazed that Barr was allowed as an expert witness) Barr did show that the code "smelled" and that it was possible to inject specific faults to induce an uncommanded acceleration, but he did not show that his proposed failure mode occurred nor that it was probable to occur. In fact, part of his failure mode was that it left no record of a DTC. It's the perfect kind of failure mode for a sympathetic plaintiff to wield against an arrogant defendant with deep pockets: possible that it occurred, possible to demonstrate, understandable for a jury (who hasn't had a BSOD?), and impossible to disprove. You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist. Proof assistants, which today remain challenging to use and integrate into an SLDC, were even more arcane. In fact, your example of TLA+ wouldn't fly even today since TLA+ doesn't do code generation, and that would be a necessary component of the verification process of a regulated SLDC. I don't think TLA+ existed at the time either. Things certainly have changed!

noir_lord 11y ago
> You bring up the practice of software engineering, but I think you need to place it in context of how software engineering was practiced around the time that the electronic throttle system was developed. MISRA, which HN readers are becoming aware of, didn't exist. DO-178B was created in 1989, Ada was created in the 70's, standards for mission and safety critical systems have been around since the 70's, I was reading about Ada and safety critical programming as a kid in the 80's (I was a weird kid). http://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf http://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_s... has a very good overview. Whether you accept that multiple independent experts in safety critical programming are right or not is up to you however the stuff that came out of the trial painted a very clear picture to me of just how bad this software is. A cynical person might consider that if Toyota is this poor how many other manufacturers are also as bad. EDIT: I'm not saying that the UA was caused by this, without the source code and some proof I can't say that however the general setting is pretty terrifying.