8 ms·

A Case Study of Toyota Unintended Acceleration and Software Safety

95 points by mzehrer 12y ago

silverpikezero 12y ago
This is an absolutely fascinating slide set. Thanks submitter very much for the link. I have written embedded C before, and the following facts just blow my mind: 1. The Throttle Angle function in the Toyota code had a McCabe Cyclomatic Complexity of 146 (over 50 is considered untestable according to slides) [slide 38] 2. The main throttle function was 1300 lines long, and had no directed tests. [slide 38] 3. I find the static analysis results quite alarming. [slide 37] 4. 80+% of variables were declared as global. [slide 40] I find this to be a stunning lapse of quality, especially for a safety-critical system.
- eropple 12y ago
  No ECC in RAM, either. Yiiiikes. Edit: or peer review, or even a bug tracker. My coworkers working on web apps have a better SDLC than these jokers.
  - threeseed 12y ago
    They did have peer review just not on every module. This is very common even amongst companies who say they do peer review. And curious about the lack of a bug tracking software. Many companies including the one I work for "technically" don't use them. I wouldn't call Rally or Trello a bug tracker but that do work just fine for that purpose. I also did find it amusing that they called out lack of formal specifications as an issue given that Agile encourages you not to create them.
    - sillysaurus3 12y ago
      It's not acceptable for a system that can kill you to be programmed without at least two programmers looking over every line of code. Common practice or not, given what we know about programming, it's immoral.
    - eropple 12y ago
      > They did have peer review just not on every module. This is very common even amongst companies who say they do peer review. This is not common at all in safety-critical applications. > I also did find it amusing that they called out lack of formal specifications as an issue given that Agile encourages you not to create them. This shouldn't be amusing if you understand Agile. Agile is for systems where you don't yet fully know the scope or the desired end behavior. The guts of a production car are as waterfall as it possibly can get.
- PhantomGremlin 12y ago
  Maybe they had a "hardware guy" with little software training or experience write the code? I've done a lot of both hardware and software, and I've seen a lot more bad software done by hardware engineers than I've seen bad hardware done by software engineers. The software guys usually know that they're out of their bailiwick when it comes to hardware design. E.g. one of the worst in my experience was a 30,000 line shell script, few if any functions, used as part of our production flow. A simple refactoring could have cut it down at least 90%. Even worse, it was totally unsupported because the guy who wrote it was reassigned.
  - ScottBurson 12y ago
    I think this is probably the essence of it. Hardware people see software as just hardware that can be easily reconfigured. That was a reasonable model back in the 8048 days, when one only had 1kB of ROM and 256B of RAM to work in. But these programs are obviously far larger, with orders of magnitude more data paths and astronomically more possible states. Different techniques are required. Someone from a hardware background isn't going to understand that.
    - aceperry 12y ago
      "Hardware people see software as just hardware that can be easily reconfigured" Not really, not if you have a realistic view of software. When I was an EE student, about half of my classmates loved software, the other half didn't like it and weren't good at it. And in silicon valley, a lot of EEs ended up doing software at all levels. I've always been jealous of how creative software could be, and didn't have the same limitations as hardware. Lots of hardware is usually programmed at low levels by EEs, which essentially ends up being drivers, and other board support software. They also realize that their training and education is not at the same level as software engineers in software. BTW, it used to be that if you couldn't cut it in engineering, EE/ME/ChemE, and you wanted to pursue a technical degree, most people would go into computer science. I realize it's not like that today, but that's how it was back in the day.
      - 12y ago
- magila 12y ago
  I'm not going to defend the other points, but most variables being global is not at all unusual for code like this. These sorts of realtime embedded applications typically have no heap allocation, so the only way to define persistent storage is as statically allocated globals. Modularization in these systems is done by convention, e.g. by limiting which include files are visible to a compilation unit.
  - DangerousPie 12y ago
    It's strange that the slides don't seem to mention this point at all, even though the author is presumably an expert in this field. Why would he make such a big point of this if it's commonly done in the field?
    - threeseed 12y ago
      There is a difference between being in an expert in the theory of software development and being an expert in the reality. The theory says that bad software is written when you don't have 100% test coverage, static analysis on checkins, peer review, pair programming, the usual guidelines around encapsulation, inheritance etc. The reality is that bad software mainly comes about through poorly thought out requirements, bad initial architecture decisions and ridiculous time constraints. It's always a shame when you see stories like this that those aspects never really get investigated or analysed.
      - nsnick 12y ago
        Koopman was one of the main witnesses for the prosecution. He has also worked in the nuclear navy and industry specifically designing safety critical systems.
  - jacquesgt 12y ago
    Non-stack variables can be static instead of global. Either file-scoped or function-scoped.
    - 12y ago
- wyager 12y ago
  >80+% of variables were declared as global This is actually important for safety-critical programming. It lessens the likelihood of running out of stack space, and it's useful for eliminating dynamic allocation. Some safety-critical software has 100% global variables, without even using a stack.
  - jacquesm 12y ago
    I think this is possibly one area where functional programming could shine if the garbage collection issue could be dealt with once and for all (and that's a hard one). It's for practical reasons impossible to test something with that much state exhaustively, but once broken up into functional units you just might get there. Something along the lines of Erlang for embedded systems.
    - mattmcknight 12y ago
      No. Functional programming does not magically free you from "state". The world has state, the unlimited number of real world inputs are the state of the program, merely expressed as function parameters. The CPU itself has registers, and stores a call stack. The abstraction of functional programming does not alleviate all of these issues, and can introduce other ones.
      - jacquesm 12y ago
        Of course it doesn't free you from state. But it can help to put the state into a more manageable context. Just like side-effect free functions can help you with that. I've done enough embedded programming (and repairs on embedded programming projects) to know just how bad the spaghetti can get and it really wouldn't hurt to borrow a few leaves from the functional world in those cases.
  - ufo 12y ago
    But even then, you want to declare those variables as static or otherwise find some way to reduce their scope. Most global variables in the Toyota code were visible and writable from the whole program. This is not even counting all the variables declared with the wrong type or more than once, uninitialized variables, etc.
- threeseed 12y ago
  Static analysis does not show you the complete picture. I have worked on countless horrifically complex, buggy and unmanageable J2EE applications that had perfect cyclomatic complexity et al scores. I have also written perfectly reviewed, manageable and well tested piece of code that didn't. I very much question the experience of anyone who can refer to a codebase as "spaghetti code" without seeing it and relying solely on static analysis. Software is not that simple or transparent.
  - userbinator 12y ago
    I'm going to guess that the applications you refer to are "simple" at the level of individual functions, but that's only because the complexity has been spread out so much that it becomes difficult to understand the whole. That's exactly why I think cyclomatic complexity and related metrics are of little benefit, and may even be harmful - "refactoring" to reduce "point complexity" can result in increasing the complexity of the whole. I've always wondered what the cyclomatic complexity of TeX is. While Knuth is a bit of an "edge case", it would be fun to see what static analysers think of his code... complete with copious use of global variables, goto, and very, very long functions. Give someone who has never had any experience with TeX the results and ask them what they think the defect rate would be, then show them the fact that it's one of the most bug-free pieces of software ever written.
    - sillysaurus3 12y ago
      it's one of the most bug-free pieces of software ever written. Not doubting you, but do you have a source to demonstrate that claim? If I make the claim to someone else, it'd be easier to provide evidence than for me to handwave. TeX version is 3.14159265, so some of those are probably bugfixes. EDIT: Um. Look, I rarely complain about downvotes, but what's up with the downvoting on HN lately? Is it me, or what? This is a simple request for more information about something I don't know about. It's not an easy thing to Google. It's up to the parent to provide evidence. https://www.google.com/search?q=tex+bug+free https://www.google.com/search?q=tex+bug+free shows a lot of evidence that TeX is absent of bugs, but that's not the question. The question is the total bugs that have been fixed since it was first written relative to every other major software project. That's not so easy to answer. https://www.google.com/search?q=low+total+bug+count https://www.google.com/search?q=low+total+bug+count brings up nothing relevant. In fact, it could turn out to be entirely false that TeX had a low total bugcount over its history relative to its size, especially during its very days. We don't know, because no one has provided evidence one way or another. All of this is exceedingly obvious, and it's getting tedious to type out huge edits like this whenever something straightforward is downvoted. I'm seriously tempted to create my own community at this point out of desperation, one that focuses on technical merit and being nice rather than posturing. I wonder if one already exists? I've heard some pretty good things about newsgroups, but haven't really looked into any.
cbd1984 12y ago
I thought the "unintended acceleration" was proven to have been due to older drivers who got confused?
- matt4077 12y ago
  Well, Toyota probably has the better PR team (and more of an interest in getting it's message out). It appears as if neither theory has ever been proven. The slides argue that the standard of evidence in civil proceedings is simply "more likely than not" and not "beyond reasonable doubt". Reading between the lines, I think this expert witness is arguing that anybody who writes code this terrible for a safety-critical application deserves to pay through the nose, irregardless of the actual chain of events.
- ArkyBeagle 12y ago
  The Micheal Barr Group was selected to analyze the Toyota controller code. They've filed online reports you can read.
- deleted 12y ago
  [deleted]
- bigtones 12y ago
  The example he gave was a young police officer who was a very experienced driver, and had a passenger call 911 in a panic while he was unable to stop the car with his foot firmly planted on the brake - and it ended up killing all four occupants. That negates the 'old people can't drive properly' theory.
  - lnanek2 12y ago
    Although in Toyota's defense the officer was too clueless to know he just had to hold the start button down for three seconds to turn off the ignition.
    - jacquesm 12y ago
      That's got to be the worst ever user interface. Press the START button to turn OFF the ignition? That's not accidental that's malicious, especially in an emergency situation. Have you ever seen a chunk of industrial machinery? Those big red mushroom buttons are the emergency shutdowns and they don't take 'three seconds' are not labeled 'start' for extra confusion in times of panic.
codys 12y ago
Anyone know what the things he "couldn't talk about" were? I was hoping to hear the open mic bit at the end for a hint but the video is cut off.
- MBCook 12y ago
  He mentioned he's still involved in legal proceedings and expects to be deposed in further trials. I imagine specific implementation details, things under NDA, etc. are what he's trying to avoid. In work like this, you certainly wouldn't want to accidentally cause a negligent company to get off the hook because of an unimportant public statement.
veidr 12y ago
I think this is one of the most interesting software stories in recent years, up there with Stuxnet. The software development process seems so staggeringly, jaw-droppingly incompetent and negligent, and it now seems clear that software flaws really did kill people despite the heavy layer of spin that it was driver error, floor mats, etc. I almost certainly couldn't buy a Toyota ever again knowing this. But it also really makes me wonder: how bad is the QA and testing for the software components of other carmakers' vehicles?
- tobinfricke 12y ago
  > I almost certainly couldn't buy a Toyota ever again knowing this. What makes you think any other car company is different/better?
  - veidr 12y ago
    That's what I mean; I don't necessarily think it is (which is scary). Maybe we only got to know how horribly flawed the software situation is at Toyota because they had finally had enough people killed that they just couldn't keep it hidden any longer, and Audi and Honda are just as bad and just haven't yet had this kind of exposure event. I would prefer to doubt that, but it does all kind of make me want to buy a restored 1978 Datsun with carburetors and mechanical everything.
    - mjevans 12y ago
      This makes me want a Tesla. That company actually seems to hire qualified engineers given the failure modes it's had in the exceedingly rare cases of accidents that have thus far made the news.
    - UrMomReadsHN 12y ago
      Idk... there was an article on here a while ago about how after the Ford/Firestone tires = death incidents a decade and a half ago actually resulted in Ford making much better quality cars.
jacquesm 12y ago
I can see an excellent case here for open sourcing such code.
- nitrogen 12y ago
  I've always wanted to build an open source ECU, but haven't had the time.
erdle 12y ago
Not sure if it's widely known how they were caught... but it was when they outsourced their translation of documents and a translator read documents and blew the whistle. Which is why the top translation firms now offer... security!
- veidr 12y ago
  Wow, do you have a link for that? I think this is a fascinating story in and of itself, but one of the tangential things that also interests me about it is that nobody I know in Japan (where I live) has ever even heard of this story.
  - erdle 12y ago
    A friend is a director at an interesting company that provides secure translations amongst other services to companies and has made bank since this happened. I guess it's a bit of a secret outside the industry.
  - bb_wordsmith 12y ago
    Here's a link to one key article: http://www.asbpe.org/wp-content/uploads/2014/07/Toyota-Cover-Story.pdf http://www.asbpe.org/wp-content/uploads/2014/07/Toyota-Cover... In Japan, some journalists know, but media organizations won't publish the story. I am the translator.
- zaroth 12y ago
  'How they were caught' you mean how we discovered Toyotas suffered from "UA"? The story claims it was the incident of the off-duty highway patrol officer on a call with 911 for 4 minutes trying to stop the car before it crashed and killed all 4 occupants which blew the lid on the story.
  - stretchwithme 12y ago
    Which is why its good to know how to shift into neutral.
upofadown 12y ago
If someone ever comes up with a practical method of creating reliable software then everyone would use it. You can always find a programmer who will claim that they would of done it better... For all we know whatever Toyota is doing is way better than what everyone else is doing and this is their only safety critical bug (assuming that the bug actually exists)... In the end the software is either correct or it is not.
- tokenrove 12y ago
  The information that has been released so far, especially Michael Barr's comments on it, suggest that this is far from the only bug. For example, http://www.edn.com/design/automotive/4423428/Toyota-s-killer-firmware--Bad-design-and-its-consequences http://www.edn.com/design/automotive/4423428/Toyota-s-killer... quotes Barr's claims: "Toyota’s electronic throttle control system (ETCS) source code is of unreasonable quality." "Toyota’s source code is defective and contains bugs, including bugs that can cause unintended acceleration (UA)." I am a little appalled at the number of apologist comments on this story. There is mounting evidence that this wasn't a "it could happen to anyone" bug, but rather a serious violation of software engineering ethics. Code must not kill.
  - upofadown 12y ago
    >...software engineering... Some of us do not believe that such a thing exists in any meaningful form. In the past every approach to creating reliable software has failed to deliver. That makes it hard to believe that any particular approach is finally the answer. I agree that the Toyota software is poorly written. That in no way means that conforming to a particular standard or method would of automatically produced software that was more reliable.
snowwrestler 12y ago
Also see this report from another experts involved in one of the suits: http://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code http://www.safetyresearch.net/blog/articles/toyota-unintende... And the slides (PDF): http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUB...
tek-cyb-org 12y ago
This happened to my parents in a 2002 civic.
- deleted 12y ago
  [deleted]
- serf 12y ago
  2002 civic used a mechanical throttle cable and mechanical transmission linkage. These Toyotas/Lexus(s) did not use any such mechanical links to the drive train, they were dependent on software wholly. The engine management on your parents' Honda cannot adjust the throttle percentage or gearing, only ignition and valve timing along with air/fuel ratios and gear hold-out timings. This could result in a surge of power, or a stalled car, but nothing like the experience of wide open throttle. The Toyota UA is thought to have been rooted mostly in the fact that they adopted fly-by-wire throttle schemes while failing to compensate for the much more major software/hardware/management risks that come with such systems.
phugoid 12y ago
My only criticism is that he's comparing Toyota's code and processes to a published but unenforced standard and clearly they are nowhere near that standard. It would be more relevant to compare Toyota to other manufacturers. It seems the legal argument against Toyota is that they were not following industry standards - but if no one else was, could you really call it an industry standard?
- rwmj 12y ago
  So if all car manufacturers are as bad as each other, that's OK?
  - phugoid 12y ago
    Not at all. But if it turned out that Toyota was already trying three times harder than everyone else to get it right, it would be harder to argue they didn't take "reasonable" precautions. In the talk, he did mention that the government agency that certifies vehicles does only basic checks and does not enforce any standards on software. One could easily argue that they are partially to blame, as they're leaving it up to the manufacturers.
aristidb 12y ago
I'm not really a fan of MISRA-C or V process and stuff like that, but the quality of the source code and process is truly appalling if these slides are right... tens of thousands of global variables, no bug tracking and so on and so on. I really hope they improved since then, for the sake of anybody driving a Toyota.
EliRivers 12y ago
I work with some people in the static analysis industry, and I'm told that Toyota did have static analysis tools that would have identified a number of these issues; as a particular example, recursive functions, which are verboten under the MISRA rules they should have been following. I heard to a less reliable degree that the tools had been used, and results ignored.

95 points by mzehrer 12y ago

This is an absolutely fascinating slide set. Thanks submitter very much for the link. I have written embedded C before, and the following facts just blow my mind: 1. The Throttle Angle function in the Toyota code had a McCabe Cyclomatic Complexity of 146 (over 50 is considered untestable according to slides) [slide 38] 2. The main throttle function was 1300 lines long, and had no directed tests. [slide 38] 3. I find the static analysis results quite alarming. [slide 37] 4. 80+% of variables were declared as global. [slide 40] I find this to be a stunning lapse of quality, especially for a safety-critical system.

eropple 12y ago
No ECC in RAM, either. Yiiiikes. Edit: or peer review, or even a bug tracker. My coworkers working on web apps have a better SDLC than these jokers.
- threeseed 12y ago
  They did have peer review just not on every module. This is very common even amongst companies who say they do peer review. And curious about the lack of a bug tracking software. Many companies including the one I work for "technically" don't use them. I wouldn't call Rally or Trello a bug tracker but that do work just fine for that purpose. I also did find it amusing that they called out lack of formal specifications as an issue given that Agile encourages you not to create them.
  - sillysaurus3 12y ago
    It's not acceptable for a system that can kill you to be programmed without at least two programmers looking over every line of code. Common practice or not, given what we know about programming, it's immoral.
  - eropple 12y ago
    > They did have peer review just not on every module. This is very common even amongst companies who say they do peer review. This is not common at all in safety-critical applications. > I also did find it amusing that they called out lack of formal specifications as an issue given that Agile encourages you not to create them. This shouldn't be amusing if you understand Agile. Agile is for systems where you don't yet fully know the scope or the desired end behavior. The guts of a production car are as waterfall as it possibly can get.
PhantomGremlin 12y ago
Maybe they had a "hardware guy" with little software training or experience write the code? I've done a lot of both hardware and software, and I've seen a lot more bad software done by hardware engineers than I've seen bad hardware done by software engineers. The software guys usually know that they're out of their bailiwick when it comes to hardware design. E.g. one of the worst in my experience was a 30,000 line shell script, few if any functions, used as part of our production flow. A simple refactoring could have cut it down at least 90%. Even worse, it was totally unsupported because the guy who wrote it was reassigned.
- ScottBurson 12y ago
  I think this is probably the essence of it. Hardware people see software as just hardware that can be easily reconfigured. That was a reasonable model back in the 8048 days, when one only had 1kB of ROM and 256B of RAM to work in. But these programs are obviously far larger, with orders of magnitude more data paths and astronomically more possible states. Different techniques are required. Someone from a hardware background isn't going to understand that.
  - aceperry 12y ago
    "Hardware people see software as just hardware that can be easily reconfigured" Not really, not if you have a realistic view of software. When I was an EE student, about half of my classmates loved software, the other half didn't like it and weren't good at it. And in silicon valley, a lot of EEs ended up doing software at all levels. I've always been jealous of how creative software could be, and didn't have the same limitations as hardware. Lots of hardware is usually programmed at low levels by EEs, which essentially ends up being drivers, and other board support software. They also realize that their training and education is not at the same level as software engineers in software. BTW, it used to be that if you couldn't cut it in engineering, EE/ME/ChemE, and you wanted to pursue a technical degree, most people would go into computer science. I realize it's not like that today, but that's how it was back in the day.
    - 12y ago
magila 12y ago
I'm not going to defend the other points, but most variables being global is not at all unusual for code like this. These sorts of realtime embedded applications typically have no heap allocation, so the only way to define persistent storage is as statically allocated globals. Modularization in these systems is done by convention, e.g. by limiting which include files are visible to a compilation unit.
- DangerousPie 12y ago
  It's strange that the slides don't seem to mention this point at all, even though the author is presumably an expert in this field. Why would he make such a big point of this if it's commonly done in the field?
  - threeseed 12y ago
    There is a difference between being in an expert in the theory of software development and being an expert in the reality. The theory says that bad software is written when you don't have 100% test coverage, static analysis on checkins, peer review, pair programming, the usual guidelines around encapsulation, inheritance etc. The reality is that bad software mainly comes about through poorly thought out requirements, bad initial architecture decisions and ridiculous time constraints. It's always a shame when you see stories like this that those aspects never really get investigated or analysed.
    - nsnick 12y ago
      Koopman was one of the main witnesses for the prosecution. He has also worked in the nuclear navy and industry specifically designing safety critical systems.
- jacquesgt 12y ago
  Non-stack variables can be static instead of global. Either file-scoped or function-scoped.
  - 12y ago
wyager 12y ago
>80+% of variables were declared as global This is actually important for safety-critical programming. It lessens the likelihood of running out of stack space, and it's useful for eliminating dynamic allocation. Some safety-critical software has 100% global variables, without even using a stack.
- jacquesm 12y ago
  I think this is possibly one area where functional programming could shine if the garbage collection issue could be dealt with once and for all (and that's a hard one). It's for practical reasons impossible to test something with that much state exhaustively, but once broken up into functional units you just might get there. Something along the lines of Erlang for embedded systems.
  - mattmcknight 12y ago
    No. Functional programming does not magically free you from "state". The world has state, the unlimited number of real world inputs are the state of the program, merely expressed as function parameters. The CPU itself has registers, and stores a call stack. The abstraction of functional programming does not alleviate all of these issues, and can introduce other ones.
    - jacquesm 12y ago
      Of course it doesn't free you from state. But it can help to put the state into a more manageable context. Just like side-effect free functions can help you with that. I've done enough embedded programming (and repairs on embedded programming projects) to know just how bad the spaghetti can get and it really wouldn't hurt to borrow a few leaves from the functional world in those cases.
- ufo 12y ago
  But even then, you want to declare those variables as static or otherwise find some way to reduce their scope. Most global variables in the Toyota code were visible and writable from the whole program. This is not even counting all the variables declared with the wrong type or more than once, uninitialized variables, etc.
threeseed 12y ago
Static analysis does not show you the complete picture. I have worked on countless horrifically complex, buggy and unmanageable J2EE applications that had perfect cyclomatic complexity et al scores. I have also written perfectly reviewed, manageable and well tested piece of code that didn't. I very much question the experience of anyone who can refer to a codebase as "spaghetti code" without seeing it and relying solely on static analysis. Software is not that simple or transparent.
- userbinator 12y ago
  I'm going to guess that the applications you refer to are "simple" at the level of individual functions, but that's only because the complexity has been spread out so much that it becomes difficult to understand the whole. That's exactly why I think cyclomatic complexity and related metrics are of little benefit, and may even be harmful - "refactoring" to reduce "point complexity" can result in increasing the complexity of the whole. I've always wondered what the cyclomatic complexity of TeX is. While Knuth is a bit of an "edge case", it would be fun to see what static analysers think of his code... complete with copious use of global variables, goto, and very, very long functions. Give someone who has never had any experience with TeX the results and ask them what they think the defect rate would be, then show them the fact that it's one of the most bug-free pieces of software ever written.
  - sillysaurus3 12y ago
    it's one of the most bug-free pieces of software ever written. Not doubting you, but do you have a source to demonstrate that claim? If I make the claim to someone else, it'd be easier to provide evidence than for me to handwave. TeX version is 3.14159265, so some of those are probably bugfixes. EDIT: Um. Look, I rarely complain about downvotes, but what's up with the downvoting on HN lately? Is it me, or what? This is a simple request for more information about something I don't know about. It's not an easy thing to Google. It's up to the parent to provide evidence. https://www.google.com/search?q=tex+bug+free https://www.google.com/search?q=tex+bug+free shows a lot of evidence that TeX is absent of bugs, but that's not the question. The question is the total bugs that have been fixed since it was first written relative to every other major software project. That's not so easy to answer. https://www.google.com/search?q=low+total+bug+count https://www.google.com/search?q=low+total+bug+count brings up nothing relevant. In fact, it could turn out to be entirely false that TeX had a low total bugcount over its history relative to its size, especially during its very days. We don't know, because no one has provided evidence one way or another. All of this is exceedingly obvious, and it's getting tedious to type out huge edits like this whenever something straightforward is downvoted. I'm seriously tempted to create my own community at this point out of desperation, one that focuses on technical merit and being nice rather than posturing. I wonder if one already exists? I've heard some pretty good things about newsgroups, but haven't really looked into any.

cbd1984 12y ago

I thought the "unintended acceleration" was proven to have been due to older drivers who got confused?

matt4077 12y ago
Well, Toyota probably has the better PR team (and more of an interest in getting it's message out). It appears as if neither theory has ever been proven. The slides argue that the standard of evidence in civil proceedings is simply "more likely than not" and not "beyond reasonable doubt". Reading between the lines, I think this expert witness is arguing that anybody who writes code this terrible for a safety-critical application deserves to pay through the nose, irregardless of the actual chain of events.
ArkyBeagle 12y ago
The Micheal Barr Group was selected to analyze the Toyota controller code. They've filed online reports you can read.
deleted 12y ago
[deleted]
bigtones 12y ago
The example he gave was a young police officer who was a very experienced driver, and had a passenger call 911 in a panic while he was unable to stop the car with his foot firmly planted on the brake - and it ended up killing all four occupants. That negates the 'old people can't drive properly' theory.
- lnanek2 12y ago
  Although in Toyota's defense the officer was too clueless to know he just had to hold the start button down for three seconds to turn off the ignition.
  - jacquesm 12y ago
    That's got to be the worst ever user interface. Press the START button to turn OFF the ignition? That's not accidental that's malicious, especially in an emergency situation. Have you ever seen a chunk of industrial machinery? Those big red mushroom buttons are the emergency shutdowns and they don't take 'three seconds' are not labeled 'start' for extra confusion in times of panic.

codys 12y ago

Anyone know what the things he "couldn't talk about" were? I was hoping to hear the open mic bit at the end for a hint but the video is cut off.

MBCook 12y ago
He mentioned he's still involved in legal proceedings and expects to be deposed in further trials. I imagine specific implementation details, things under NDA, etc. are what he's trying to avoid. In work like this, you certainly wouldn't want to accidentally cause a negligent company to get off the hook because of an unimportant public statement.

veidr 12y ago

I think this is one of the most interesting software stories in recent years, up there with Stuxnet. The software development process seems so staggeringly, jaw-droppingly incompetent and negligent, and it now seems clear that software flaws really did kill people despite the heavy layer of spin that it was driver error, floor mats, etc. I almost certainly couldn't buy a Toyota ever again knowing this. But it also really makes me wonder: how bad is the QA and testing for the software components of other carmakers' vehicles?

tobinfricke 12y ago
> I almost certainly couldn't buy a Toyota ever again knowing this. What makes you think any other car company is different/better?
- veidr 12y ago
  That's what I mean; I don't necessarily think it is (which is scary). Maybe we only got to know how horribly flawed the software situation is at Toyota because they had finally had enough people killed that they just couldn't keep it hidden any longer, and Audi and Honda are just as bad and just haven't yet had this kind of exposure event. I would prefer to doubt that, but it does all kind of make me want to buy a restored 1978 Datsun with carburetors and mechanical everything.
  - mjevans 12y ago
    This makes me want a Tesla. That company actually seems to hire qualified engineers given the failure modes it's had in the exceedingly rare cases of accidents that have thus far made the news.
  - UrMomReadsHN 12y ago
    Idk... there was an article on here a while ago about how after the Ford/Firestone tires = death incidents a decade and a half ago actually resulted in Ford making much better quality cars.

jacquesm 12y ago

I can see an excellent case here for open sourcing such code.

nitrogen 12y ago
I've always wanted to build an open source ECU, but haven't had the time.

erdle 12y ago

Not sure if it's widely known how they were caught... but it was when they outsourced their translation of documents and a translator read documents and blew the whistle. Which is why the top translation firms now offer... security!

veidr 12y ago
Wow, do you have a link for that? I think this is a fascinating story in and of itself, but one of the tangential things that also interests me about it is that nobody I know in Japan (where I live) has ever even heard of this story.
- erdle 12y ago
  A friend is a director at an interesting company that provides secure translations amongst other services to companies and has made bank since this happened. I guess it's a bit of a secret outside the industry.
- bb_wordsmith 12y ago
  Here's a link to one key article: http://www.asbpe.org/wp-content/uploads/2014/07/Toyota-Cover-Story.pdf http://www.asbpe.org/wp-content/uploads/2014/07/Toyota-Cover... In Japan, some journalists know, but media organizations won't publish the story. I am the translator.
zaroth 12y ago
'How they were caught' you mean how we discovered Toyotas suffered from "UA"? The story claims it was the incident of the off-duty highway patrol officer on a call with 911 for 4 minutes trying to stop the car before it crashed and killed all 4 occupants which blew the lid on the story.
- stretchwithme 12y ago
  Which is why its good to know how to shift into neutral.

upofadown 12y ago

If someone ever comes up with a practical method of creating reliable software then everyone would use it. You can always find a programmer who will claim that they would of done it better... For all we know whatever Toyota is doing is way better than what everyone else is doing and this is their only safety critical bug (assuming that the bug actually exists)... In the end the software is either correct or it is not.

tokenrove 12y ago
The information that has been released so far, especially Michael Barr's comments on it, suggest that this is far from the only bug. For example, http://www.edn.com/design/automotive/4423428/Toyota-s-killer-firmware--Bad-design-and-its-consequences http://www.edn.com/design/automotive/4423428/Toyota-s-killer... quotes Barr's claims: "Toyota’s electronic throttle control system (ETCS) source code is of unreasonable quality." "Toyota’s source code is defective and contains bugs, including bugs that can cause unintended acceleration (UA)." I am a little appalled at the number of apologist comments on this story. There is mounting evidence that this wasn't a "it could happen to anyone" bug, but rather a serious violation of software engineering ethics. Code must not kill.
- upofadown 12y ago
  >...software engineering... Some of us do not believe that such a thing exists in any meaningful form. In the past every approach to creating reliable software has failed to deliver. That makes it hard to believe that any particular approach is finally the answer. I agree that the Toyota software is poorly written. That in no way means that conforming to a particular standard or method would of automatically produced software that was more reliable.

snowwrestler 12y ago

Also see this report from another experts involved in one of the suits: http://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code http://www.safetyresearch.net/blog/articles/toyota-unintende... And the slides (PDF): http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUBBED.pdf http://www.safetyresearch.net/Library/BarrSlides_FINAL_SCRUB...

tek-cyb-org 12y ago

This happened to my parents in a 2002 civic.

deleted 12y ago
[deleted]
serf 12y ago
2002 civic used a mechanical throttle cable and mechanical transmission linkage. These Toyotas/Lexus(s) did not use any such mechanical links to the drive train, they were dependent on software wholly. The engine management on your parents' Honda cannot adjust the throttle percentage or gearing, only ignition and valve timing along with air/fuel ratios and gear hold-out timings. This could result in a surge of power, or a stalled car, but nothing like the experience of wide open throttle. The Toyota UA is thought to have been rooted mostly in the fact that they adopted fly-by-wire throttle schemes while failing to compensate for the much more major software/hardware/management risks that come with such systems.

phugoid 12y ago

My only criticism is that he's comparing Toyota's code and processes to a published but unenforced standard and clearly they are nowhere near that standard. It would be more relevant to compare Toyota to other manufacturers. It seems the legal argument against Toyota is that they were not following industry standards - but if no one else was, could you really call it an industry standard?

rwmj 12y ago
So if all car manufacturers are as bad as each other, that's OK?
- phugoid 12y ago
  Not at all. But if it turned out that Toyota was already trying three times harder than everyone else to get it right, it would be harder to argue they didn't take "reasonable" precautions. In the talk, he did mention that the government agency that certifies vehicles does only basic checks and does not enforce any standards on software. One could easily argue that they are partially to blame, as they're leaving it up to the manufacturers.

aristidb 12y ago

I'm not really a fan of MISRA-C or V process and stuff like that, but the quality of the source code and process is truly appalling if these slides are right... tens of thousands of global variables, no bug tracking and so on and so on. I really hope they improved since then, for the sake of anybody driving a Toyota.

EliRivers 12y ago

I work with some people in the static analysis industry, and I'm told that Toyota did have static analysis tools that would have identified a number of these issues; as a particular example, recursive functions, which are verboten under the MISRA rules they should have been following. I heard to a less reliable degree that the tools had been used, and results ignored.