18 ms·

A case study of Toyota unintended acceleration and software safety (2014) [pdf]

89 points by georgecmu 4y ago

29athrowaway 4y ago
If you write dirty code knowing it is going to be used in a car, you should be charged with manslaughter, children endangerment and any form of legal punishment arising from creating long functions with high cyclomatic complexity being called in spaghetti in the context of controlling a vehicle that can cause people to die. Seriously, when will the excuses for dirty code end? Whatever you think the business is, clearly killing 89 people is not it. Use a linter and remember your contribution to the business as a programmer is programming properly. If programming is below you go do something else. If you have management aspirations go interview for a management job. If you don't like coding properly go code something else or consider another career. Write proper code and stop talking about how dirty code is good for the business. Not only it is not true, nobody wants to hear about it, and nobody should have to deal with your 4d jenga tower in a professional setting, goddamn it.
shadowgovt 4y ago
This actually marks the second incident I had tucked away in my brain where the initial story was "Sometimes the world is just weird" and the actual issue was computer systems. The reason we all keep seatbelts bucked in flight even when the sign is off these days is Qantas Flight 72, which took two sharp nose-dives in flight, rebounding passengers and crew off the ceiling and causing severe injuries. At the time, initial reporting was that they'd caught two freak mid-Pacific downdrafts, which had me nervous about flying for awhile. Turns out, the root cause was bad angle-of-attack data coming in from a faulty inertial measurement unit, which should have been caught by a safety system but proved so aggressively faulty that the safety system was overruled. It resulted in the plane's avionics becoming convinced it was flying belly-on into the wind at cruising speed and about to stall, so it did the thing it's supposed to do: pitched down hard to stall-recover. Don't ask me why I'm more comfortable with the story being "Sometimes computer engineers screw up" than "Sometimes our understanding of the physics of aviation just breaks down," but I am. ;)
shadowgovt 4y ago
From a software design and engineering perspective, I agree with the NASA report's take on global variable use. ... but I've also done game development, where maximum performance in minimum time is absolutely essential, and I wonder if it was actually feasible to maintain the performance the control system needed without global state. There's also the possibility of a culture clash. I've noticed that in embedded systems architecture development, global variables aren't frowned upon nearly as hard as they are in enterprise software, mostly because of the industry history of doing more with less. So I imagine a software culture risk developing from getting a bunch of hardware guys on a project where they now have access to microcontrollers with more memory than ever, but their risk tradeoff mindset is still "take the risk to squeeze every byte because we might run out."
XnoiVeX 4y ago
All motorcycles come with a standard electronic kill switch. Should cars be mandated to have them too?
- throwaway-jim 4y ago
  I don't understand where there is a kill switch when you can just turn off your vehicle using the key? Besides killing the engine will mean no power to the brakes and steering as well.
  - michaelt 4y ago
    Motorcycle kill switches are always on the handlebars, so you can reach it without letting go of the handlebars. Also makes it easier to find - some bikes have the keyswitch on the side, but the kill switch is always on the handlebars.
  - sokoloff 4y ago
    The brake booster will hold some vacuum for a few brake applications and, if the car remains in gear, the rotation will continue to create vacuum to recharge the brake booster (and turn the power steering pump, but power steering is not needed in practice at anything approaching “runaway car” speed).
jonsen 4y ago
I haven't heard of any incidents since that time. Did they fix all the cars, or what?
- MBCook 4y ago
  My understanding is it was never clear if it actually happened in the first place or was just operator error. I seem to remember there was also a recall on floor mats that could have caused issues. All the software stuff was discovered as a result of lawsuits. The software was atrocious and may have been capable of causing issues. But I don’t think anyone ever proved it did to any reasonable degree.
deleted 4y ago
[deleted]
tempnow987 4y ago
I'm always curious in these cases. Was the key not usable? Ie, turn off vehicle? Was neutral not available? To get my car to go I need key in the on position AND the gear in drive. I can go to neutral and turn off key anytime. These stories of barreling down the highway with key locked, throttle locked, gear selector locked seem bogus.
- weaksauce 4y ago
  you going 100mph and turning off your car is not a great idea since that locks up the steering wheel all but guaranteeing a crash. I am pretty sure most cars should be able to switch into neutral though which would be advisable. who knows though how someone operates under pressure like that. they did call 911 and you can hear the final moments iirc.
  - Nextgrid 4y ago
    > turning off your car is not a great idea since that locks up the steering wheel all but guaranteeing a crash To the best of my knowledge, steering locks only engage when the key is removed.
    - NoPicklez 4y ago
      Just to add, I have a keyless car and the steering lock is engaged when the car has been switched off, in park and the door has been opened. But I wonder what would happen if I was travelling and I just pressed the start/stop button, if the car would actually switch off.
      - velosol 4y ago
        If you have a safe area it's worth testing how your car works, usually push-to-start is actually a multi-state button between Ignition & Accessory On, Ignition Off & Accessory On, and All Off.
        NoPicklez 4y ago
99_00 4y ago
When this happened the vast majority of people I talked to were convinced that there was no unintended acceleration problem and that the US government was going after Japanese automakers to protect the investment it made in US automakers with the auto bailout (because of the 2008 financial crisis).
coupdejarnac 4y ago
Several years ago, my last company hired Denzo to write firmware for a new thermostat project. Denzo does electronics for Toyota. The software quality was so bad, it all had to be scrapped. Their software engineers were doing things like making their own mutexes instead of using the rtos builtins. It's been a while, so maybe there has been a reckoning since.
- ranger_danger 4y ago
  Japanese programmers are just terrible, also it's Denso.
jshprentz 4y ago
Prof. Koopman's list[1] of embedded system lecture notes and presentations includes a companion YouTube video[2] about the case study. [1] https://users.ece.cmu.edu/~koopman/lectures/index.html https://users.ece.cmu.edu/~koopman/lectures/index.html [2] https://youtu.be/DKHa7rxkvK8 https://youtu.be/DKHa7rxkvK8
mark-r 4y ago
I had a 2007 4Runner; I got the recall notices. I ignored them because I knew they were not getting to the root of the problem and were just trying to wallpaper over it.
- dotancohen 4y ago
  Though that may be true, you put yourself at liability in case of collision resulting in bodily harm. The recalls protect you from liability as well as wallpaper over existing issues.
Dorothy014 4y ago
[dead]
PaulHoule 4y ago
I'm no fan of the mitigation of killing the throttle if somebody stomps on the brake. If you're trying to go around corners as fast as possible in a front-wheel drive car you're going to drive with one foot on the brake and one foot on the gas and will apply the brakes and the gas at the same time. The article also says that global variables will kill you, putting variables on the stack will kill you but doesn't say where it is safe to put variables. I can't imagine the heap is any better, that a garbage collector isn't going to pause and kill you, etc. As an Arduino enthusiast I see statically allocated (global?) variables are frequently the way to go for an embedded system, the frontier is to be able to prove the correctness of what you're doing.
- LeifCarrotson 4y ago
  > The article also says that global variables will kill you, putting variables on the stack will kill you but doesn't say where it is safe to put variables. I can't imagine the heap is any better, that a garbage collector isn't going to pause and kill you, etc. Function-scope static or file-scope static variables are the best place for variables in an embedded system written in C. They don't pollute the shared namespace, nor do they require allocation and de-allocation. In an Arduino program, for example, you might have something like this, with a sketch.ino file: #include <stdint.h> #include "MyFile.h" uint32_t Bad_Global_Variable; // This doesn't conflict with same name in MyFile.cpp static uint8_t state = 1; void setup() { setup_myFile(); state = 0; } void loop() { loop_myFile(); } a MyFile.h header: #ifndef _MY_FILE_H_ #define _MY_FILE_H_ #include <stdint.h> void setup_myFile(void); void loop_myFile(void); #endif // End include guard and a MyFile.cpp library: #include "MyFile.h" static uint32_t MyFileScopeVariable = 0; // This doesn't conflict with same name in sketch.ino static uint8_t state = 0; void setup_myFile(void) { MyFileScopeVariable = 1; // Can't access or have conflicts with myFunctionScopeVariable here state = 1; } void loop_myFile(void) { static uint32_t myFunctionScopeVariable = 0; MyFileScopeVariable++; myFunctionScopeVariable++; } Statically allocated variables are definitely the only the way to go. Statically allocated global variables are easy and available, suitable for small embedded systems written by one person or a small group of collaborators who can be expected to know about every variable in the program or at least to be able to refactor their code if it collides with an existing name. Adding a prefix to your statically allocated global variables (eg. 'mf_state' for a global variable in the MyFile library above) is another way to reduce collisions, but isn't compiler-enforced. Statically allocated file-scope or function-scope variables are a best practice in an automotive ECU scale projects, and can reduce issues if you've got lots of vendors each contributing code and not a lot of visibility between projects.
bob1029 4y ago
Option B: Remove the computer and use a cable between the throttle body and the accelerator pedal, as was traditional for about 3/4 of a century.
- MengerSponge 4y ago
  Because physical throttle cables never got stuck for that 3/4 of a century.
  - MarcoZavala 4y ago
  - bob1029 4y ago
    The failure mode of a physical throttle cable is almost always much more gradual than with a computer bug, which can effectively kill you instantly without warning.
    - jandrese 4y ago
      My grandfather had a truck on his farm with an occasionally sticky throttle. He failed to mention this when I had just gotten my learners permit and was trying to get some practice. So the family hopped in the back and he took shotgun so we could drive over to the cattle pasture and check on the herd. Halfway there I hit a bump in the dirt road and accidentally smacked the throttle with my foot, which causes it to stick on the floorboard just as we are approached the stream to crossing. Luckily for everyone involved my grandfather knew what to do and slapped the stick into neutral before we all went careening over the rocky crossing. I was a bit slower getting the clutch in and the brakes on being a new driver. It was pretty weird feeling stopped in the middle of a stream with brakes fully on and the engine roaring. He then reached over and turned off the key, making me feel like an even bigger idiot for not thinking of it. The point is if you're not mentally prepared for a runaway engine it's hard to figure it out on the fly in the mere moments you have before your doom. Follow on note: Grandpa gave that truck to my uncle a few years later and my uncle managed to blow up the engine almost immediately. Turns out some of the head bolts were not fully tightened down at the factory but my grandfather drove it so gently that it wasn't a problem except for a persistent but low grade oil consumption issue for its entire life.
kirykl 4y ago
Keep in mind the US Gov also owned a large amount of General Motors debt at the time
- GiorgioG 4y ago
  What does this have to do with Toyota?
  - adolph 4y ago
    Since Toyota was a competitor to GM it implies a conflict of interest on the government's part.
darknavi 4y ago
This is from 2014 by the way.
RcouF1uZ4gsC 4y ago
> I testified as a Plaintiff expert witness This slide is written by someone that has a bias. I have not seen any independent confirmation that the Toyota problem was caused by a software issue (and indeed the slides mention that NASA looked at it and did not find such an issue).
- matthewdgreen 4y ago
  NASA looked at it well before the analysis mentioned in the slide deck, and it was a much shorter analysis and simply did not find this flaw. Toyota had their own technical experts and plenty of opportunity to publish technical reports rebutting it. I am not aware of any place where they did so, and they certainly had the opportunity (nobody is muzzling them.) I believe that maybe Toyota did argue that there was no evidence that this flaw caused the actual events in the field, which is true but also not a rebuttal exactly.
- photochemsyn 4y ago
  The list of issues seems pretty independent of any bias of the presenter: > Copyright 2014, Philip Koopman. CC Attribution 4.0 International license. • Poor isolation of task functions • “Kitchen Sink” “Task X” both computes throttle angle AND is responsible for many of the failsafes (same CPU, same task). Brake Override function in 2010 MY Camry is in this same task. [Bookout 2013-10-14 AM 80:5-82:16] • OSEK RTOS not certified; 80% CPU load (> 70% RMA limit) [Bookout 2013-10-14PM 42:6-25] [NASA App. A p. 119] • Many large functions – 200 functions exceeded 75 lines of non-comment code [NASA App. A p. 23] • Reviews informal and only on some modules [Bookout 2013-10-11 PM 29:24-30:5; 2013-10-14 49:17-21] • No formal specifications [Bookout 2013-10-11 PM 29:24-30:5] • No bug tracking system [Bookout 2013-10-14 PM 49:3-50:23] • No configuration management [Bookout 2013-10-11 PM 30:7-10] That doesn't look good. No bug tracking system? Large functions with no comments? Certainly not best practices as I've ever seen them described.
  - elfchief 4y ago
    They also followed almost none of the guidelines in the MISRA C standard, which is a standard developed by the automotive industry specifically for helping to ensure that code running on car computers is up to at least some standard.
pjdesno 4y ago
When this first came out, I recall reading that the reason the watchdog timer was reset from the timer interrupt was because as they added more software and overloaded the CPU, things got slower and the timer started triggering during normal operation. This caused the engine to hiccup if the vehicle was moving, or stall if you were at a stop. (IIRC reboot takes ~125ms, so no spark during this time) The watchdog is the last-ditch failsafe, in a safety-critical system, and they basically disabled it because it was inconvenient. In some businesses (aviation?) you could probably be convicted of something for doing that, whether or not it ever hurt anyone.
- elfchief 4y ago
  One thing that I found amusing/interesting/horrifying back when I read about this originally is that you can measure the computer reboot time in feet. IIRC, in the problem vehicles, the reboot time was something like 150 feet @ 60mph.
  - hatsunearu 4y ago
    Well, good thing is during that reboot time your brakes will likely work to its full potential, you'll have engine braking, your clutch will work, your transmission might work if you have a separate TCM, your steering will work with power steering. You will just lose power, as though you let off the throttle fully, during this time. In fact your car turns off fueling when you let off the throttle fully to conserve fuel. I think the only danger is that if you were going full tilt on a sports car, and the throttle failed close, you will get lift off oversteer probably.
    - winrid 4y ago
      > "your transmission might work if you have a separate TCM" This is why well engineered vehicles have a separate computer for everything and not just one little chip.
- civilized 4y ago
  What's infuriating is, think about how much of that software was probably worthless bloat for features no one needs. As many as 89 people may have died so Toyota could claim Bluetooth and MP3s in their latest models.
foxyv 4y ago
I used to have a Honda Civic with a sticky throttle. Luckily it was a manual transmission so it wasn't a big deal. Just push in the clutch and pull the pedal up with my toe then replace/repair the throttle whenever is convenient. No unintended acceleration and no fiery wrecks. I kind of wonder if we're heading the wrong direction with drive by wire cars. In aerospace it makes a lot of sense, especially when you are super dedicated to safe coding. But in a consumer vehicle? Not so great. Then again, Boeing hasn't been doing great with it either.
- subinsebastien 4y ago
  (1) Automotive (2) Medical Devices (3) Aerospace In my mind, all three are done with more or less similar rigour conforming to strict standards. Yet, exceptions are seen in all of them.
  - bborud 4y ago
    I remember when this was happening and I was surprised to learn that not only were there no good standards in automotive, reputable manufacturers like toyota didn’t even follow their own internal standards when developing software for critical bits. Or to rephrase: it would appear they made no credible effort given the types of defects that occurred.
  - bombcar 4y ago
    The problem is the first is regularly not maintained anywhere near the levels required, especially as the car gets older.
  - tgsovlerkhgsel 4y ago
    In my mind, the software for at least the first two is typically written by people who don't primarily have an IT background, and at least in the past it was very clear that no skilled IT security engineers with any even reasonably recent knowledge were involved.
    - slenk 4y ago
subinsebastien 4y ago
Here is the YouTube video. https://youtu.be/NCTf7wT5WR0 https://youtu.be/NCTf7wT5WR0
a-dub 4y ago
throttle angle function - 1300 lines
Overtonwindow 4y ago
When the driver pressed the brake pedal, why doesn't the vehicle disengage the throttle? I remember when this happened and many demonstrations were done that even if you were pressing the pedal to the floor, the brake disengages the throttle and brings the car to a stop.
- HPsquared 4y ago
  Ironically on a car with vacuum assisted brakes and no mechanical vacuum pump, it's the other way round: when full throttle, there is no vacuum available and (after a few uses) the brakes become substantially weaker. That's on naturally-aspirated engines without a software override, anyway.
  - 0xfeba 4y ago
    But they'll typically have a check valve good for one assist. Assuming you press once and only once and *hard*, they will win. But that's a big assumption to make for people panicking.
    - jandrese 4y ago
      Also, the general guidance for someone who thinks their brakes are failing is to pump them to try to build pressure. In this case that guidance is the exact opposite of what you should do. The guidance is for a different but more common failure mode: brake line rupture. The correct solution to a runaway engine is to slap the car into neutral and then apply brakes and pull over if necessary. But that's not something most driving schools teach nor is it something your average person will practice.
    - throwaway0a5e 4y ago
      >But they'll typically have a check valve good for one assist 2-3 in practice.
- 0xfeba 4y ago
  Some do (at least now) but there's no standard/regulation for it.
nvusuvu 4y ago
My father was gifted a 1972 Buick Regal. I was asked to drive it on I-85 (interstate) During acceleration on the onramp, the cable controlling the throttle got stuck and the gas was flowing into the carb and UA continued to occur. I was reaching speeds in excess of 100 MPH (160 Kmph). The brakes did nothing to slow me me down. My wife, driving adjacent to me in another vehicle, thought I was showboating or just being wreckless. I had to kill the engine with the key, slow down, and then crank the car, begin accelerating, then kill the engine to 'manage' the vehicle until I could get off the interstate. Terrifying ordeal.
- tempnow987 4y ago
  I was boating and same issue - broke to full throttle on a gas engine. So I just came flying in and killed engine. I could have restarted probably if needed. These things are almost always stoppable. Another boat had a diesel. This boat had serious flooding and I went over to help ($1M+ boat, owner had backed it hard into concrete pier and not noticed a transom crack that opened). Engine would not shut off (electrical all shorted out). That diesel engine was running full tilt almost totally submerged - was pretty impressive. Eventually someone got a mask and snorkel and dove to fuel cutoff before it ingested a bunch of water through air intake (which I'm sure would have stopped it). With diesels I always liked knowing where manual fuel cutoff was after that.
  - moron4hire 4y ago
    A place I worked years ago was a manufacturer of industrial lifting cranes. They had a problem one particularly hot summer of the diesels in the cranes starting up in their own. These were massive engines. The block was bigger than my whole car. Terrifying watching one of them run away. They tended towards "violent, unscheduled disassembly" if they couldn't be shut down.
  - hatsunearu 4y ago
    Hijacking (kinda): diesels are vulnerable to a phenomenon called "diesel runaway", where unintended fuel (for instance, engine oil leaks) seems into the intake/cylinder. The engine is controlled by how much fuel is injected, so if you just add an extra amount of fuel it will start increasing torque all on its own, and in the worst case, will cause unintended acceleration. Nowadays diesels actually come with throttle bodies for emissions reasons which I think also will serve a safety function against runaways.

89 points by georgecmu 4y ago

If you write dirty code knowing it is going to be used in a car, you should be charged with manslaughter, children endangerment and any form of legal punishment arising from creating long functions with high cyclomatic complexity being called in spaghetti in the context of controlling a vehicle that can cause people to die. Seriously, when will the excuses for dirty code end? Whatever you think the business is, clearly killing 89 people is not it. Use a linter and remember your contribution to the business as a programmer is programming properly. If programming is below you go do something else. If you have management aspirations go interview for a management job. If you don't like coding properly go code something else or consider another career. Write proper code and stop talking about how dirty code is good for the business. Not only it is not true, nobody wants to hear about it, and nobody should have to deal with your 4d jenga tower in a professional setting, goddamn it.

shadowgovt 4y ago

This actually marks the second incident I had tucked away in my brain where the initial story was "Sometimes the world is just weird" and the actual issue was computer systems. The reason we all keep seatbelts bucked in flight even when the sign is off these days is Qantas Flight 72, which took two sharp nose-dives in flight, rebounding passengers and crew off the ceiling and causing severe injuries. At the time, initial reporting was that they'd caught two freak mid-Pacific downdrafts, which had me nervous about flying for awhile. Turns out, the root cause was bad angle-of-attack data coming in from a faulty inertial measurement unit, which should have been caught by a safety system but proved so aggressively faulty that the safety system was overruled. It resulted in the plane's avionics becoming convinced it was flying belly-on into the wind at cruising speed and about to stall, so it did the thing it's supposed to do: pitched down hard to stall-recover. Don't ask me why I'm more comfortable with the story being "Sometimes computer engineers screw up" than "Sometimes our understanding of the physics of aviation just breaks down," but I am. ;)

shadowgovt 4y ago

From a software design and engineering perspective, I agree with the NASA report's take on global variable use. ... but I've also done game development, where maximum performance in minimum time is absolutely essential, and I wonder if it was actually feasible to maintain the performance the control system needed without global state. There's also the possibility of a culture clash. I've noticed that in embedded systems architecture development, global variables aren't frowned upon nearly as hard as they are in enterprise software, mostly because of the industry history of doing more with less. So I imagine a software culture risk developing from getting a bunch of hardware guys on a project where they now have access to microcontrollers with more memory than ever, but their risk tradeoff mindset is still "take the risk to squeeze every byte because we might run out."

XnoiVeX 4y ago

All motorcycles come with a standard electronic kill switch. Should cars be mandated to have them too?

throwaway-jim 4y ago
I don't understand where there is a kill switch when you can just turn off your vehicle using the key? Besides killing the engine will mean no power to the brakes and steering as well.
- michaelt 4y ago
  Motorcycle kill switches are always on the handlebars, so you can reach it without letting go of the handlebars. Also makes it easier to find - some bikes have the keyswitch on the side, but the kill switch is always on the handlebars.
- sokoloff 4y ago
  The brake booster will hold some vacuum for a few brake applications and, if the car remains in gear, the rotation will continue to create vacuum to recharge the brake booster (and turn the power steering pump, but power steering is not needed in practice at anything approaching “runaway car” speed).

jonsen 4y ago

I haven't heard of any incidents since that time. Did they fix all the cars, or what?

MBCook 4y ago
My understanding is it was never clear if it actually happened in the first place or was just operator error. I seem to remember there was also a recall on floor mats that could have caused issues. All the software stuff was discovered as a result of lawsuits. The software was atrocious and may have been capable of causing issues. But I don’t think anyone ever proved it did to any reasonable degree.

deleted 4y ago

[deleted]

tempnow987 4y ago

I'm always curious in these cases. Was the key not usable? Ie, turn off vehicle? Was neutral not available? To get my car to go I need key in the on position AND the gear in drive. I can go to neutral and turn off key anytime. These stories of barreling down the highway with key locked, throttle locked, gear selector locked seem bogus.

weaksauce 4y ago
you going 100mph and turning off your car is not a great idea since that locks up the steering wheel all but guaranteeing a crash. I am pretty sure most cars should be able to switch into neutral though which would be advisable. who knows though how someone operates under pressure like that. they did call 911 and you can hear the final moments iirc.
- Nextgrid 4y ago
  > turning off your car is not a great idea since that locks up the steering wheel all but guaranteeing a crash To the best of my knowledge, steering locks only engage when the key is removed.
  - NoPicklez 4y ago
    Just to add, I have a keyless car and the steering lock is engaged when the car has been switched off, in park and the door has been opened. But I wonder what would happen if I was travelling and I just pressed the start/stop button, if the car would actually switch off.
    - velosol 4y ago
      If you have a safe area it's worth testing how your car works, usually push-to-start is actually a multi-state button between Ignition & Accessory On, Ignition Off & Accessory On, and All Off.
      - NoPicklez 4y ago

99_00 4y ago

When this happened the vast majority of people I talked to were convinced that there was no unintended acceleration problem and that the US government was going after Japanese automakers to protect the investment it made in US automakers with the auto bailout (because of the 2008 financial crisis).

coupdejarnac 4y ago

Several years ago, my last company hired Denzo to write firmware for a new thermostat project. Denzo does electronics for Toyota. The software quality was so bad, it all had to be scrapped. Their software engineers were doing things like making their own mutexes instead of using the rtos builtins. It's been a while, so maybe there has been a reckoning since.

ranger_danger 4y ago
Japanese programmers are just terrible, also it's Denso.

jshprentz 4y ago

Prof. Koopman's list[1] of embedded system lecture notes and presentations includes a companion YouTube video[2] about the case study. [1] https://users.ece.cmu.edu/~koopman/lectures/index.html https://users.ece.cmu.edu/~koopman/lectures/index.html [2] https://youtu.be/DKHa7rxkvK8 https://youtu.be/DKHa7rxkvK8

mark-r 4y ago

I had a 2007 4Runner; I got the recall notices. I ignored them because I knew they were not getting to the root of the problem and were just trying to wallpaper over it.

dotancohen 4y ago
Though that may be true, you put yourself at liability in case of collision resulting in bodily harm. The recalls protect you from liability as well as wallpaper over existing issues.

Dorothy014 4y ago

[dead]

PaulHoule 4y ago

I'm no fan of the mitigation of killing the throttle if somebody stomps on the brake. If you're trying to go around corners as fast as possible in a front-wheel drive car you're going to drive with one foot on the brake and one foot on the gas and will apply the brakes and the gas at the same time. The article also says that global variables will kill you, putting variables on the stack will kill you but doesn't say where it is safe to put variables. I can't imagine the heap is any better, that a garbage collector isn't going to pause and kill you, etc. As an Arduino enthusiast I see statically allocated (global?) variables are frequently the way to go for an embedded system, the frontier is to be able to prove the correctness of what you're doing.

LeifCarrotson 4y ago
> The article also says that global variables will kill you, putting variables on the stack will kill you but doesn't say where it is safe to put variables. I can't imagine the heap is any better, that a garbage collector isn't going to pause and kill you, etc. Function-scope static or file-scope static variables are the best place for variables in an embedded system written in C. They don't pollute the shared namespace, nor do they require allocation and de-allocation. In an Arduino program, for example, you might have something like this, with a sketch.ino file: #include <stdint.h> #include "MyFile.h" uint32_t Bad_Global_Variable; // This doesn't conflict with same name in MyFile.cpp static uint8_t state = 1; void setup() { setup_myFile(); state = 0; } void loop() { loop_myFile(); } a MyFile.h header: #ifndef _MY_FILE_H_ #define _MY_FILE_H_ #include <stdint.h> void setup_myFile(void); void loop_myFile(void); #endif // End include guard and a MyFile.cpp library: #include "MyFile.h" static uint32_t MyFileScopeVariable = 0; // This doesn't conflict with same name in sketch.ino static uint8_t state = 0; void setup_myFile(void) { MyFileScopeVariable = 1; // Can't access or have conflicts with myFunctionScopeVariable here state = 1; } void loop_myFile(void) { static uint32_t myFunctionScopeVariable = 0; MyFileScopeVariable++; myFunctionScopeVariable++; } Statically allocated variables are definitely the only the way to go. Statically allocated global variables are easy and available, suitable for small embedded systems written by one person or a small group of collaborators who can be expected to know about every variable in the program or at least to be able to refactor their code if it collides with an existing name. Adding a prefix to your statically allocated global variables (eg. 'mf_state' for a global variable in the MyFile library above) is another way to reduce collisions, but isn't compiler-enforced. Statically allocated file-scope or function-scope variables are a best practice in an automotive ECU scale projects, and can reduce issues if you've got lots of vendors each contributing code and not a lot of visibility between projects.

bob1029 4y ago

Option B: Remove the computer and use a cable between the throttle body and the accelerator pedal, as was traditional for about 3/4 of a century.

MengerSponge 4y ago
Because physical throttle cables never got stuck for that 3/4 of a century.
- MarcoZavala 4y ago
- bob1029 4y ago
  The failure mode of a physical throttle cable is almost always much more gradual than with a computer bug, which can effectively kill you instantly without warning.
  - jandrese 4y ago
    My grandfather had a truck on his farm with an occasionally sticky throttle. He failed to mention this when I had just gotten my learners permit and was trying to get some practice. So the family hopped in the back and he took shotgun so we could drive over to the cattle pasture and check on the herd. Halfway there I hit a bump in the dirt road and accidentally smacked the throttle with my foot, which causes it to stick on the floorboard just as we are approached the stream to crossing. Luckily for everyone involved my grandfather knew what to do and slapped the stick into neutral before we all went careening over the rocky crossing. I was a bit slower getting the clutch in and the brakes on being a new driver. It was pretty weird feeling stopped in the middle of a stream with brakes fully on and the engine roaring. He then reached over and turned off the key, making me feel like an even bigger idiot for not thinking of it. The point is if you're not mentally prepared for a runaway engine it's hard to figure it out on the fly in the mere moments you have before your doom. Follow on note: Grandpa gave that truck to my uncle a few years later and my uncle managed to blow up the engine almost immediately. Turns out some of the head bolts were not fully tightened down at the factory but my grandfather drove it so gently that it wasn't a problem except for a persistent but low grade oil consumption issue for its entire life.

kirykl 4y ago

Keep in mind the US Gov also owned a large amount of General Motors debt at the time

GiorgioG 4y ago
What does this have to do with Toyota?
- adolph 4y ago
  Since Toyota was a competitor to GM it implies a conflict of interest on the government's part.

darknavi 4y ago

This is from 2014 by the way.

RcouF1uZ4gsC 4y ago

> I testified as a Plaintiff expert witness This slide is written by someone that has a bias. I have not seen any independent confirmation that the Toyota problem was caused by a software issue (and indeed the slides mention that NASA looked at it and did not find such an issue).

matthewdgreen 4y ago
NASA looked at it well before the analysis mentioned in the slide deck, and it was a much shorter analysis and simply did not find this flaw. Toyota had their own technical experts and plenty of opportunity to publish technical reports rebutting it. I am not aware of any place where they did so, and they certainly had the opportunity (nobody is muzzling them.) I believe that maybe Toyota did argue that there was no evidence that this flaw caused the actual events in the field, which is true but also not a rebuttal exactly.
photochemsyn 4y ago
The list of issues seems pretty independent of any bias of the presenter: > Copyright 2014, Philip Koopman. CC Attribution 4.0 International license. • Poor isolation of task functions • “Kitchen Sink” “Task X” both computes throttle angle AND is responsible for many of the failsafes (same CPU, same task). Brake Override function in 2010 MY Camry is in this same task. [Bookout 2013-10-14 AM 80:5-82:16] • OSEK RTOS not certified; 80% CPU load (> 70% RMA limit) [Bookout 2013-10-14PM 42:6-25] [NASA App. A p. 119] • Many large functions – 200 functions exceeded 75 lines of non-comment code [NASA App. A p. 23] • Reviews informal and only on some modules [Bookout 2013-10-11 PM 29:24-30:5; 2013-10-14 49:17-21] • No formal specifications [Bookout 2013-10-11 PM 29:24-30:5] • No bug tracking system [Bookout 2013-10-14 PM 49:3-50:23] • No configuration management [Bookout 2013-10-11 PM 30:7-10] That doesn't look good. No bug tracking system? Large functions with no comments? Certainly not best practices as I've ever seen them described.
- elfchief 4y ago
  They also followed almost none of the guidelines in the MISRA C standard, which is a standard developed by the automotive industry specifically for helping to ensure that code running on car computers is up to at least some standard.

pjdesno 4y ago

When this first came out, I recall reading that the reason the watchdog timer was reset from the timer interrupt was because as they added more software and overloaded the CPU, things got slower and the timer started triggering during normal operation. This caused the engine to hiccup if the vehicle was moving, or stall if you were at a stop. (IIRC reboot takes ~125ms, so no spark during this time) The watchdog is the last-ditch failsafe, in a safety-critical system, and they basically disabled it because it was inconvenient. In some businesses (aviation?) you could probably be convicted of something for doing that, whether or not it ever hurt anyone.

elfchief 4y ago
One thing that I found amusing/interesting/horrifying back when I read about this originally is that you can measure the computer reboot time in feet. IIRC, in the problem vehicles, the reboot time was something like 150 feet @ 60mph.
- hatsunearu 4y ago
  Well, good thing is during that reboot time your brakes will likely work to its full potential, you'll have engine braking, your clutch will work, your transmission might work if you have a separate TCM, your steering will work with power steering. You will just lose power, as though you let off the throttle fully, during this time. In fact your car turns off fueling when you let off the throttle fully to conserve fuel. I think the only danger is that if you were going full tilt on a sports car, and the throttle failed close, you will get lift off oversteer probably.
  - winrid 4y ago
    > "your transmission might work if you have a separate TCM" This is why well engineered vehicles have a separate computer for everything and not just one little chip.
civilized 4y ago
What's infuriating is, think about how much of that software was probably worthless bloat for features no one needs. As many as 89 people may have died so Toyota could claim Bluetooth and MP3s in their latest models.

foxyv 4y ago

I used to have a Honda Civic with a sticky throttle. Luckily it was a manual transmission so it wasn't a big deal. Just push in the clutch and pull the pedal up with my toe then replace/repair the throttle whenever is convenient. No unintended acceleration and no fiery wrecks. I kind of wonder if we're heading the wrong direction with drive by wire cars. In aerospace it makes a lot of sense, especially when you are super dedicated to safe coding. But in a consumer vehicle? Not so great. Then again, Boeing hasn't been doing great with it either.

subinsebastien 4y ago
(1) Automotive (2) Medical Devices (3) Aerospace In my mind, all three are done with more or less similar rigour conforming to strict standards. Yet, exceptions are seen in all of them.
- bborud 4y ago
  I remember when this was happening and I was surprised to learn that not only were there no good standards in automotive, reputable manufacturers like toyota didn’t even follow their own internal standards when developing software for critical bits. Or to rephrase: it would appear they made no credible effort given the types of defects that occurred.
- bombcar 4y ago
  The problem is the first is regularly not maintained anywhere near the levels required, especially as the car gets older.
- tgsovlerkhgsel 4y ago
  In my mind, the software for at least the first two is typically written by people who don't primarily have an IT background, and at least in the past it was very clear that no skilled IT security engineers with any even reasonably recent knowledge were involved.
  - slenk 4y ago

subinsebastien 4y ago

Here is the YouTube video. https://youtu.be/NCTf7wT5WR0 https://youtu.be/NCTf7wT5WR0

a-dub 4y ago

throttle angle function - 1300 lines

Overtonwindow 4y ago

When the driver pressed the brake pedal, why doesn't the vehicle disengage the throttle? I remember when this happened and many demonstrations were done that even if you were pressing the pedal to the floor, the brake disengages the throttle and brings the car to a stop.

HPsquared 4y ago
Ironically on a car with vacuum assisted brakes and no mechanical vacuum pump, it's the other way round: when full throttle, there is no vacuum available and (after a few uses) the brakes become substantially weaker. That's on naturally-aspirated engines without a software override, anyway.
- 0xfeba 4y ago
  But they'll typically have a check valve good for one assist. Assuming you press once and only once and *hard*, they will win. But that's a big assumption to make for people panicking.
  - jandrese 4y ago
    Also, the general guidance for someone who thinks their brakes are failing is to pump them to try to build pressure. In this case that guidance is the exact opposite of what you should do. The guidance is for a different but more common failure mode: brake line rupture. The correct solution to a runaway engine is to slap the car into neutral and then apply brakes and pull over if necessary. But that's not something most driving schools teach nor is it something your average person will practice.
  - throwaway0a5e 4y ago
    >But they'll typically have a check valve good for one assist 2-3 in practice.
0xfeba 4y ago
Some do (at least now) but there's no standard/regulation for it.

nvusuvu 4y ago

My father was gifted a 1972 Buick Regal. I was asked to drive it on I-85 (interstate) During acceleration on the onramp, the cable controlling the throttle got stuck and the gas was flowing into the carb and UA continued to occur. I was reaching speeds in excess of 100 MPH (160 Kmph). The brakes did nothing to slow me me down. My wife, driving adjacent to me in another vehicle, thought I was showboating or just being wreckless. I had to kill the engine with the key, slow down, and then crank the car, begin accelerating, then kill the engine to 'manage' the vehicle until I could get off the interstate. Terrifying ordeal.

tempnow987 4y ago
I was boating and same issue - broke to full throttle on a gas engine. So I just came flying in and killed engine. I could have restarted probably if needed. These things are almost always stoppable. Another boat had a diesel. This boat had serious flooding and I went over to help ($1M+ boat, owner had backed it hard into concrete pier and not noticed a transom crack that opened). Engine would not shut off (electrical all shorted out). That diesel engine was running full tilt almost totally submerged - was pretty impressive. Eventually someone got a mask and snorkel and dove to fuel cutoff before it ingested a bunch of water through air intake (which I'm sure would have stopped it). With diesels I always liked knowing where manual fuel cutoff was after that.
- moron4hire 4y ago
  A place I worked years ago was a manufacturer of industrial lifting cranes. They had a problem one particularly hot summer of the diesels in the cranes starting up in their own. These were massive engines. The block was bigger than my whole car. Terrifying watching one of them run away. They tended towards "violent, unscheduled disassembly" if they couldn't be shut down.
- hatsunearu 4y ago
  Hijacking (kinda): diesels are vulnerable to a phenomenon called "diesel runaway", where unintended fuel (for instance, engine oil leaks) seems into the intake/cylinder. The engine is controlled by how much fuel is injected, so if you just add an extra amount of fuel it will start increasing torque all on its own, and in the worst case, will cause unintended acceleration. Nowadays diesels actually come with throttle bodies for emissions reasons which I think also will serve a safety function against runaways.