5 ms·

The Migration Of Engine ECU Software From Single-Core To Multi-Core

48 points by zdw 5y ago

JonChesterfield 5y ago
Well that's horrifying. Not where I want multithreaded code. Definite risk of it being written in C with volatile~=atomic as the working assumption.
- KennyBlanken 5y ago
  People found the concept of computer control over an idle stabilization valve terrifying in the early 80's. Killer cars, everyone predicted. It never materialized. The lawsuits against Audi never went anywhere, and Audi even won some counter-suits. Most cars have drive-by-wire throttles. Hundreds of millions of cars out there, and DBW throttle failures leading to crashes are unheard of. You know what is an issue? Ford unable to make ignition switches that don't cause cars parked in house garages to catch fire. GM cheaping out on their own ignition switches, causing them to suddenly fail while driving....and thus shutting off the engine, ABS, and airbag systems, something that is known to have caused the deaths of half a dozen people and likely tens if not hundreds of other Cruze owners.
  - cillian64 5y ago
    I don’t have a source for this but I recall reading that mechanical throttle wires are actually not especially reliable so drive-by-wire doesn’t have to be super reliable in order to be better. But perhaps it’s the same as self driving cars where people get especially disturbed by computers screwing up which sets the reliability bar higher than is otherwise logical.
    - gameswithgo 5y ago
      Have you ever had either fail on you? I never have. I have heard of a few failures but always on very modified or old vehicles. Some failure modes can happen whether actuated by wire or motor as well. it isn’t a first order concern really, dozens of higher risk, more complex things going on like tires, brakes, explosives in the steering wheels.
      - dtgriscom 5y ago
        Anecdata isn't helpful here; when you are one of millions of users, thousands of people may be inconvenienced (or killed) before you have personal experience with a problem.
  - NullPrefix 5y ago
    >DBW throttle failures leading to crashes are unheard of I doubt Michael Hastings crash would have happened if he didn't drive by wire.
  - sebazzz 5y ago
    > Most cars have drive-by-wire throttles. Cars where? Many cars here in Europe have a valve directly mechanically linked to the pedal. Any car without cruise control has it that way.
- tonmoy 5y ago
  While I agree with you in general about being careful about the pitfalls of multi-threading, the automotive industry is fairly competent at safety and correctness. They are probably the only few industry making good use of formal verification. In short, I am less worried about cars not working due to misunderstanding of C language than any other device like my PC or iPhone
- tsomctl 5y ago
  I've been reverse engineering a modern ABS control module. It has 2 cores, and they are separate CPU's with their own memory (although one cpu can access the RAM of the other CPU) and role. One reads/writes the CAN bus and does verification on that data, the other CPU is the one that monitors wheel speed sensors and determines when to control the solenoid valves. There is a cleanly defined area where data is copied from one CPU to the other.
culopatin 5y ago
Can anyone enlighten me why you need multi thread to control something of much slower frequency than a CPU? I think there is more to be gained in the sensors than in a faster ECU. Of course I’m only speaking about Engine control, not all the other stuff in the car.
- potatochup 5y ago
  CPUs in the automotive space are primarily chosen for being cheap, and with the right number/kind of IO. If you do enough stuff¹, it's certainly possible to reach the CPU load limits of a single core CPU, and it might cheaper to upgrade to a dual core than put in another ECU (also, faster ECUs with the same IO/arch might not be available). I've also seen arrangements where the dual-cores have separate memory spaces, and one core acts like a safety processor/monitor while the other is the main compute module. 1: examples of CPU intensive stuff include active damping, active aero (spoilers), active rear steering, steer by wire, etc. You may think these things are irrelevant w.r.t. an engine ECU, but if the engine ECU is responsible for producing torque, then systems like traction control, stability control and etc are all directly related to torque production
  - rurban 5y ago
    I've worked in the field, even with the biggest and fastest ECU's in existance, Formula 1, with 10Khz cycles on 4Ghz CPU's we never saw the need to use the other cores. the CPU is a few million times faster than the cycles, if there is a problem it's a HW IO or blocking driver. even with normal 500Hz ECU's you can always upgrade the CPU freq to run faster, than needing more cores. even better would be to use better drivers with zero copy and MMU access. it's nice that threading got better recently in embedded Realtime OS's, HW always gets better, but certainly drivers not and SW not. and developers even less. maybe they think rust will help, but then they should chose a proper concurrency-safe system. certainly not rust, which only lies about it. also this thing is only simulation verified, not formally verified.
    - koollman 5y ago
      which systems would you consider "proper concurrency-safe" ?
olliej 5y ago
Can't wait for all the ... race conditions Thank you, thank you, I'll see myself out
- bitwize 5y ago
  Enjoy the veal, everybody! Tip your waitstaff!
bastijn 5y ago
From 2014-2017 I have participated in a large European subsidiary project [0] that focused on the major industrial challenge arising from the need to face cost efficient integration of different applications with different levels of safety and security on a single computing platform. Use of multi-core fpgas and the like in aviation, automative and medical industry is an enormous challenge. Mixed-critical, multi-core programming models would mostly make sure critical tasks would run entirely on their isolated core and registers leaving the remainder for non-critical tasks. That solved being able to buy common hardware but did not allow programs to actually benefit from multi-core boards. [0] http://www.emc2-project.eu/ http://www.emc2-project.eu/
guai888 5y ago
How do they plan to get through ISO 26262 verification? You do not need to do HARA to know ECU is ASIL D. ASIL D Item requires developer to do Modified Condition and Decision Coverage. Running through all the test cases will be a huge task. The golden rule for Automobile Software is simplicity. Always keep it simple unless it is absolutely necessary. They need to read through ISO 26262 chapter 6.
bob1029 5y ago
Few have demonstrated actual competency with using 1 computer. I struggle to comprehend a reality in which auto manufacturers or their vendors are able to manage more than 1 without causing net harm to themselves or society. The only arguments for more compute power in cars boil down to distracting the driver or making them feel like they are somewhat less responsible for the laws of physics.

48 points by zdw 5y ago

Well that's horrifying. Not where I want multithreaded code. Definite risk of it being written in C with volatile~=atomic as the working assumption.

KennyBlanken 5y ago
People found the concept of computer control over an idle stabilization valve terrifying in the early 80's. Killer cars, everyone predicted. It never materialized. The lawsuits against Audi never went anywhere, and Audi even won some counter-suits. Most cars have drive-by-wire throttles. Hundreds of millions of cars out there, and DBW throttle failures leading to crashes are unheard of. You know what is an issue? Ford unable to make ignition switches that don't cause cars parked in house garages to catch fire. GM cheaping out on their own ignition switches, causing them to suddenly fail while driving....and thus shutting off the engine, ABS, and airbag systems, something that is known to have caused the deaths of half a dozen people and likely tens if not hundreds of other Cruze owners.
- cillian64 5y ago
  I don’t have a source for this but I recall reading that mechanical throttle wires are actually not especially reliable so drive-by-wire doesn’t have to be super reliable in order to be better. But perhaps it’s the same as self driving cars where people get especially disturbed by computers screwing up which sets the reliability bar higher than is otherwise logical.
  - gameswithgo 5y ago
    Have you ever had either fail on you? I never have. I have heard of a few failures but always on very modified or old vehicles. Some failure modes can happen whether actuated by wire or motor as well. it isn’t a first order concern really, dozens of higher risk, more complex things going on like tires, brakes, explosives in the steering wheels.
    - dtgriscom 5y ago
      Anecdata isn't helpful here; when you are one of millions of users, thousands of people may be inconvenienced (or killed) before you have personal experience with a problem.
- NullPrefix 5y ago
  >DBW throttle failures leading to crashes are unheard of I doubt Michael Hastings crash would have happened if he didn't drive by wire.
- sebazzz 5y ago
  > Most cars have drive-by-wire throttles. Cars where? Many cars here in Europe have a valve directly mechanically linked to the pedal. Any car without cruise control has it that way.
tonmoy 5y ago
While I agree with you in general about being careful about the pitfalls of multi-threading, the automotive industry is fairly competent at safety and correctness. They are probably the only few industry making good use of formal verification. In short, I am less worried about cars not working due to misunderstanding of C language than any other device like my PC or iPhone
tsomctl 5y ago
I've been reverse engineering a modern ABS control module. It has 2 cores, and they are separate CPU's with their own memory (although one cpu can access the RAM of the other CPU) and role. One reads/writes the CAN bus and does verification on that data, the other CPU is the one that monitors wheel speed sensors and determines when to control the solenoid valves. There is a cleanly defined area where data is copied from one CPU to the other.

culopatin 5y ago

Can anyone enlighten me why you need multi thread to control something of much slower frequency than a CPU? I think there is more to be gained in the sensors than in a faster ECU. Of course I’m only speaking about Engine control, not all the other stuff in the car.

potatochup 5y ago
CPUs in the automotive space are primarily chosen for being cheap, and with the right number/kind of IO. If you do enough stuff¹, it's certainly possible to reach the CPU load limits of a single core CPU, and it might cheaper to upgrade to a dual core than put in another ECU (also, faster ECUs with the same IO/arch might not be available). I've also seen arrangements where the dual-cores have separate memory spaces, and one core acts like a safety processor/monitor while the other is the main compute module. 1: examples of CPU intensive stuff include active damping, active aero (spoilers), active rear steering, steer by wire, etc. You may think these things are irrelevant w.r.t. an engine ECU, but if the engine ECU is responsible for producing torque, then systems like traction control, stability control and etc are all directly related to torque production
- rurban 5y ago
  I've worked in the field, even with the biggest and fastest ECU's in existance, Formula 1, with 10Khz cycles on 4Ghz CPU's we never saw the need to use the other cores. the CPU is a few million times faster than the cycles, if there is a problem it's a HW IO or blocking driver. even with normal 500Hz ECU's you can always upgrade the CPU freq to run faster, than needing more cores. even better would be to use better drivers with zero copy and MMU access. it's nice that threading got better recently in embedded Realtime OS's, HW always gets better, but certainly drivers not and SW not. and developers even less. maybe they think rust will help, but then they should chose a proper concurrency-safe system. certainly not rust, which only lies about it. also this thing is only simulation verified, not formally verified.
  - koollman 5y ago
    which systems would you consider "proper concurrency-safe" ?

olliej 5y ago

Can't wait for all the ... race conditions Thank you, thank you, I'll see myself out

bitwize 5y ago
Enjoy the veal, everybody! Tip your waitstaff!

bastijn 5y ago

From 2014-2017 I have participated in a large European subsidiary project [0] that focused on the major industrial challenge arising from the need to face cost efficient integration of different applications with different levels of safety and security on a single computing platform. Use of multi-core fpgas and the like in aviation, automative and medical industry is an enormous challenge. Mixed-critical, multi-core programming models would mostly make sure critical tasks would run entirely on their isolated core and registers leaving the remainder for non-critical tasks. That solved being able to buy common hardware but did not allow programs to actually benefit from multi-core boards. [0] http://www.emc2-project.eu/ http://www.emc2-project.eu/

guai888 5y ago

How do they plan to get through ISO 26262 verification? You do not need to do HARA to know ECU is ASIL D. ASIL D Item requires developer to do Modified Condition and Decision Coverage. Running through all the test cases will be a huge task. The golden rule for Automobile Software is simplicity. Always keep it simple unless it is absolutely necessary. They need to read through ISO 26262 chapter 6.

bob1029 5y ago

Few have demonstrated actual competency with using 1 computer. I struggle to comprehend a reality in which auto manufacturers or their vendors are able to manage more than 1 without causing net harm to themselves or society. The only arguments for more compute power in cars boil down to distracting the driver or making them feel like they are somewhat less responsible for the laws of physics.