15 ms·

Ask HN: Risk of unsafe software in automobiles?

75 points by jacobevelyn 4y ago

I may buy a car (new or used) soon and am a little worried about all the software in cars these days. Software can control pretty much every aspect of a modern vehicle, and so the idea of bugs in a vehicle's software scares me from a safety perspective. Poor software engineering has been implicated in automobile safety incidents in the past[1]. I'm aware of the NASA/JPL rules for developing safety-critical software[2] but I'm not sure if any car manufacturers follow anything similar. Does anyone here have any knowledge of the software development practices of any automakers and what they do to ensure safety and reliability? And is there anything else I can do to mitigate this risk (short of buying a very old car, which would have other safety downsides)? [1] https://en.wikipedia.org/wiki/Sudden_unintended_acceleration https://en.wikipedia.org/wiki/Sudden_unintended_acceleration [2] http://spinroot.com/gerard/pdf/P10.pdf http://spinroot.com/gerard/pdf/P10.pdf

Tomte 4y ago
ISO 26262.
PaulHoule 4y ago
I am scared of the infotainment system myself as it could distract you to death. That goes for cell phone and tablet and the stick-on GPS which gets confused in the most complex urban areas, falls into your lap when the suction cup fails, etc.
- SoftTalker 4y ago
  My big complaint is with the transition of controls from dedicated, tactile knobs, switches, and levers to touch-screen buttons or menus which demand more visual attention (i.e. distraction from driving) to operate.
  - izzydata 4y ago
    This is my least favorite trend in new cars. If there aren't any new cars in 10 years that aren't just a giant screen in the center then I guess I'm going to be walking. You can make a modern electric vehicle with actual buttons and dials. There is nothing about a car not having a gas motor that requires every tiny bit of functionality being controlled by a touch tablet. If anything it just seems like laziness in car design.
  - jimbomins 4y ago
    You may get your wish. Recent research has shown that real knobs and switches are much less of a driver distraction than trying to fiddle with screens or get voice commands correct. The two either mean you look away from the driving because you can't just feel your way across a screen like physical controls or you add cognitive load whilst thinking and talking. I'm with you and hope all the idiot touch screen crap is ditched.
    - vel0city 4y ago
      > I'm with you and hope all the idiot touch screen crap is ditched. I agree many manufacturers have gone a bit overboard in making even things like vent positions and glove boxes behind touchscreens. But at the same time, I don't want touch screen to completely disappear. Punching in a destination and controlling the navigation interface is way better with a touchscreen than using a dial. As a passenger or while stopped, changing the media with a touchscreen interface is better than a dial. Changing a lot of the finer settings in the car (ones you wouldn't be doing while driving) with a touchscreen is better than scrolling through menus with a dial. For the most part, a lot of things you'd do with the system while not moving or while being the passenger can be better on the screen than with a bunch of physical controls, controls which would necessitate making the screen a lot smaller. Also, then when wanting to quickly reference the navigation system, having a larger screen with larger items on it means it is easier to understand what it is telling the driver in a glance. Having a ton of physical controls means it is a much smaller map, meaning smaller items on the map, meaning harder to understand at a quick glance. It's nice having the next turn up in the driver's information cluster or on a heads-up display as well to reduce the needs for the driver to reference the larger navigation system. There definitely needs to be a balance of physical controls versus software buttons, but I wouldn't buy a new car without a screen these days.
    - twiddling 4y ago
      Muscle memory to know that the third button in past the volume knob turns on the windshield defroster, vs. looking at a menu on a screen whilst driving...
  - mikehodgson 4y ago
    Mazda did the research, and transitioned back to real tactile controls in 2019. I expect more automakers will be or are already following their lead. https://www.motorauthority.com/news/1121372_why-mazda-is-purging-touchscreens-from-its-vehicles https://www.motorauthority.com/news/1121372_why-mazda-is-pur...
    - ModernMech 4y ago
      I have a 2016 Mazda and they had tactile controls then. What they did in 2019 was remove the touch screen feature on the center display, but there were always tactile controls to access the display. I have a 2022 Mazda and it's largely the same interface as the 2016 controls.
    - ptmcc 4y ago
      I love Mazda for resisting the touchscreen-everything car interior. Everything in my 2021 CX-5 is physical buttons and knobs. The design and ergonomics are just fantastically nice and usable. Like you said, the infotainment is also not a touchscreen and is rotary control only. While not as immediately intuitive as a touchscreen, once you get the hang of it it is much safer and more accurate to use while driving. Trying to use a touchscreen while moving is just awful. And I don't think it is wise to optimize short term intuitiveness over long term safety and usability for a vehicle I will own for several years.
- sokoloff 4y ago
  Indeed, the software most likely to kill or injure you in a car is running on someone’s smartphone. It’s probably not even close.
mcqueenjordan 4y ago
ISO 26262 is the functional safety standard that automobile manufacturers adhere to. Furthermore, companies with a strong safety culture may also have other safety controls, including MISRA, AUTOSAR, and others. I think reputable car companies take this stuff very seriously, but your concern is also well placed.
- addaon 4y ago
  ISO 26262 is optional, and some car makers (e.g. Tesla) may choose not to use it directly; but in general they use another process that they feel provides equivalent safety properties, and feel so strongly enough that they're comfortable substantiating this in the inevitable lawsuits.
- AlotOfReading 4y ago
  26262 covers system failures that produce unsafety, like broken accelerator sensors. ISO 21448 (SOTIF) covers system failure to detect and respond to the environment appropriately (e.g. does the radar detect small children?), which is also a common concern for people in my experience. Both are acceptable standards, but ISO 26262 is a behemoth of a standard that most people have never read. Many companies don't even make the full standard available to their development teams, let alone educate people to employ it effectively. Similarly, MISRA is fine in theory, but the practical usage often ends with running code through an automatic checker that can only detect half the rules.
- uncletammy 4y ago
  > I think reputable car companies take this stuff very seriously, but your concern is also well placed. I trusted Volkswagen because of their reputation. Then the news broke about them systematically lying and breaking the law with respect to engine emissions. Shortly after this came to light, other "reputable car companies" turned out to have been not trustworthy at all. Yes there are good standards in place and some companies claim to adhere to them but no company should be trusted on their word or reputation alone. The better question is what kind of regulatory oversight is in place to make sure those claiming to adhere to certain standards are actually doing so? Also, how much power do the regulatory organizations have in addressing violators?
weakfortress 4y ago
A long time ago I attended a DEFCON where this was discussed. Long before it became a big deal in the industry to have all this tech in cars. CANBUS was broken reliably, and if my memory serves me they even had a car you could take a shot at hacking yourself. After playing with it the entire conference I came to the conclusion I would never own a modern car if I can avoid it. Any car running CANBUS is vulnerable to a potentially fatal attack. They have not resolved this. However, you also generally cannot avoid it. Even the base model Honda civic is vulnerable to attacks on the drive-by-wire system. In a less morbid sense, most modern cars cannot even be serviced at home without going to the dealer for a reset of whatever subsystem. ABS comes to mind. I would not detract from an old car. A car 25 years old has 99% of the safety features of a modern car and, in good working order, will protect you just the same. Or maybe I just don't worry about it because the probability of anything greater than a minor fender bender killing you is pretty high even with modern tech.
- vel0city 4y ago
  > CANBUS was broken reliably So can my brake or fuel lines, if you're needing physical access. Get this, the door locks aren't even 100% secure, there's this whole thing that side steps them called "Windows". I do lament not being able to fully flush my brakes at home and do wish the programming harness would be freely available to override the system and have the ABS clear the lines. However, I wouldn't for a second choose to not have ABS on any vehicle I own, including my motorcycle.
- jdminhbg 4y ago
  When it comes to potentially fatal attacks on my car, I'm a lot more worried about drunk drivers than CANBUS.
  - vel0city 4y ago
    In which case, having a more modern car with better safety designs is way better than not having a CANBUS. And I definitely agree; I'm way more likely to be harmed because of a drunk driver or someone running a red because they're just too busy to bother stopping at this light today rather than some hacker remoting into my car to change the car from drive to neutral or remotely disable ABS or something like that.
vel0city 4y ago
Your link of Sudden unintended acceleration contains a lot of entries related to this issue which don't involve computers at all. It lists pedal misapplication, entrapped pedals, stuck throttles, electrical shorts, and diesel engine runaway as other things which can cause such an issue. A lot of the reported incidents had nothing to do with software. Either way, if you've had a fuel injected car you were still exposed to these issues. You would have to go buy a carbureted engine from the 80s or before to get away from these "unintended acceleration" issues, as in the end a car with EFI probably has a computer actually controlling the injection. I'd be way more wary of daily driving an 80s or older car from a general safety standpoint than a software issue. You're way more likely to be t-boned at an intersection than a software glitch causing an accident; having a much more modern car will help from a crash safety standpoint than having a carburetor. There's a ton of things that can go wrong in a car which can cause an accident. The software stack is surely one of those things, but even a 100% mechanical car can have a lot of failures as well. Ever have vacuum hoses fail on an old car? Carburetors get stuck or clogged? Personally, I'd prefer a computer controlling components directly instead of tons of vacuum lines and springs trying to keep things tuned right. On top of that I'll also get much better efficiency and reduce harmful emissions which hurt my family and my neighbors.
- dsfyu404ed 4y ago
  >There's a ton of things that can go wrong in a car which can cause an accident. And pretty much none of them ever do if the driver doesn't react exceptionally poorly. Even the spectacular stuff that the internet absolutely loves to hand wring about, like a wheel falling off for whatever reason, almost always results in the car coming to a controlled stop on the side of the road. The conversion rate between "failures" and "meaningful harm to anyone or anything is abysmal."
  - vel0city 4y ago
    Agreed. The biggest component failure leading to injuries is by far the one in the driver's seat.
  - kube-system 4y ago
izzydata 4y ago
All of that firmware that isn't a self driving autopilot blackbox is probably quite safe. Such as anti-lock breaks and fuel injection timing. I am never going to put my life in the hands of some software doing image analysis using machine learning.
- notdonspaulding 4y ago
  > I am never going to put my life in the hands of some software doing image analysis using machine learning. Well...on your car, at least. I'm not sure how comforting that approach is when you're surrounded on the interstate by Tesla "FSD"s.
  - izzydata 4y ago
    I'm ok with doing a bit of defensive driving against Teslas or bad drivers in general.
  - filoleg 4y ago
    Ironically, I feel at much more ease driving surrounded by Tesla "FSD"s, as opposed to being surrounded by average Seattle drivers. At least FSD always uses turn signals when switching lanes and is way more cautious (while still remaining reasonable). Back when I spent a full year commuting to work daily on a motorcycle, Teslas with "FSD" were the least of my worries. All while people who sharply switched into my lane in front of me with no turn signals used, they ended up almost killing me a couple of times.
protortyp 4y ago
If you're interested in this topic, you might want to check out ASRG[1]. It's a community of safety and security professionals in the industry working to improve standards. They share their research openly and have regular in-person and online events where you can reach out to people. [1] https://asrg.io/ https://asrg.io/
- AlotOfReading 4y ago
  "Safety" and "Security" are considered separate topics in the automotive world, even though there's some overlap between them. There are usually completely separate teams with different skillsets responsible for each at any given manufacturer. ASRG is pretty focused on the security side rather than the safety side.
  - owalt 4y ago
    Short version of the difference: Safety = This shouldn't cause harm. Security = This should be hard to hack.
- uconnectlol 4y ago
  It's sad that this and many other admirable movements like langsec will never amount to much because of the false dilemma that everything needs to be complicated because of "reasons" (politics, in automobiles, and "the poor user who needs a complicated to implement interface that changes every day" in the case of langsec). That is, most electronics in vehicles are obviously not needed, and if it was up to me infotainment would just be illegal aside from a basic physical knob to change volume of some audio in the 3.5mm jack (and not by sending a digital signal to a system that makes the user readjust it several times after the large input delay). I was being half serious but now that I think of it bluetooth and "smart" tech and such atrocities should just be illegal. They are like what's already accepted as illegal harmful Chinese products.
luciusdomitius 4y ago
I don't think it is bugs which worries you, but rather the completely wrong concept of modern day car production. Almost all cars built in the past 20-years have 0 compliance with the laws of physics, being extremely front-heavy and with transversely-mounted engines - disbalanced even on the Y-axis. What keeps them from spinning is the so-called DSC/ESC/ESP which is basically a neural network. We all know how reliable those are. It is in fact a very similar situation to the 737 Max, just on wheels. Due to regulations you will not be able to find a non-veteran vehicle without those systems, nor you'd want to, but BMW, Mercedes, Subaru and Lexus still have models which are well balanced and don't rely on those to such a heavy degree. This would be my advise as well. Disclaimer: I am not against (almost) perfectly deterministic safety systems such as ABS. On the contrary - I consider them to be a massive advantage or almost mandatory.
- jimbomins 4y ago
  You're kidding if you think the premium brands you list don't have massive dependence on software systems. The basic physics is, as you say, better because they've kept mass distribution in line with how it was. And how about the additional failure mode for say BMW. It's modern controllers have all the software for all the features in, just disabled unless you pay more. So a theoretic sophisticated attack could throw all sorts of crap into operation.
- kylecordes 4y ago
  I don’t have numbers in front of me, but my impression is that carmakers have done a much better job in the last decade with front-rear balance decent on nearly all cars. Versus decades past when some were wildly front-heavy (or occasionally, wildly back-heavy, ouch).
- slt2021 4y ago
  Most BMWs have nearly perfect 50/50 weight balance between front and rear axles. Generally speaking, most sportier cars have better handling and better weight distribution and are in fact regularly driven with DSC disabled (on racing track etc)
Throw4949 4y ago
[dead]
greenthrow 4y ago
Cars have had software controlling essential functionality for decades. It didn't suddenly appear when touchscreens did.
- f1shy 4y ago
  Now is totally different. The software used to be done with graphical tools, with boxes like simulink, but with only basic logical and arithmetic operations. Now they are trying to write C++.
Mikeb85 4y ago
Thankfully, most non-Tesla manufacturers don't trust critical systems to software.
- deleted 4y ago
  [deleted]
smt88 4y ago
Don't buy a Tesla and you'll be fine. I work in auto insurance and the other OEMs actually care about safety and testing of software. Tesla has the most bugs by far.
- manscrober 4y ago
  I don't know about tesla since I haven't had a chance to drive one yet, but the number of bugs especially in the interface but also in driver's "assistance" in other cars(VW, BMW) makes me doubt that it makes a noticeable difference - at least with tesla there is a chance they will fix it
  - kube-system 4y ago
    Many of the electronic systems in traditional automaker's vehicles are made by a variety of other OEMs anyway. So judging by automaker brand may not really be a good judge of the engineering development of those subsystems. The infotainment might be made by Panasonic and the driver assistance might be made by Bosch.
    - piceas 4y ago
      They are interconnected though. Reputation for faults add up. To me the following is vw's fault. I've recently had an id4's location fail, and the map was all over the place. It got worse as I drove on. Annoying, but I thought not a safety issue until the speed limiter caused the car to attempt to slow down by more than 100kph as it was reporting it was somewhere else.
    - someweirdperson 4y ago
      The industry is really good at inventing the wheel multiple times. Even in a simple case: Two car manufacturers, same supplier for the same kind of control unit - the hardware will likely look similar, but most likely both are running almost completely different software.
      - kube-system 4y ago
w_t_payne 4y ago
The problem is mostly a cultural one. In my personal experience, the automotive industry has a problem with aggression and dishonesty, both of which seem to go hand-in-hand. Both of these cultural traits tend to have a negative impact on quality and safety.
KaiserPro 4y ago
So the answer is yes, and no. > https://illmatics.com/carhacking.html https://illmatics.com/carhacking.html is a good starting point. But there are a bunch of buses on a modern car, some of them are critical, some less so. Some are firewalled off, others are open. As you know you can get access to a lot of the car's inner workings by plugging into the ODB2 port. Its perfectly possible to brick some cars by fuzzing the ODB2 port. In principle, most things in cars _should_ fail safe. even if they are electric or talking over a bus of somesort.
- fmntf 4y ago
  OBD is a diagnostic bus. On modern car the access is authenticated on several layers (eg. guest, carmaker, ECU supplier). I would not call that interesting. Accessing a CAN/FlexRay/LIN/Eth bus is better.
staunch 4y ago
1. Cars have had computers in them for a very long time now. 2. The code, in many cases, is probably an unmaintainable mess. Embedded programming is not always modern programming, for good and bad. 3. Today, the computers in cars are doing more, and the systems are more complex. It's reasonable to expect more serious problems as a result. 4. Companies do safety testing, of course, but there's no such thing as as "100%" test coverage for complex physical machines running outside of a lab. 5. The best way to judge the safety of cars is the best way to judge safety for airplanes: let other people test them out for a while and then check whether or not they report problems.
- f1shy 4y ago
  This is absolutely the case. Until more or less now, the software was made with systems like simulink (but much much simpler), where a mechanical engineer could build blocks of very simple functions, using AND, NOT, OR gates, and mathematical operations. Now the companies are migrating to real programming in C++, and it is a terrible mess. There are just not enough people with software competence to drive it. I've seen people trying to do L4 automated systems with this blocks. Pages and pages and pages of boxes (which can only be the basic logical function, and the 4 basic arithmetic operations!!!). Of course the project didn't go anywhere!
- fmntf 4y ago
  Your comment is pretty accurate. Just remove "probably" from point 2.
freedomben 4y ago
Your concern is well founded given the proliferation of utter shit that is modern car "entertainment" centers. Scare quotes because some safety important things are handled by those. If you don't think so, drive my 2017 Ford Expedition when it's -10F outside and the windshield fogs up beyond visibility levels because the damn defroster (and other environment controls) doesn't work and and the windshield is fogging up. Or try to back up and see the backup-camera glitching out. However, most of the safety systems software is held to a very high standard, and much happens in embedded systems where the surface level for software foot-guns (such as state) are minimal. I wouldn't worry about buying a new car for these reasons. Though I would try to find one with as many physical buttons as possible.
kkfx 4y ago
I know a connected car, as most modern cars are, can potentially be controlled from remote. That means you can get OTA upgrades that 99% of the times will work flawlessly, but a day may do not, the day you are in a rush in the early morning. Since most connected cars are de-facto owned by their vendor a potential breach or deliberate sabotage might brick ALL at once across the globe or in some specific areas/countries. ... A modern car is a car co-piloted by a human and a computer. A local airgapped computer might have bugs, a connected one might have vulnerabilities. Be more scared about them. In mere local safety terms I can say most cars I know are partially mechanical that means for instance your steering wheel can auto-steer BUT with (more than) a bit of force you can steer it mechanically even if automation completely fail. Similar the break pedal have some servo systems but still partially work in mechanical forms, so might became very hard to push but still able to break a bit. The most dangerous common design I know are: - impossibility to turn off certain ADAS who might act really badly in certain weather condition, like the classic ABS on icy roads; - automatic doors lock when car move, NO DAMN WAYS to unlock them while the car still moving; - manual parking break disappeared so a kind of emergency breaking ALSO usable by a passenger (for instance if the driver fell ill suddenly) ABSENT and no electronic replacement either since the electronic one if present refuse to engage if the car is moving; - cockpit design that makes very hard/slow for a passenger to push the driver feet out of accelerator etc if he/she fell ill suddenly. I consider the above as a sign of VERY BAD design, so I doubt those who made it can be trusted for anything else in safety terms...
slt2021 4y ago
There is a risk of automobile security. Nowadays cars have vast attack surface and uplink accesses directly into internal car's CAN/Ethernet bus: OBD-II port, bluetooth, GSM/5G, WiFI, NFC, access via OEM's web portal (these are big piles of unsecure code), via mobile app API, dealer network applications. Plus OEMs have a vast parts and software supply chain that can be compromised. I suspect that in couple years timeframe we can see massive incident, like ransomware, that will disable entire fleet of a single OEM globally. Like imagine all Mercedes around the world to just stop operating - these kind of incidents
- uconnectlol 4y ago
  This is the correct take. Also just found yet another problem puiblished a few weeks ago: https://medium.com/@doctoreww/day-2-your-car-is-trackable-by-law-1d5f74388850 https://medium.com/@doctoreww/day-2-your-car-is-trackable-by... This stuff falls completely within any infosec person's expectations. Privacy leaks are expected, as are interference from remote signals.
f1shy 4y ago
No amount of MISRA or ISO26262 or tests or any kind will help if the people doing the software are direct out-of-the-university mechanical engineers or physicist, who had at most one or two semesters of some kind of programming course. The reality is that this is the current state of affairs. Most of people doing software for cars have not the foggiest idea what software is really about. All the software I read is just impossible to understand. And no standard help in many cases. Some examples I've seen in code: - Use of kind of hungarian notation to the point that a loop variable was named something like "uibe32bb_i_lns" - Comments in other human languages that were not english - Use of recursion - Have seen a call like name1::name2::name3::name4::name5::name6::name7::name8::name9::name10::name11::name12::name13::name14. The names where some kind of hungarian notation, those calls where everywhere in the code. - Lines more than 1000 characters wide, as a rule - Files north of 100kB of code I can go on and on and on.... Some examples of exchanges with people: 1) Software architect, of a ECU: one programer asks for the memory and CPU budget for a function. The reply was "I'm the architect, I've no idea what you are talking about" 2) System chief architect, for a very important project of a big auto-maker: one engineer says something about software errors. The architect interrupts, and explains that the software never makes an error. Because a computer only does what it is told to make. -- that is terrible enough, for example ignoring the possibility or a SEU, but he goes further, to say that any kind of test is not necessary, because, SW, as stated, makes no errors. Some general points: - 99% of people in "SW" do not know what gdb is. They debug by "cout <<" - I found nobody who knows what tail recursion is - 90% are only able to program, to some extent, in one of C++ or Python, but no other language. - Mentioning Ada, Lisp, Forth will trigger a waterfall of insults saying those are old and should never be used. I keep buying the most basic cars. I'm genuinely terrified to think in anything automatic in my car.
- slt2021 4y ago
  you would be more terrified after interaction with security engineers at OEMs - people who hold keys for the OEM's entire kingdom with admin access to everything running outdated OSs/systems well past last day of support date
  - f1shy 4y ago
simonbarker87 4y ago
Thoughts from my wife who has worked in electrical and software for OEM automakers (high volume, luxury sport and start up) for 10 years: (I’m typing while she is, ironically, driving our Volvo) To answer your last question first, buy a car that hasn’t been launched within the last 12 to 18 months. That’s not software specific, that general vehicle safety across the board as they will be working through the initial warranty issues. So if you are looking at second hand and you know model ABC was launched 2016, don’t buy one made in the 2016/2017 period. ISO 26262 rates every system on a critically rating, if they have a ASIL rating of C or D they have multiple back up systems in place. This falls under functional safety which is a newer (5 years or so) area targeting that cars are now highly complex interconnect systems linked with software - the idea being that you target specific subsystems to make sure their function isn’t totally taken out due to some failure or error in the wider system. Cyber security wise there is an EU reg coming in from 2024 making sure that OTA updates are safe, reducing hacking attack vectors and the like. This is being introduced to new cars and designs as a result of the issues cited above. As far as people hacking in via the infotainment to access the car control systems - there are firewalls between infotainment and primary car control to mitigate against that issue. There multiple networks in a single vehicle to isolate systems so that no one central unimportant system (infotainment for eg) can take out the whole vehicle. Software in cars to this level is new, it’s evolving and it takes 7 or so years to create a new platform. This means there is a lag in the system, especially during this transitionary period. However car makers take this stuff incredibly seriously and their software teams are absolutely not run in the same way as a lower consequence dev situation. Lives are on the line and the type of devs who work in this field know that. Nothing is perfect but the safety downsides of an old car are widely considered to be far greater than the threat of hacking or bad code in a new car.
- simonbarker87 4y ago
  Follow up to this: The one thing that could cause a lot of problems for cars and software is Agile/Scrum. The projects that are being run in this, new for the industry way, are always late and people hate working on them. CEOs and other C suite people see the massively shorter lead times that software can offer and are getting greedy. They saved a year or more of time on a feature thanks to code and over the air and then they decide they want it made in 4 weeks, when 3 months would be prudent. There’s something about the intangibility of software that makes traditional automotive people’s brains break. Thankfully many rank and file engineers and PMs in OEMs are pushing back against Scrum etc so a more pragmatic layer of management will come up in the coming years. Sadly Agile/Scrum will cause some preventable issues in the meantime. Unlikely to be safety critical stuff due to the rounds of QA and safety council sign offs and gateways they need to go through. But less safety critical stuff may slip through.
londonReed 4y ago
As long as you're not buying a Tesla/other "self driving car", you will not be the cause of an accident due to automobile software.
NotYourLawyer 4y ago
I’m way more worried about having a wreck while distracted by some horrible touchscreen interface than I am about a bug causing a wreck. When will auto companies wake up and realize that physical controls are better in every way?
amelius 4y ago
Normally, cars require certification to be allowed on public roads. My main problem with software in automobiles is that vendors can change my car even without any certification agency involved in the process.
qbasic_forever 4y ago
You're thinking about this the wrong way. Don't optimize for the car behaving perfectly--like you said it's impossible for you to verify this. And even if you could formally prove a car is perfectly behaved, you are driving on streets with other cars and other unpredictable people who could just as easily crash into you. Optimize this problem by buying a car with the best safety rating. This is something that can be objectively measured, both in crash testing/labs and from reviews of real-world crash results. Expect that a crash could be inevitable as it is totally out of your control. Optimize for the best odds of surviving a crash without issues.
- Yujf 4y ago
  Problem is that if you do that, you make the road less safe for everyone else
  - vel0city 4y ago
    Only if you're then also willing to go up a size in vehicle. You can still choose more or less safe vehicles from within the same class. A 2022 Corolla is way safer than a 1998 Corolla, that 1998 Corolla is safer than a 1966 Corolla, etc.
sokoloff 4y ago
Vehicle software faults are pretty far down the injury risk depth chart. Once you've committed to never driving after having had a drink (and surely never more than 1 drink), never driving while tired or on medication, have completed several advanced driving courses/car control clinics, chosen the top cars based on safety and crash testing, only then might it make sense to use software development methods as a tie-breaker to pick a car.
uconnectlol 4y ago
If it has software in it, it's bad. As a hacker like any other who realized that all supposedly ultra safe American quality (TM) software in mission critical applications is in fact less secure on average than random amateur projects, I have been worried about software in vehicles for 20 years. I correctly predicted that it will lead to remote control vulnerabilities such as the uConnect vulnerability disclosed a decade later. There are obviously more of such vulnerabilities out there, just nobody is researching this. I also suggest people start looking at HVAC. In 2015, some security researchers found a vulnerability in the Chrysler Uconnect software which allowed them to connect to the car's IP address (yes, each car had an IP address, which you can't get rid of), and control the vehicle (as in actually control it). There were 1.5 million vehicles IIRC that were vulnerable to this. So if a bad guy found it first he could have controlled all those vehicles at once from the comfort of his home, probably causing 10% of them to crash and kill people (given that 1/10 of your average modern driver would probably panic (or not panic but still fuck up) from the slightest surprise on the road). I also am of the opinion that people regularly die from software faults in vehicles, but we just haven't figured this out yet. What is NASA/JPL rules? Some more misra C crap where it's just making the code more "readable"? Most "software engineers" have extremely wide gaps in their understandings of basic things from programming, to math, to physics. The problem has much more to do with this than cute little best practices recommendations.

jimbomins 4y ago

Faking those things is a totally different state of affairs to safe operation. I've been out of automotive for 20 years but never met anyone who would compromise normal operational safety. Let's face it most people who work on cars love to drive cars. They would be putting their own and families lives at risk.

moooo99 4y ago
> I've been out of automotive for 20 years but never met anyone who would compromise normal operational safety. There is nothing to gain with not taking safety seriously, but fortunes to loose. Many of the software components of traditional automakers are built by suppliers. If a serious safety issue would arise and cause fatalities, the suppliers and the OEMs would not only have to pay enormous amounts in damages, they'd also need to issue recalls that might even pose an existential threat to the company.
AlotOfReading 4y ago
I've also never seen intentional compromises, but I've seen unintentional compromises made through ignorance or lack of foresight plenty of times. The bigger automakers seem to be better at this by virtue of developed processes, but what I've seen from smaller companies is terrifying.
f1shy 4y ago
So you are oblivious to the history of Takata? Or the problem with the Ford Pinto, or the ignition key of GM, or the recursive function calls in Toyota code... just to name the most important that I can cite out of the top of my head.
- airbreather 4y ago
  Functional safety standards are performance based standards, which means they are shades of grey rather than prescriptive "follow these design clauses". The designers get together and in a formal process try and come up with every possible adverse outcome and the probability it is likely to occur. They then rank and use this info to assign performance requirements to various safety aspects and functions. But a key part of the overarching parent IEC61508 standard is that there is a safety lifecycle - the designers make their best guess but the manufacturer has to at regular intervals compare actual gathered data against the predicted design data used and adjust accordingly, iterating to a better place. Just like you might win the lottery first time you buy a ticket, under a performance based standard you might experience an adverse outcome in the first day of use, doesn't mean the design was necessarily deficient. Infinite safety takes infinite cost, which would mean no cars, and what would the cost of that be to society. Like I said,it's all shades of grey.
  - f1shy 4y ago
    My comment was in reply to "I've been out of automotive for 20 years but never met anyone who would compromise normal operational safety"

wolrah 4y ago

> CANBUS was broken reliably, and if my memory serves me they even had a car you could take a shot at hacking yourself. This statement shows a fundamental lack of understanding of how automotive computer networks operate. The CAN bus is just a network. It's an industrial control protocol that's been adopted by the automotive world. It doesn't offer security by design, it's intended for use in limited environments where all hardware on the network is known and trusted. CAN provides methods for prioritization of devices, that's it. Any security is left to higher layers of the stack. There is no such thing as "breaking" CAN, you just physically connect to the network and you're able to talk to whatever controllers are on that network (most modern cars have multiple CAN buses connected to different subsets of the vehicle systems). At that point it's about the security features implemented by the devices on the network. > Any car running CANBUS is vulnerable to a potentially fatal attack. They have not resolved this. There is nothing to resolve at the network level. To put it another way, almost every computer that's ever been hacked over the internet was running Ethernet but that's just as irrelevant as CAN in cars. If you are able to physically connect to the network, you can talk to and potentially spoof devices on the network. > A car 25 years old has 99% of the safety features of a modern car and, in good working order, will protect you just the same. You couldn't possibly be more wrong. Pick your favorite vehicle from 1997 and look up the crash test videos, then compare against a similar recent model. Here's the most popular vehicle sold in the US, the Ford F-150, from 1997 (https://www.youtube.com/watch?v=_i5EmJBaGeQ https://www.youtube.com/watch?v=_i5EmJBaGeQ) versus one from 2016 (https://www.youtube.com/watch?v=Cou88zi4pMY https://www.youtube.com/watch?v=Cou88zi4pMY). You tell me which one you'd rather be in. You might say, correctly, that the 1997 F-150 is particularly bad, but here you can see a 1997 Volvo V70 versus a 2009 Volvo V70 (https://www.youtube.com/watch?v=msnJK0ce-VM https://www.youtube.com/watch?v=msnJK0ce-VM). Volvo has a reputation for building some of the safest vehicles on the road, and even those twelve years show substantial gains in crash performance where the older car's passenger compartment is clearly compromised while the newer one's crumple zones work as intended. > Or maybe I just don't worry about it because the probability of anything greater than a minor fender bender killing you is pretty high even with modern tech. Again, absolutely wrong. I say this as someone who's flipped a truck off the road at highway speed and walked away with minor abrasions and bruising from the seatbelt and a few cuts from broken glass as the rest of the truck got ruined but the cab stayed intact. My anecdote is of course statistically meaningless, but the data agrees. Crash fatality rates have consistently trended downward from the '80s until 2020. The main reason modern vehicles have gained so much exterior size without gaining nearly as much interior size is all the space taken up by modern safety equipment, crumple zones, etc.

melony 4y ago

In their defense, gdb is a pain in the ass to use if your editor doesn't come with integrated support. The debugger that comes with CLion/Visual Studio is perfectly adequate. Ada is useful for automotive. Lisp and Forth, not so much (especially since Lisp isn't usually used in hard real-time applications). This isn't the 1980s, MCUs aren't that memory constrained. Knowledge of obscure programming languages doesn't necessarily make you a better software engineer. I want my automotive embedded engineer to have a solid grasp of computer architecture, real-time safety protocols, and defensive programming. I don't see a problem with a Korean/Japanese car manufacturer having their documentation in a non-English language. As long as they do everything in-house and don't outsource to India like Boeing I have no problem with it.

f1shy 4y ago
> In their defense, gdb is a pain in the ass to use if your editor doesn't come with integrated support. The debugger that comes with CLion/Visual Studio is perfectly adequate. I meant, they do not know what a debugger is. As stated, they use "cout << "At line 26, after call to xx" as debugging tool. For gdb there are plenty of python extensions, and GUIs, even web front-end, that are not bad... but it may be difficult for some people, I understand that. > Ada is useful for automotive. Lisp and Forth, not so much (especially since Lisp isn't usually used in hard real-time applications). This isn't the 1980s, MCUs aren't that memory constrained. Well, first, they do not have the foggiest idea what Ada is. That is my problem. Once somebody suggested we should look into it, for L4 autonomous driving. He was laugh at, and it was said "it is a dead language from the 60, like Cobol or Fortran, nobody has used it in 50 years, there are no compilers for it!!!". I've seen forth being used in some 8-bit uC in the automotive industry still. Now is 99% gone, but was very much used. Lisp, can be used in hard real-time. BTW another thing always hanging around is the "hard real-time" for automotive. It is interesting, because other than for airbags, ignition and injection, you are talking about 100ms response times, which can be achieved very easily. > Knowledge of obscure programming languages doesn't necessarily make you a better software engineer. I'm not taking proficiency in the languages, I'm talking knowing it exist, having an idea of what is possible. I mean, I know no good C programmer that is not at least aware or the existence or Rust. And no, 90% of the programmers writing safety critical SW have no idea that a language called rust is available. > I want my automotive embedded engineer to have a solid grasp of computer architecture, real-time safety protocols, and defensive programming. Well, again, another example, with a chief software architect, in an ECU, so embedded: I ask "do we have some kind of stack monitoring?" Reply: "What?! we have no stack, stack is a data structure, ..." goes on with a long explanation of what stack, queue and tree are, and when are used... My personal opinion is, if you search for people that "know" only C++, and have no idea of asm, that is what you get. That is my experience at least. > I don't see a problem with a Korean/Japanese car manufacturer having their documentation in a non-English language. As long as they do everything in-house and don't outsource to India like Boeing I have no problem with it. I'm talking in-code comments, not documentation. But anyway, honesty, thinking you can do everything in-house today, and you will be able to maintain that in that way for the next 10 or 20 years when you have to maintain the code, sounds optimistic to me, at least. But again, I'm talking code I had to read, and maintain... so... yes... I'm talking a case, where it was BAD to have not english comments.

eimrine 4y ago
It has been meant that there is no any car with FLOSS injector software. Every injector hardware devise has proprietary software. How on Earth can I drive an injector car if any proprietary software is not an option for me?
- vel0city 4y ago
  They weren't talking about FLOSS here. They weren't talking about free or proprietary software, they wanted no software > Personally, I prefer no computer control. They're not arguing for free software controlling their car, they're arguing for no software controlling the car. No software controlling the car means generally less safe cars with higher emissions on the road. Higher emissions hurt the people around them. Not having common safety systems like ABS and traction control make the cars harder to operate on normal roadways, leading to a higher likelihood of collisions. They're arguing they should be free to operate less-safe cars which emit higher emissions on public roadways, and that everyone around should just accept it. Arguing otherwise is arguing for an authoritarian socialist dystopia.
  - eimrine 4y ago
    > no software controlling the car. I understand that my opinion is heavily biased to RMS' philosophy, but if looking the facts, we can see that if the car (engine controller, ABS, comfort controller, pesky notifications about belts/weather/maintenance) would be controlled by FLOSS it is controlled by user (not manufacturer) de-facto. All the cons of software just disappear in FLOSS: if you don't like the pesky notifications or anything else - you just remove few lines from the source code, recompile and replace the binary in your car. I really prefer driving carburetor Volga with no software goodies just because there is no any car which has a software which let me use the software instead of being used by it. And I really prefer absence of modern safety like ABS if the safety is controlled by software manufacturer only. I know that my car has significant higher emissions (not to mention efficiency and safety) and I am a big fan of global warming movement but all I can do is to use my car as little as possible because using proprietary software is bigger sin for me than to poison our planet. > They're arguing they should be free to operate less-safe cars which emit higher emissions on public roadways I also talk about this, but in a slightly different way.
    - vel0city 4y ago
      > And I really prefer absence of modern safety like ABS if the safety is controlled by software manufacturer only. I'm really wondering where this mindset ends though. You're foregoing safety equipment which is proven to make you far safer on the roads because you can't access the firmware of the device. You're choosing a much older car with far worse safety designs because of this. How far would you take this idea? If you were seriously injured and in the hospital, would you refuse for the doctors to use machines which contain non-free software to save your life? Would you rather die than interface with non-free software? Don't get me wrong, I'm a generally FOSS supporting person and agree with many of RMS' ideas surrounding software freedom. If there were two cars out there which were roughly the same, one with free software and one with closed, I'd pick the free software version. But in the end I value my family's safety far more than my opinion on free software, so it is an obvious choice to choose the safer car.
      - eimrine 4y ago
        > which is proven to make you far safer on the roads because you can't access the firmware of the device. I don't like the wording. ABS is safer not because I can not alter the firmware, it is safer on different reason. > If you were seriously injured and in the hospital, would you refuse for the doctors to use machines which contain non-free software to save your life? Would you rather die than interface with non-free software? I see benefits of this approach in learning: I do not want to learn any proprietary framework. I do not care about the doctor's software, he is free to learn anything.
        vel0city 4y ago
        Language ambiguities are leading to confusion, my bad. I get it. With this sentence: > You're foregoing safety equipment which is proven to make you far safer on the roads because you can't access the firmware of the device The "because you can't access" is relating back to the "you're foregoing safety equipment" part of the sentence. You're choosing to not use proven safety equipment because you can't access the firmware of the ABS controller. I do agree it is not the fact that the ABS system is closed that makes it safer; it could easily be an open-source/FLOSS firmware and still provide similar safety features. > I do not want to learn any proprietary framework. I don't need to learn about the intricate details of how ABS systems work in order to benefit from having one. The doctors using those machines aren't necessarily knowing about how the ECG monitor actually takes the samples from the leads and renders the lines on the screen, but they benefit from having one of those tools. Do you also not use plain metal tools unless you completely understand their mixture and how the metal is made and fully comprehend how they're forged/machined/etc? Or do you sometimes just accept "yes, this is a screwdriver, it can drive screws." I don't know everything about how the local metro train system works and probably wouldn't be able to know everything about it, but I'll still ride on the train. I learn to use the resources available to me, even if some parts are opaque to me.