4 ms·

> It really should be illegal to have network connected cars. Any software security engineer knows that. Everybody loves network connected new cars. Nobody thin

> It really should be illegal to have network connected cars. Any software security engineer knows that. Everybody loves network connected new cars. Nobody thinks about the fact that they are "fly by wire" I fully agree that automotive and medical equipment/IT could greatly benefit from not having security as an afterthought. However, I don't know any vehicle with connectivity (other than that Jeep Cherokee controversy), which does not have safety critical CAN/FlexRay buses segregated from user facing 'infotainment' systems. What that means is that the network bus in which your 'compromised' infotainment system is able to operate is completely separate from Engine, ABS, AEB, ESP, Airbags etc. The solutions vary but there is usually a physical gateway that prevents a passthrough MITM attacks, so for example you cannot simple send a message frame from your infotainment pretending to be an Emergency braking request to your ABS system.

Silhouette 6y ago
The problem is that there is an increasing overlap between physical vehicle control systems, cabin controls and displays, and remote communications. For example, how do you design a system that can automatically call for help in the event of an accident without giving that system access to the sensors that also trigger airbag deployment and similar safety features? How do you build "self-driving" automation (or even less ambitious driver aids that are already in widespread use) without relying on sensors observing the environment around the vehicle, which may be subject to interference or deliberate deception?
- DoingIsLearning 6y ago
  All the use cases you described are already implemented in today's cars, bus segregation doesn't mean zero communication. Gateways can allow for certain messages to pass-through, the point is that they preserve the garantee that certain safety-critical messages can only originate from within a specific network. It's basically a crude MITM mitigation.
  - TeMPOraL 6y ago
    > All the use cases you described are already implemented in today's cars, bus segregation doesn't mean zero communication. That's the problem, though. Once bidirectional communication exists, you can pretty much guarantee there's an exploitable hole in it that can be used to break the firewall.
    - DoingIsLearning 6y ago
      These systems are crude but they implement a physical airgap. You would have to hack a gateway (on-site) or physically access a segregated bus, at which point you could argue that you could also physically tamper with brakes or engine without any software being involved.
      - Silhouette 6y ago
        Unfortunately, a physical air gap won't prevent a malicious actor from, for example, projecting a misleading image designed to confuse your automated driving systems when they process that image and so prompt an adverse and potentially dangerous reaction. This is not only about the communications channels in the internal architecture, it's a much broader problem than that.
        DoingIsLearning 6y ago
        Please read the whole thread, this is going completely off-topic from the original discussion in the start of these comments. The original claim was that vehicles should never be connected to any network because of unspecified online attacks that could actuate brakes or steering. I explained why this was unfounded. Adversarial patterns against computer vision based ADAS are only a real issue for system which are not sufficiently redundant . Autonomous systems in particular should also apply a degree of sensor fusion between multiple sources of data such as optical computer vision, radar, long range ultrasound and LIDAR (once it becomes cost effective). If any of those systems provides erroneous data the remaining ones can negate that and allow for a fail-safe behaviour. If you want my opinion, as someone who has moved away from automotive R&D a few years ago, Tesla's decision of depending too heavily in computer vision systems without addtional sensor redundancy seems like an architectural defect that has already cost lives. Either they are not integrating other sensor data sources or their voting weight appears underestimated.
marcosdumay 6y ago
I can think of at least one manufacturer that pushes motor performance and battery charging updates over the net.
- deleted 6y ago
  [deleted]
- jayd16 6y ago
  Teslas are still not drive by wire though.
  - ajxs 6y ago
    The article we're discussing would suggest that the attack surface for any kind of complex machinery is very wide. A car doesn't need to be drive-by-wire to be potentially endangered by malicious code affecting the engine or battery.
  - Veserv 6y ago
    At least the Model S is. It has remote summon and Autopilot which necessarily means that it has software controlled accelerator, steering, and brakes.
    - jayd16 6y ago
      That's not what "by wire" means though. My understanding is the steering wheel and brakes are mechanically attached to the steering and braking mechanisms.
      - bayindirh 6y ago
        "by wire" means non-physical ways in automotive. So accelerator and possibly brake pedals have only cable connections which carry data. Even if the brake pedal is connected via an actual wire, emergency brake actuators can work by themselves on the system. Electric assist motors on the steering column can also overpower a human with ease. So, actual connections doesn't matter.
Veserv 6y ago
What justification do you have for asserting that the safety critical systems are actually separated/isolated? The people at Jeep probably thought their systems were secure, yet the researchers on the Jeep Cherokee attack demonstrated they were not and claim that they could have simultaneously affected 471,000(!) vehicles [1]. Do you think anybody at a car company would dare to claim that their systems are safe enough to bet the lives of 471,000 people on them and take responsibility if they were wrong? Do you think any engineer would, in good conscience, support such a statement and share that responsibility if asked directly? I doubt you could find a single engineer that would feel even remotely comfortable that their procedures are good enough for 1/100th that number and you would never find an executive who would dare claim such a thing in a legally binding way. Absolutely zero benefit of the doubt should be given to systems where a single error could cause grievous harm to tens of thousands of lives. Such systems should require an absolutely ironclad public legally-binding positive assertion of security with strict criminal liability for failure (blame can not be shifted) that is independently validated before we should even think of accepting their use. We already accept nothing less for systems that are less dangerous such as nuclear reactor meltdowns and systems that are equally dangerous such as nuclear weapons systems. If systems with failure modes many times worse than the atomic bombings of Japan can not be made safe, then they must not be made. Anything less would be so criminally irresponsible that we lack a word for the magnitude of irresponsibility. [1] https://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/ https://www.wired.com/2015/07/hackers-remotely-kill-jeep-hig...
- AlotOfReading 6y ago
  I'd guess the fact that GM hired those guys for Cruise [1] probably suggests they took it seriously and made sure their own cars were isolated. [1] https://www.detroitnews.com/story/business/autos/mobility/2017/07/31/gm-hires-jeep-hackers-cruise-automation/104158212/ https://www.detroitnews.com/story/business/autos/mobility/20...
  - Veserv 6y ago
    This is exactly the sort of justification that I am cautioning against. Would it be great if they did take the problem seriously and solve it? Of course. But it is not our duty to justify their decisions when the consequences of their decisions may cause grievous harm to thousands. It is their job, the company's, to justify to the satisfaction of engineers and the people that we should trust the lives of thousands to millions on their systems. If anything, it is our job to be skeptical of their claims and grill them ruthlessly before they should even be allowed to consider deploying such systems. I mean, think of any other software system, would you trust anybody's lives to any of them? Would the engineers on those projects agree? Most software engineers I know would be horrified if their systems were used in something responsible for even a single life let alone thousands. The claim that we should trust these companies with hundreds of thousands of lives is an extraordinary claim that they should be required to provide extraordinary evidence for. And, even if we are convinced, they should still be ultimately liable for their choice of decision to make sure they are actually putting their money where their mouth is. It is what we demand from bridge builders, it is what we demand from nuclear reactor designs, it should be what we demand from mass-produced internet-connected safety-critical devices.
Retric 6y ago
Tesla’s has over the air updates for it’s autopilot software, you even edit destinations from the infotainment system. So, it’s clearly both on the same network and could be hacked remotely. The car can even be turned on and summoned remotely. Which means if nothing else it could be remotely driven onto a freeway to cause accidents even if it’s ‘autopilot’ had some kind of independent backup safely system to avoid collisions.

Silhouette 6y ago

The problem is that there is an increasing overlap between physical vehicle control systems, cabin controls and displays, and remote communications. For example, how do you design a system that can automatically call for help in the event of an accident without giving that system access to the sensors that also trigger airbag deployment and similar safety features? How do you build "self-driving" automation (or even less ambitious driver aids that are already in widespread use) without relying on sensors observing the environment around the vehicle, which may be subject to interference or deliberate deception?

DoingIsLearning 6y ago
All the use cases you described are already implemented in today's cars, bus segregation doesn't mean zero communication. Gateways can allow for certain messages to pass-through, the point is that they preserve the garantee that certain safety-critical messages can only originate from within a specific network. It's basically a crude MITM mitigation.
- TeMPOraL 6y ago
  > All the use cases you described are already implemented in today's cars, bus segregation doesn't mean zero communication. That's the problem, though. Once bidirectional communication exists, you can pretty much guarantee there's an exploitable hole in it that can be used to break the firewall.
  - DoingIsLearning 6y ago
    These systems are crude but they implement a physical airgap. You would have to hack a gateway (on-site) or physically access a segregated bus, at which point you could argue that you could also physically tamper with brakes or engine without any software being involved.
    - Silhouette 6y ago
      Unfortunately, a physical air gap won't prevent a malicious actor from, for example, projecting a misleading image designed to confuse your automated driving systems when they process that image and so prompt an adverse and potentially dangerous reaction. This is not only about the communications channels in the internal architecture, it's a much broader problem than that.
      - DoingIsLearning 6y ago
        Please read the whole thread, this is going completely off-topic from the original discussion in the start of these comments. The original claim was that vehicles should never be connected to any network because of unspecified online attacks that could actuate brakes or steering. I explained why this was unfounded. Adversarial patterns against computer vision based ADAS are only a real issue for system which are not sufficiently redundant . Autonomous systems in particular should also apply a degree of sensor fusion between multiple sources of data such as optical computer vision, radar, long range ultrasound and LIDAR (once it becomes cost effective). If any of those systems provides erroneous data the remaining ones can negate that and allow for a fail-safe behaviour. If you want my opinion, as someone who has moved away from automotive R&D a few years ago, Tesla's decision of depending too heavily in computer vision systems without addtional sensor redundancy seems like an architectural defect that has already cost lives. Either they are not integrating other sensor data sources or their voting weight appears underestimated.

marcosdumay 6y ago

I can think of at least one manufacturer that pushes motor performance and battery charging updates over the net.

deleted 6y ago
[deleted]
jayd16 6y ago
Teslas are still not drive by wire though.
- ajxs 6y ago
  The article we're discussing would suggest that the attack surface for any kind of complex machinery is very wide. A car doesn't need to be drive-by-wire to be potentially endangered by malicious code affecting the engine or battery.
- Veserv 6y ago
  At least the Model S is. It has remote summon and Autopilot which necessarily means that it has software controlled accelerator, steering, and brakes.
  - jayd16 6y ago
    That's not what "by wire" means though. My understanding is the steering wheel and brakes are mechanically attached to the steering and braking mechanisms.
    - bayindirh 6y ago
      "by wire" means non-physical ways in automotive. So accelerator and possibly brake pedals have only cable connections which carry data. Even if the brake pedal is connected via an actual wire, emergency brake actuators can work by themselves on the system. Electric assist motors on the steering column can also overpower a human with ease. So, actual connections doesn't matter.

Veserv 6y ago

What justification do you have for asserting that the safety critical systems are actually separated/isolated? The people at Jeep probably thought their systems were secure, yet the researchers on the Jeep Cherokee attack demonstrated they were not and claim that they could have simultaneously affected 471,000(!) vehicles [1]. Do you think anybody at a car company would dare to claim that their systems are safe enough to bet the lives of 471,000 people on them and take responsibility if they were wrong? Do you think any engineer would, in good conscience, support such a statement and share that responsibility if asked directly? I doubt you could find a single engineer that would feel even remotely comfortable that their procedures are good enough for 1/100th that number and you would never find an executive who would dare claim such a thing in a legally binding way. Absolutely zero benefit of the doubt should be given to systems where a single error could cause grievous harm to tens of thousands of lives. Such systems should require an absolutely ironclad public legally-binding positive assertion of security with strict criminal liability for failure (blame can not be shifted) that is independently validated before we should even think of accepting their use. We already accept nothing less for systems that are less dangerous such as nuclear reactor meltdowns and systems that are equally dangerous such as nuclear weapons systems. If systems with failure modes many times worse than the atomic bombings of Japan can not be made safe, then they must not be made. Anything less would be so criminally irresponsible that we lack a word for the magnitude of irresponsibility. [1] https://www.wired.com/2015/07/hackers-remotely-kill-jeep-highway/ https://www.wired.com/2015/07/hackers-remotely-kill-jeep-hig...

AlotOfReading 6y ago
I'd guess the fact that GM hired those guys for Cruise [1] probably suggests they took it seriously and made sure their own cars were isolated. [1] https://www.detroitnews.com/story/business/autos/mobility/2017/07/31/gm-hires-jeep-hackers-cruise-automation/104158212/ https://www.detroitnews.com/story/business/autos/mobility/20...
- Veserv 6y ago
  This is exactly the sort of justification that I am cautioning against. Would it be great if they did take the problem seriously and solve it? Of course. But it is not our duty to justify their decisions when the consequences of their decisions may cause grievous harm to thousands. It is their job, the company's, to justify to the satisfaction of engineers and the people that we should trust the lives of thousands to millions on their systems. If anything, it is our job to be skeptical of their claims and grill them ruthlessly before they should even be allowed to consider deploying such systems. I mean, think of any other software system, would you trust anybody's lives to any of them? Would the engineers on those projects agree? Most software engineers I know would be horrified if their systems were used in something responsible for even a single life let alone thousands. The claim that we should trust these companies with hundreds of thousands of lives is an extraordinary claim that they should be required to provide extraordinary evidence for. And, even if we are convinced, they should still be ultimately liable for their choice of decision to make sure they are actually putting their money where their mouth is. It is what we demand from bridge builders, it is what we demand from nuclear reactor designs, it should be what we demand from mass-produced internet-connected safety-critical devices.

Retric 6y ago

Tesla’s has over the air updates for it’s autopilot software, you even edit destinations from the infotainment system. So, it’s clearly both on the same network and could be hacked remotely. The car can even be turned on and summoned remotely. Which means if nothing else it could be remotely driven onto a freeway to cause accidents even if it’s ‘autopilot’ had some kind of independent backup safely system to avoid collisions.