6 ms·

Someday, all cars from a particular brand will be made to crash during rush hour. The carnage will be immense. Emergency services will have to go off-road to by

• jxurueydyys 6y ago
  An easy solution would be to not allow self driving or remotely updated cars until there's a reliable solution to this. People already go to auto-shops for repairs, certified auto-shops could easily double as places to update software and the certification requirements can be tailored to require an external oversight agent come and evaluate their security practices.
  • freeopinion 6y ago
    Certified autoshops are a solution in the same way that Norton is a solution for PC malware. Problem solved?
    • jxurueydyys 6y ago
      The difference is that the pc is exposed to everything it interacts with including the internet while the car would only be interacting with shops that were meeting their certification obligations. Tying updates to physical locations also reduces the severity of a successful bad actor since most people in a city don't all go to the same auto-shop. A problem like a nation-wide cyber attack on vehicles is only possible if we allow personal vehicles to support the attack vector.
      • throwaway0a5e 6y ago
        I don't see how certification solves anything that just not doing OTA updates doesn't also solve.
        • jxurueydyys 6y ago
          It reduces the attack surface to places that have been verified to be following security obligations. Otherwise a physical attack becomes much easier because you just need to work at an auto-shop that doesn't pay much attention to you. The same sort of attack is still possible with certs but it's more difficult because it would require compromising both the shop and the person who vets their practices.
      • est31 6y ago
        People still want regular map updates, live updating traffic information, and play back stuff from their phone on the in-car entertainment system. All this exposes cars to data communication outside of the car repair shop. Yes, the entertainment system is different from the system that runs the car, but there is some level of communication between the two.
        • tsimionescu 6y ago
          There is some level of communication, but there really shouldn't be.
          • Silhouette 6y ago
            The problem is that total separation is difficult to achieve with the requirements being placed on these vehicles. Sure, whatever is playing your favourite music tracks over the speakers probably doesn't need to know anything about steering and acceleration. However, your self-driving software is going to have a tough time getting your car to a location it doesn't know exists because its onboard navigation maps predate the existence of that building, or planning a route that avoids an accident it doesn't know about because it has no real-time information about road closures.
            • tsimionescu 6y ago
              As others have pointed out, self driving software is inherently unsafe at this moment - we don't know how to make safe network-connected software yet. Anyway, this is moot in most cars, as they don't have sold driving software anyway.
              • Silhouette 6y ago
                If you read my comments elsewhere in this discussion, you'll see that I've been one of those people. In this thread, I'm simply observing that there can't be a clear separation between control systems and information systems if we're going to achieve these kinds of autonomous functionality. We need a more nuanced solution.
  • slg 6y ago
    I definitely think Tesla can be over aggressive with their updates. However that doesn't mean that the basic idea of remote updating cars is inherently flawed or unsafe when compared with the alternative. It is all about trade-offs. Both the Prius[1] and the Model 3[2] had similar software bugs related to their anti-lock brakes. Both companies had a software fix a few days after the bugs were discovered. Tesla's fix was pushed out immediately to every vehicle. Toyota couldn't push out a fix. They had to issue a recall and have a technician update the software whenever that car ended up being serviced. How many months or more likely years did it take for every Prius to be updated with the software? How many miles were driven in cars that were known to have faulty brake software because it was hard to update them? You have to consider situations like this when discuss banning remote updating. [1] - https://www.networkworld.com/article/2245704/toyota-to-recall-prius-hybrids-over-abs-software.html https://www.networkworld.com/article/2245704/toyota-to-recal... [2] - https://www.wired.com/story/tesla-model3-braking-software-update-consumer-reports/ https://www.wired.com/story/tesla-model3-braking-software-up...
    • Aeolun 6y ago
      > You have to consider situations like this when discuss banning remote updating. Isn't that exactly what we're doing? If someone pushes out a malicious change, do you know how long it would take for that change to propagate to the entire Toyota fleet? It wouldn't.
    • user234683 6y ago
      > Toyota couldn't push out a fix. They had to issue a recall and have a technician update the software whenever that car ended up being serviced. This gives all the more incentive to get the software correct in the first place. The model of "get the software as bug-free as possible upfront using stringent processes, testing, formal methods, and not using software in the first place when it's not actually needed" is better than the model of "put software into as many components as possible to make it shiny and get the software good enough to release before our competitors and play whack-a-mole on the bugs later through updates". Instantaneous updates make it easier for an attacker to take control of the update infrastructure and push an update that will trigger a mass-crash of cars during rush hour. When people have to asynchronously take the car to dealerships over many months, it makes this attack harder to go undetected.
  • dmix 6y ago
    Infosec is a worse gov regulation than drug prohibition. It will do little than make naive people feel better.
    • r77ruuddj 6y ago
      I disagree. I think in an ideal world this is true but the reality is that the bar for security is exceptionally low for industries that aren't traditionally software industries (not that software is excluded). When the stakes are just accounts and transactions that can be reversed the situation is different, when the stakes are life and death, in the real way, I don't think it's unwise to exercise caution. Let someone else take the risk of cyber terrorism and if they go a decade without any hiccups then leapfrog them. There's no reason to expose consumers to these sorts of risks just because it gets the futurists hard.
• brippalcharrid 6y ago
  Whoever did this would likely select some combination of valuable/soft/flammable targets. Control over a sizable fraction of all vehicles in a country would enable them to create utter pandemonium in tunnels, bridges and underpasses during rush-hour - even larger highways. Aside from fire, I'd imagine that the "disable vehicle on sensing a crash" functionality would end up being hackable as well. Cars on the whole had become less effective as murder weapons up until now, but I suppose that all changes when you can control them remotely via software at scale.
  • o-__-o 6y ago
    >disable vehicle on sensing a crash Most vehicles won’t let you reprogram the firmware without power cycling the car. Disable sensing of a crash is definitely its own ECM that is on a high priority bus. I am assuming your common <$40k car. When you head into bmw, merc Benz land this statement changes slightly.
    • brippalcharrid 6y ago
      I'd definitely like to think that all of that stuff would be air-gapped, hard-wired and baked into the silicon (and E2E-encrypted, with a Trusted Computing model and auditable supply-chains), but I do worry that people are going to cut corners, fudge things when they're approaching deadlines, and not take into account an appropriate threat model when they're designing this mass-market consumer automotive stuff. The Chrysler hack in 2015 managed to get some fairly low-level remote access to things like the braking system. I'm also considering the possibility that governments might backdoor their own manufacturers with their knowledge in order to gain exploits to systems overseas or to carry out the odd covert assassination.
• doopy-loopy2 6y ago
  lol wtf
• Barrin92 6y ago
  >Someday, all cars from a particular brand will be made to crash during rush hour. This is also why it's always very, very wrong to compare potential faults of automated cars to humans as in "the automated car is X percent safer!", becuase it ignores the fact that mistakes in automated systems, at least as they are built now, are highly correlated. If there is one bug in an ML system that is rolled out to an entire fleet that results in an unknown weather condition leading to fatal crashes you may create mass carnage. Human driver errors are not correlated like this, which makes them much more robust as an ecosystem.
  • feteru 6y ago
    I think this is a good point, and largely agree. However, there are also correlations in driver behavior, like it being more dangerous driving on July 4 or New Year's Eve in America as so many people celebrate and drive drunk, increasing accident rates. According to NHSTA, 144 drunk-driving deaths on July 4, compared to 36 on average: https://www.bactrack.com/blogs/expert-center/35042821-the-most-dangerous-times-on-the-road https://www.bactrack.com/blogs/expert-center/35042821-the-mo... I would say that's mass carnage.
  • manicdee 6y ago
    I can tell it has started raining by the number of ambulances leaving the station down the road. Human driver errors not correlated my arse!
• tyingq 6y ago
  The thing that worries me is that it doesn't take self-driving tech to make this an issue. Existing safety systems on most cars can control brakes, steering, throttle, airbags, etc. The threshold issue is remote updates of car software. And Tesla had made that more mainstream and attractive to other manufacturers.
• alynn 6y ago
  As someone who’d already been a bit worried about future mass-car hacks, I found the zombie car hacking sequence in 2017's "The Fate of the Furious" particularly terrifying to see in the theater. Rewatching it now, it actually looks somewhat tame compared to what might be since the hacked cars only inflict property damage, not injury. > Villainess: I want every with chip with a 0-day exploit in a two mile radius around that motorcade now. > Computer guy: There's over a thousand of 'em > Villainess: Hack ‘em all. It’s zombie time. > [zombie cars drive around causing mayhem] https://www.youtube.com/watch?v=TQmMnRQu9YQ https://www.youtube.com/watch?v=TQmMnRQu9YQ
• wrkronmiller 6y ago
  Care to elaborate on why you believe this?
  • burfog 6y ago
    It's like being in 1995 and predicting that Windows botnets will be created. The coming disaster is inevitable. State-sponsored hackers are not going to ignore the opportunity. They probably have the capability already, in dozens of countries, and are just waiting for orders from the leaders. If war is starting, the order will be given. Sanctions could be enough to trigger it.
    • thoughtstheseus 6y ago
      I thought it was well known that infrastructure has been a target of state hacking?
      • godelski 6y ago
        I think this is well known to tech literate people. Problem is that that's not most people. On HN we have sampling bias because we're more likely to be associated with people with similar interests, i.e. tech.
    • deleted 6y ago
      [deleted]
    • gfxgirl 6y ago
      So you predicted botnets in 1995 and ..... nothing much happened. botnets suck but there wasn't some world crashing event like the person above is predicting for computer controlled cars. All Windows computers didn't shut off on one day.
      • actuator 6y ago
        I would disagree, sure botnets can't be used for threatening life directly; but botnets have been proven to be quite effective in attacking services and do denial of services attacks. The one thing we can take from botnets is, vulnerable and unpatched devices can be infiltrated in high numbers and attackers can lie low until they decide to pull the trigger. Imagine even 0.1% cars being controlled, the mayhem and loss of life that they can cause is just immense. Power plants are also dangerous targets in a similar sense, but hopefully there is network separation for control services.
• abledon 6y ago
  the guys who make power drills are writing the software for Self-Driving Cars[1]? Who is running that ship lol [1] https://www.bosch.com/stories/future-vehicles/ https://www.bosch.com/stories/future-vehicles/
  • sukilot 6y ago
    Huge companies have more than one product.
  • throwaway0a5e 6y ago
    Bosch has been doing electronic vehicle control systems since the first electric wiper motor. It is one of their core businesses.
  • BoorishBears 6y ago
    The guys who have been making safety critical automotive electronics since it's infancy (https://en.m.wikipedia.org/wiki/Electronic_stability_control https://en.m.wikipedia.org/wiki/Electronic_stability_control ) are getting questioned on core competency - Meanwhile there's the company who took Mobileye's LKA, which was already in cars with the same hardware but not continuously enabled because of inherent flaws related to non-moving objects... and turned it on all the time so they could call it Autopilot. Then it killed some people because of the flaws that kept other manufacturers from using it the way they were. And they're writing the software for self-driving cars? Who's running that ship lol - In a better world people with experience with safety critical stuff and the culture for it would partner with companies like Tesla to create something like a "plug in" system for SDCs. Where the safety guys could focus on defining a minimal viable envelope of operation the same way existing ABS and ESC systems mesh with drivers. Something like if the car is less than 100ms from crashing intervene separately of "normal" object avoidance (And before someone nitpicks that's an arbitrary number of ms, yes there's no "isCrashing" variable, and yes it would be hard work to define how SDCs would handle intervention, but people crashing into parked firetrucks is worthwhile "hard thing" to solve)
  • Jtsummers 6y ago
    A “fire extinguisher” company makes many of the fire/smoke/overheat detection and suppression systems used in commercial and military aircraft. These kinds of companies are massive and have a lot more depth to them than you seem to realize.