16 ms·

Self-Crashing Cars

248 points by devy 9y ago

emveeoh 9y ago
Funniest headline of the day award...
kenning 9y ago
> This article is geared towards people with a STEM background. For something shorter try this article in The Weekly Standard. From personal experience I can tell you that people with STEM backgrounds can also be too impatient to read an article this long. edit: > (Bio) After years of building startups and advising organizations large and small on data science and cyber security, I'm turning my focus to improving public literacy on technology. Starting your article like this is a bad way to do it.
- dcow 9y ago
  You may have a point or two, but you're being downvoted because of the tactless way you've stated them. Also since when is a longer read only for "STEM people"? edit: Fooled me, looks like it's been corrected.
  - deleted 9y ago
    [deleted]
  - kenning 9y ago
    Yeah its the first line in the article I imagine (some of) the downvotes are from people with a STEM background that didn't even open the link
    - brokenmachine 9y ago
      Thus proving he was correct, lol!
csours 9y ago
> "Every single electrical and radiological device can be utilized to transmit data. What's the baud of opening and closing HVAC vents and reading them from space satellites?" https://www.wired.com/2017/02/malware-sends-stolen-data-drone-just-pcs-blinking-led/ https://www.wired.com/2017/02/malware-sends-stolen-data-dron... Also reminds me of the stories of reverse engineers using, lights, motors, speakers, etc to exfiltrate boot images. > "If hacking a Jeep is as straightforward as hacking a server, and servers are routinely breached, then where are all the hacked cars? It's a bit like the Fermi Paradox." One barrier to entry is the actual cost of acquiring a vehicle. You can buy a lot of iPhones to hack on for the price of a Tesla. > "We think of Teslas as cars just like we think of an iPhone as a phone, but a more accurate account of reality is that they're both just computers." It's actually kind of worse than that, cars are actually rolling datacenters with multiple computers and multiple networks. (CAN bus[es], Ethernet, proprietary networks) //Disclaimer: I work for GM, but not on any of this.
- hinkley 9y ago
  I know things change over time, but the old advice on hackers was that most of them are not as malicious as they seem. For instance, many viruses have been accidentally more damaging than the creator intended. Hacking a car to kill people is a lot different than kicking someone out of a video game. Even people who SWAT don't anticipate their rival/victim being killed. They want the person to be miserable, not dead. It's cruel and vindictive, but it's not homicidal. Or at least not most of the time.
  - c3534l 9y ago
    The world isn't short of homicidal maniacs. All that is needed is a perverse ideological motivation and a single person could, theoretically, cause unprecedented harm.
    - crpatino 9y ago
      The thing is when you have two or more pretty unlikely traits as required components of X, X's likelyhood becomes negligible. That's why the weapons of choice for homicidal maniacs are everyday tools like kitchen knives or baseball clubs. Even the use of handguns sugest the involvement of a more purpose-driven kind of homicidal. So, IMHO, what you need to fear is not a hacker going homicidal. Rather, it is the raise of a black market of "assasination thru tech"; specially if there's enough separation of labour so that unethical but otherwise non-violent hackers can enable the efforts of conventional assassins in order to accomplish their grim deeds.
deleted 9y ago
[deleted]
anonu 9y ago
tldr: Cyber attack is 1000x easier than defense. There are enough Tesla cars on the road today that can be hacked into and turned into WMDs. Lots of ideas added in on how to bolster defense.
- lucb1e 9y ago
  Weapons of Mass Destruction, that is^ I knew what you meant only because I had read the article.
  - ravitation 9y ago
    "WMDs" has been a common acronym in the American lexicon since 2003 and the invasion of Iraq. (This is just a cultural aside. I know there is a world outside of the United States.)
- Zigurd 9y ago
  Car bombs are bad, but they are not "WMDs." Nevermind that it is cheaper to find a suicidal driver, or trick a driver into going on a suicide mission than it is to buy a Tesla and hack it into a car bomb. Many imagined terrorist threats are threats only in a vacuum. A rifle and a tall building resulted in 50+ deaths and 500+ wounded. Any technologically complex attack has to beat those numbers. The only notable attack with an actual WMD was the sarin gas attack on the Tokyo subway. It killed about one fourth the number of people as a man with a rifle in Las Vegas. It took a secretive cult organization to implement. That's why nobody uses WMDs for terror attacks. Trucks and rifles do enough damage without even having to acquire bomb-making skills. Autonomous vehicles that are programmed to avoid hitting pedestrians are likely to make it harder to use vehicles as a weapon, not more deadly.
  - wehadfun 9y ago
    buy a Tesla and hack it into a car bomb You did not read the article
  - aidenn0 9y ago
    Fast forward 10 years; the features in the Model S have trickled down to whatever car is 2027's Toyota Camry. You figure out how to hack it over the internet and load a program that uses its cameras to identify a pedestrian and drive straight towards it at full speed. You turn on this program at some busy time of day. There are times of days in which there are tens of thousands of late model Camry's on the road in the US, and this is the very first example I could think of when considering the problem raised in the article.
Animats 9y ago
Real problem, clueless author. The big problem is allowing firmware updates. Many years ago, I knew some of the people involved with the Ford Electronic Engine Control IV, the "EEC IV", used in 1980s Fords. This used a custom CPU, an Intel 8061, which was an Intel 8051 with some extra timer features. The program was etched onto the CPU silicon. There is no way to change that program without replacing the whole unit. There's an external ROM with some tables, different for each engine model, but it's a true ROM, not something that can be rewritten. If you want to replace it, it takes a wrench. The level of paranoia and testing that went into that program was very high. Any bug meant bringing back hundreds of thousands of Fords in a recall. That didn't happen, and there are still many EEC IV vehicles running today, 30 years later. That's what it was like in the days of hardcore embedded programming. I'd argue that safety critical vehicle software should not be downloadable. If a fix is needed, the vehicle has to go back to the dealer for a new memory module. This would discourage "shit early, shit often" software development, and encourage manufacturers to keep the safety critical systems, like ABS, totally disconnected from the entertainment system. Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.)
- ohazi 9y ago
  My car is >10 years old, and thankfully still running like a champ. It'd be nice to get a new one someday, if only for the updated safety features like curtain airbags, but I honestly don't feel very comfortable with anything much more recent, for the exact reasons you've described.
- PinguTS 9y ago
  > Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.) You seem to be out of touch what happens today. Car2Car or Car2Infrastructure is about as aircraft to aircraft communication is used in TCAS. It is about what car (object) is at which position, so that collision may be detected as early as possible to prevent them. Also next generation cars will have their maintenance modes locked down as far as possible. OBDII will be closed down as far as possible. All those dongles, which are hip right now, will be rendered useless. But of course the tuning scene has still a lot of incentives like in the past to thwart all this and to reverse engineer. It will be a cat-and-rat game like it always was.
stcredzero 9y ago
Since organizations do not observe an attacker's failure, the market generally does not reward extreme competence in cyber defense... Blackhats, in stark contrast, are sexy. Bounty hunters are sexy. Why doesn't the government start paying bounties? Paying bounty hunters to run honeypots would convert many black hats into hunters of black hats. This could well create an ecosystem where the more knowledgeable hackers directly prey on the script kiddies for fun and profit. Taking useful idiots out of the ecosystem strikes me as desirable. Such a program would also be useful for recruitment. In a way, this is analogous to such bounties in the transition of the wild west into a more normal society.
- im3w1l 9y ago
  This would incentivize black hats to frame innocents. And black hats strike me as the type of people that would be able and willing to run with it.
  - stcredzero 9y ago
    This would incentivize black hats to frame innocents. I thought of that too. This also happened with bounty hunters. There will need to be safeguards.
- lucb1e 9y ago
  If you have a rat problem and start paying for rat corpses, people might start breeding rats.
  - stcredzero 9y ago
    The analogy falls down a bit, because while rats are born rats, a hacker is first a person. People can witness people becoming examples, and decide not to do that. That said, problems can arise, due to national borders, limited jurisdictions, and differences in access to socioeconomic resources.
- roywiggins 9y ago
  The last time we paid bounties, we ended up with a bunch of people in Guantanamo who had no reason to be there.
bo1024 9y ago
I think this is a really important issue and glad someone is working on awareness. It extends far beyond cars but those are probably the most 'frightening' example. We need to (1) Admit we have a big problem. (2) Admit it will be difficult to solve and there aren't easy solutions. (3) Start working on solutions. Unfortunately, these steps never seem politically popular. But it will be tough. Except in very rare scenarios (NASA), software security and correctness aren't treated as important in the scheme of things, relative to functionality and features. Nobody wants to invest in actually secure, carefully-audited code. (They might claim they do, but their standards for "secure" and "careful" would be litigation-worthy in any field but software.) For example, I believe in freedom of software and ownership rights, which seems to mean that I believe people should be allowed to run open-source code on their own cars' computers. But this impacts public safety. How do we develop reasonable regulations similar to those for physical "street-lega" modifications?
- sparkie 9y ago
  I have reason to believe that a competent hobbyist programmer is less dangerous than the electronic engineer, pretend-to-be-programmers who wrote the original firmware on the vehicle. Based on years of working with electronic engineers' code. (Apologies to those of you who are competent in both)
  - bo1024 9y ago
    I appreciate your point and am inclined to agree. But my agreement doesn't matter. Your point is only one starting point for a challenging and nuanced discussion over the legality and safety of people installing their own software on their own cars. My broader point is that this (among others) is a discussion we should already be having and eventually must have -- if not amongst "ourselves", then with legislators and laypeople. Yet the state of software security, awareness, and incentives is such that it could be years or a decade until we do...
dfabulich 9y ago
> One of the problems I've had over the past year and a half is how to communicate this idea without: > 1. Sounding like a crank. > 2. Giving ideas to terrorists or hostile foreign governments. #1 is this article's biggest problem. These problems are real, but I think the author sounds like a crank. (Based on the way this article is written, I suspect that the author actually is a crank--obsessed with security, but fundamentally lacking in the relevant skills to do anything about it.) If the author is serious about this, here's what you do. 1. Become/join a non-profit organization. You want to be frequently quoted in the press, but not as "well-known software security expert Zach Aysan" but as "Zach Aysan, president of the Organization for Global Security." 2. Build your reputation by finding and earning credit for security problems. Do ethical reporting, but when the issues get fixed, exhibit them in a flashy way. 3. White papers, not blog posts. This "article" starts with a subtitle "with apologies to Elon Musk", followed by a personal dedication to Zach's father. This is not the tone of a white paper from a think tank. The subtitle of the post should be the thesis statement of the article. "Self-driving cars lack adequate security protections." The first paragraph should be an executive summary of the argument of the post. Each paragraph should support the thesis statement. Have someone read your articles; update your work based on their feedback. Thank them in the footer. 4. Separate "how to" articles from arguments. This article is long because it is both attempting to persuade the reader that we haven't invested enough effort into securing critical systems and also to give a list of proposals. These should be separate articles, with one article arguing why security is important, and another article giving a list of proposals.
- lucb1e 9y ago
  > #1 is this article's biggest problem I don't know, but I had the same idea years ago, and I later read about an author using the idea in a 2006 book (Daemon, by Daniel Suarez). I'd say this shouldn't be an issue, and it also solves number two: if we can think of it, so can hostile governments* and mad men. That said, the article does (after he gets to the point of "weapons of mass destruction") seem to get a little obsessive and, yeah, crank-y. But that has little to do with the concern itself. * Not "hostile foreign governments" because every government is foreign (and perhaps even hostile) to someone.
mtgx 9y ago
> They rebutted that individual cars are much easier to hack and after they are first used in a terror attack we will get the political will to fix the problem. Was that meant to be a joke? Sure, they can establish a new standard that will apply to all the new car models coming out four years later. But who actually expects hundreds of millions of cars that are already on the market, to receive a software overall with a new architecture? This is why it has always bothered me that almost no one seems to bring this problem to the forefront - certainly not carmakers. They're all too focused on how awesome self-driving technology will be and how it will save us from drunk drivers. Thus, disregarding the fact that once we have 100 million to 2 billion self-driving cars on the road, that will be a huge market for cyber criminals, from ransomware and cryptojacking (hello powerful GPU computer + free solar power charging!) to assassinations. And before anyone says "how much harder it is to hack a car than a PC", consider the fact that most cars today aren't actually connected to the internet. And most of those that are, only have their entertainment systems connected to the internet. Self-driving cars will be able to receive OTA updates that will improve their engine, steering, and brake performance = the OTA software has access to everything. Combine this level of access to the high level of recklessness in the name of profits carmakers seem to be showing today, when they advertise features such as "unlocking your doors through an app". EFF's former chair and someone who worked on Google's Waymo, has some decent ideas about how to protect self-driving cars, if only carmakers would listen: http://ideas.4brad.com/disconnected-car-right-security-plan-robocars http://ideas.4brad.com/disconnected-car-right-security-plan-...
- 3pt14159 9y ago
  Airgapping was my first instinct too, but the problem is we're dealing with state-level actors. Airgapping doesn't work with them. They're patient, well funded organizations. Trying to rely on never having a single type of car (any of which could have a hundred thousand copies on the road) hacked is a fools errand. We need ways of disabling autonomous devices and detecting when they get hacked, not trying to win an impossibly hard game.
  - bradtemp 9y ago
    The disabling system becomes another attack surface, and unless it is pretty independent, is itself disabled by a sophisticated attack. But scared as we are of external attack, allowing the government to shut off all cars is like letting Mubarak shut off the internet in Egypt. That's a bigger danger than foreign enemies in many countries.
icefox 9y ago
> It's a bit like the Fermi Paradox. During the late 90's and early 00's I often wondered why with all of the Microsoft hate why someone didn't create a simple virus that did something destructive like just format windows hard drives. There must have been other incentives at play that caused this to never occur. Perhaps in the same logic if you can infect every Tesla it is more valuable to not crash them, but instead scrub the data and sell that back to [insert company] or something. If there was ever something that would cause a formal programming guild to sprout I would be willing to bet that it would form its roots around security.
- sp332 9y ago
  There were plenty of destructive viruses. Some (e.g. CIH) would erase the MBR of the disk making it unbootable. Others like ILOVEYOU would overwrite user's data directly. Blaster specifically had a message about Windows' poor security and tried to DDoS Windows Update. Sometimes viruses are written to by self-limiting. MyDoom, which caused about 10% of all email traffic for a time, was programmed to deactivate on a certain date. Also once viruses get to a certain level of infamy they get a lot of attention. Blaster was mitigated in just days, so by the time it started its DDoS it was already mostly wiped out.
  - marcosdumay 9y ago
    Even more relevant, Internet access wasn't common back then, and the only large-range virus vector was the slow paced sneaker-net.
- Piskvorrr 9y ago
  From the other virology: a virus that outright kills its host doesn't get very far. A virus that keeps the host limping along, creating copies of the virus in the process, is far more likely to spread...which is, IIRC, precisely what happened in the early 2000s. (And indeed, what would be the incentive for disabling Windows hosts? "W1nd0z3 suxx0rz" would barely count as one, given the lack of general-public alternatives at the time - the afflicted would pay a tech grunt to repave with the same Windows again, been there, done the repaving.)
deleted 9y ago
[deleted]
LinuxBender 9y ago
Unpopular opinion ahead. Having worked in the security industry for a few thousand years (computer years), I can say I would never own a car that can talk to the internet. I plan to move far away from cities very soon for this and several other reasons. People will argue about this and meanwhile the "impossible" will happen, repeatedly. I just replaced the engine and transmission in my non internet vehicle and hope to get another 500k miles.
- joshl3253 9y ago
  Yes, a self-driving car would probably be hacked in the future, plowing into pedestrians or drive off a cliff. However, what's the odds of that, compared to thousands of deaths caused by incompetent human? "In 2013, 3,154 people were killed in distraction-related crashes." [Source just google it].
- deeg 9y ago
  I understand your concern but right now 30k Americans die in car accidents. Will (potentially) hackable self-driving cars be any more dangerous than today? If you're that concerned about being involved in a car crash it seems to me that you should never leave the house.
  - ori_b 9y ago
    > I understand your concern but right now 30k Americans die in car accidents. And a few thousdand died on September 11th, nearly 20 years ago. The collective fear from that made the world a far worse place than the actual death toll. If there's a terrorist attack that targets internet connected cars, what do you think the collective reaction might be?
    - beat 9y ago
      These are Americans we're talking about. The collective reaction would be to run over people in the street if they wear glasses and nerdy t-shirts.
      - 9y ago
vlucas 9y ago
Easy hack to prevent remote control/hijacking of your car: drive a standard. It will never be able to start or drive anywhere remotely.
- eric_h 9y ago
  Excellent idea. Too bad it's so much harder to get a standard transmission car in the US these days.
  - barsonme 9y ago
    I'd be curious to see the manual vs automatic accident rates per miles driven. I have a feeling automatic are a lot higher. I mean, these days, if somebody's driving a manual either they want to or had no other choice. I wonder if that correlates with driving ability at all.
    - Wehrdo 9y ago
      That would be interesting to look into of one had the data. I would bet insurance companies have investigated it. The only evidence I could find was this study: http://journals.sagepub.com/doi/abs/10.1177/1087054706288103 http://journals.sagepub.com/doi/abs/10.1177/1087054706288103 Although I can't access the article, the abstract says that teens with ADHD had higher attention to driving when using a manual transmission vs an automatic.
      - barsonme 9y ago
        That's funny you mention ADHD. I was diagnosed as a teenager and I've always felt more focused while driving a manual than an automatic—it's not as easy to zone out. I've also heard of doctors "prescribing" manuals for people with ADHD. That's just second-hand, though.
antirez 9y ago
Still cars are already computers but so far nothing like that is happening... In theory the fact of being autonomous should be a protection because you could add some control system that can't be updated and can make choices reagreless of what the other systems are telling to do.
skywhopper 9y ago
I don't think the author is wrong about the threat. Unfortunately, his apparent solution--a government mandated standardized "safety module" to intermediate the Internet-connected bits of the vehicle from the non-connected bits--is likely even more risky. Monocultures can be devastated by discovering a single flaw. And there are guaranteed to be many flaws in any non-trivial computer system. Imagine if a straightforward exploit for Spectre that totally compromised a system leaving a hard-to-detect back door via some simple Javascript had existed on January 1, and had been injected into a few popular websites via ad networks... for that matter, how do we know such a thing didn't happen?
olivermarks 9y ago
A witness to Michael Hasting's 2013 'car crash' describes what he saw https://youtu.be/fweyFCFKcp0 https://youtu.be/fweyFCFKcp0
- olivermarks 9y ago
  DARPA discussion https://youtu.be/6OfcgJ-pl7Q https://youtu.be/6OfcgJ-pl7Q
- gowld 9y ago
  "car crash" is the only non-debated aspect of the situation. No scare quotes needed. (The crash wasn't caused by remote control hacking -- that's an absurdly complicated and high-profile way for well-funded professional murderers to murder a single relatively minor person. If we was murdered, it was by some kind of mechanical sabotage or by impairing his mental state with some drug.)
bufferoverflow 9y ago
Cars are different though, they aren't general-purpose computers (even if they are based on them). So you can have a chip that checks the checksum of every file, checks if it's all signed with the correct certificate, and shuts down the car even if one bit is off. That's quite different from your laptop or a smartphone, which can run pretty much any code. It would be very very very hard to hack something like that. As in, steal the signing keys from Tesla hard without being noticed. I wouldn't be surprised if their signing server is air-gapped.
- deleted 9y ago
  [deleted]
- beat 9y ago
  Cars are also different in that software updates are harder to pull off. There are millions of cars out there on the road right now that have internet connections, and only minimal security between human-convenience internet like map software, and the embedded systems that do things like steering and brakes. And the average lifespan of a car is what, 15 years? Whatever the vulnerabilities, they're going to be there for a long time. And even if there's a firmware update mechanism available, the manufacturers' abilities to maintain older software will also degrade, like legacy systems do everywhere.
- rcxdude 9y ago
  Car software is designed with safety in mind. While there is some overlap in techniques with security, they are not the same goal. It is possible to be (and I think most cars are) very safe, but not very secure.
carapace 9y ago
FWIW, Open Kernel Labs was acquired by General Dynamics in September 2012.
deleted 9y ago
[deleted]
vermilingua 9y ago
> What I learned was that there were not only no regulations there were no plans for them either. While I think my MP took my concerns seriously and did what she could, I came to understand that political will lags public outcry This interested me more than the idea of car-bombs, are we moving towards a post-law society? Thinking about it, I haven't seen a single piece of legislation regarding self driving cars, cryptocurrencies, or shared computing in my country; and are transport, currency, and communication not the underpinnings of civilisation?
0xCMP 9y ago
I looked and noticed this post has been submitted several times (not a complaint): https://news.ycombinator.com/from?site=zachaysan.com https://news.ycombinator.com/from?site=zachaysan.com Then I noticed he'd recently published an article on Zero Width characters being used for fingerprinting which got some attention on HN in the last 30 days: https://news.ycombinator.com/item?id=16046329 https://news.ycombinator.com/item?id=16046329
woolvalley 9y ago
Defense is harder than attack, but the attackers can be attacked back. Jail isn't worth a few million dollars vs. working in silicon valley. I think that is also a big reason why many potential black hats do not become black hats. The only 'sustainable' black hats are ones associated with a criminal organization or nation states.
whiddershins 9y ago
Am I the only one who thinks it is ironic that Elon Musk gives so much money to research regarding the existential threat of AI to humanity, while also basing a business on putting AI in devices that can so easily kill humans?
- daveguy 9y ago
  Pretty sure he has a guilty conscience. He wants to build an autonomous self driving car, which will have to get pretty close to general AI and he wants to put it in multi-ton robots. He knows this could be dangerous and scary. At least he is giving money to research the problem rather than just forging ahead with the potentially dangerous part. Personally I think the robot uprising concern is at least 50 years premature. But the "what if someone could take the controls" concern is a concern we should have been already been considering yesterday.
  - Veedrac 9y ago
    The fact you said "50" and not "5000" is enough reason to not consider it premature.
schlowmo 9y ago
While I agree with the author about the undervalued risks of autonomous vehicles in the light of careless security practices his proposed solution is just insane. In my opinion the demand for a government controlled kill switch in every piece of hardware that is somehow able to harm people is much more threatening in so many ways than the insecurity the author is trying to reduce. Just a few: 1. Based on the asumption, that ones current government/state is for the good of all people, what gives him the confidence that this will stay that way? What if your beloved government goes rogue? That's a lot power for an autocratic regime. 2. Why even trust the government in the first place with that kill switch? They are the same people which are careless about infrastructure critical ITSec since decades. 3. An univervsal security module which is highly standardized is a very profitable target. While the author is aware that finding an attack vector of one particular vehicle can mean that all vehicles of that type can be compromised, he doesn't come to the conclusion that the same logic applies to his security module.
- 3pt14159 9y ago
  I’m the author. I was nodding in agreement with you for your first two points. I don’t think I’ve come to a perfect solution, but I want you to know that I actually have many of the same reservations that you have and that I’m open to changing my mind. Here is where I still sit, however: the government had predator drones and will soon have killbots able to take out individuals based on facial recognition software. If we can’t trust them to turn off our cars we’re fucked anyway.
  - nickodell 9y ago
    These safety modules sound really complicated. Let's say a security vulnerability has been discovered. An attacker has wormed their way into as many cars as they can. They send a 'go' signal from their command and control servers. Across the world, cars start looking for opportunities to kill their passengers and bystanders. We send the shutdown signal. The safety modules wake up and take over from the compromised computers. What do they do? They could brake. However, the car might be in a turn, on a rainy day. Braking could send the car into a skid and kill its passengers. Maybe they brake, but slowly. However, the car might be behind someone who just suddenly changed lanes and slammed on the brakes. Braking hard might be the right move in that circumstance. Maybe they do something conditional on the sensor input they get. However, if the control computer can do something to 'blind' the safety computer, that doesn't help. For example, can the control computer issue firmware updates to the camera sensors? Can the control computer fill a sensor's bus line until it can't respond? I can't think of any solution to this that doesn't involve duplicating a substantial part of the control computer.
deevolution 9y ago
The human mind is also hackable... it just takes a considerably longer time to hack it compared to installing software on a computer. Id argue that the internet and social media has increased the speed at which the mind can be hacked by a malicious actor. Im not so sure which is worse now, autonomous vehicles or human drivers? I think its much harder to detect when a human has been "hacked" than a fleet of cars. Cars could have subsystems build in for detecting anomolies, axuilary computers and manual overdrive settings. Detecting when a human has been "hacked" requires invasion of privacy, constant surveillance, reliable friends and family, and a whole bunch of other variables. Preventing a human from being hacked might require arduous and expensive changes and experiments in legal systems, incentive mechanism, censorship, education,etc...
- deevolution 9y ago
  There's going to be casualties no matter what... that seems to be the nature of technological advancement.
Klasiaster 9y ago
First step would be to stop public funding of 0day purchase and development in NSA and co. People believe that more 'cyberweapons' would be good against cyberthreats from 'others' but meanwhile all use the same systems. If these systems do not get fixed because the NSA wants to use its purchased 0days then everybody has a problem. What is needed is joint effort for secure open source solutions. Also, for computer security programmers need to overcome convenience practices and start behaving responsible, but also people in management/politics should not make technical decisions if they can't understand the implications.
matthewowen 9y ago
I know this is somewhat off topic, but what's the deal with "Look no further than the Clinton email breach to see how much a single hacker can change the world."? To my knowledge, there's no evidence that Clinton's email was breached. The DNC, sure, but the Clinton email controversy wasn't based on any actual known breach. It's sort of alarming to see this weird retconning of history.
bdamm 9y ago
Some solutions are strangely misguided. UDP is insecure but TCP is somehow magically secure? Certificates are not to be trusted? Can't agree with all the conclusions but the specific paranoia is well founded.
King-Aaron 9y ago
With the continued reporting of these possible attack vectors, it makes me feel that there's a lot to be said about having a carburettor and points running your car, instead of a computer with network connectivity.
hueving 9y ago
>because obscurity is a valid defensive measure. It's how passwords and authorization tokens work. This is extremely dangerous thinking. Passwords and tokens are not in the 'security by obscurity' category because you can observe how the entire system works, review its source code, read its deployment configuration, and do packet captures of the encrypted valid traffic and still not have a way to gain access to the system. Security by obscurity refers to hiding how the system actually works, and that's a lot harder to keep a secret because you are one compromised device away from revealing all of that and it can't be easily changed. Do not call passwords and tokens "security by obscurity" or claim security by obscurity practices (e.g. Running on non-standard port numbers) is on the same level as passwords/tokens/keys. There is a reason the strongest crypto is using public algorithms and only private keys. Obscuring a system just means that your silly bugs don't get found and exposed early.
- Piskvorrr 9y ago
  Not a very strong measure, definitely not against a determined attacker, but valid nevertheless. A mosquito net for opsec, so to speak.

248 points by devy 9y ago

Funniest headline of the day award...

> This article is geared towards people with a STEM background. For something shorter try this article in The Weekly Standard. From personal experience I can tell you that people with STEM backgrounds can also be too impatient to read an article this long. edit: > (Bio) After years of building startups and advising organizations large and small on data science and cyber security, I'm turning my focus to improving public literacy on technology. Starting your article like this is a bad way to do it.

dcow 9y ago
You may have a point or two, but you're being downvoted because of the tactless way you've stated them. Also since when is a longer read only for "STEM people"? edit: Fooled me, looks like it's been corrected.
- deleted 9y ago
  [deleted]
- kenning 9y ago
  Yeah its the first line in the article I imagine (some of) the downvotes are from people with a STEM background that didn't even open the link
  - brokenmachine 9y ago
    Thus proving he was correct, lol!

csours 9y ago

> "Every single electrical and radiological device can be utilized to transmit data. What's the baud of opening and closing HVAC vents and reading them from space satellites?" https://www.wired.com/2017/02/malware-sends-stolen-data-drone-just-pcs-blinking-led/ https://www.wired.com/2017/02/malware-sends-stolen-data-dron... Also reminds me of the stories of reverse engineers using, lights, motors, speakers, etc to exfiltrate boot images. > "If hacking a Jeep is as straightforward as hacking a server, and servers are routinely breached, then where are all the hacked cars? It's a bit like the Fermi Paradox." One barrier to entry is the actual cost of acquiring a vehicle. You can buy a lot of iPhones to hack on for the price of a Tesla. > "We think of Teslas as cars just like we think of an iPhone as a phone, but a more accurate account of reality is that they're both just computers." It's actually kind of worse than that, cars are actually rolling datacenters with multiple computers and multiple networks. (CAN bus[es], Ethernet, proprietary networks) //Disclaimer: I work for GM, but not on any of this.

hinkley 9y ago
I know things change over time, but the old advice on hackers was that most of them are not as malicious as they seem. For instance, many viruses have been accidentally more damaging than the creator intended. Hacking a car to kill people is a lot different than kicking someone out of a video game. Even people who SWAT don't anticipate their rival/victim being killed. They want the person to be miserable, not dead. It's cruel and vindictive, but it's not homicidal. Or at least not most of the time.
- c3534l 9y ago
  The world isn't short of homicidal maniacs. All that is needed is a perverse ideological motivation and a single person could, theoretically, cause unprecedented harm.
  - crpatino 9y ago
    The thing is when you have two or more pretty unlikely traits as required components of X, X's likelyhood becomes negligible. That's why the weapons of choice for homicidal maniacs are everyday tools like kitchen knives or baseball clubs. Even the use of handguns sugest the involvement of a more purpose-driven kind of homicidal. So, IMHO, what you need to fear is not a hacker going homicidal. Rather, it is the raise of a black market of "assasination thru tech"; specially if there's enough separation of labour so that unethical but otherwise non-violent hackers can enable the efforts of conventional assassins in order to accomplish their grim deeds.

deleted 9y ago

[deleted]

anonu 9y ago

tldr: Cyber attack is 1000x easier than defense. There are enough Tesla cars on the road today that can be hacked into and turned into WMDs. Lots of ideas added in on how to bolster defense.

lucb1e 9y ago
Weapons of Mass Destruction, that is^ I knew what you meant only because I had read the article.
- ravitation 9y ago
  "WMDs" has been a common acronym in the American lexicon since 2003 and the invasion of Iraq. (This is just a cultural aside. I know there is a world outside of the United States.)
Zigurd 9y ago
Car bombs are bad, but they are not "WMDs." Nevermind that it is cheaper to find a suicidal driver, or trick a driver into going on a suicide mission than it is to buy a Tesla and hack it into a car bomb. Many imagined terrorist threats are threats only in a vacuum. A rifle and a tall building resulted in 50+ deaths and 500+ wounded. Any technologically complex attack has to beat those numbers. The only notable attack with an actual WMD was the sarin gas attack on the Tokyo subway. It killed about one fourth the number of people as a man with a rifle in Las Vegas. It took a secretive cult organization to implement. That's why nobody uses WMDs for terror attacks. Trucks and rifles do enough damage without even having to acquire bomb-making skills. Autonomous vehicles that are programmed to avoid hitting pedestrians are likely to make it harder to use vehicles as a weapon, not more deadly.
- wehadfun 9y ago
  buy a Tesla and hack it into a car bomb You did not read the article
- aidenn0 9y ago
  Fast forward 10 years; the features in the Model S have trickled down to whatever car is 2027's Toyota Camry. You figure out how to hack it over the internet and load a program that uses its cameras to identify a pedestrian and drive straight towards it at full speed. You turn on this program at some busy time of day. There are times of days in which there are tens of thousands of late model Camry's on the road in the US, and this is the very first example I could think of when considering the problem raised in the article.

Animats 9y ago

Real problem, clueless author. The big problem is allowing firmware updates. Many years ago, I knew some of the people involved with the Ford Electronic Engine Control IV, the "EEC IV", used in 1980s Fords. This used a custom CPU, an Intel 8061, which was an Intel 8051 with some extra timer features. The program was etched onto the CPU silicon. There is no way to change that program without replacing the whole unit. There's an external ROM with some tables, different for each engine model, but it's a true ROM, not something that can be rewritten. If you want to replace it, it takes a wrench. The level of paranoia and testing that went into that program was very high. Any bug meant bringing back hundreds of thousands of Fords in a recall. That didn't happen, and there are still many EEC IV vehicles running today, 30 years later. That's what it was like in the days of hardcore embedded programming. I'd argue that safety critical vehicle software should not be downloadable. If a fix is needed, the vehicle has to go back to the dealer for a new memory module. This would discourage "shit early, shit often" software development, and encourage manufacturers to keep the safety critical systems, like ABS, totally disconnected from the entertainment system. Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.)

ohazi 9y ago
My car is >10 years old, and thankfully still running like a champ. It'd be nice to get a new one someday, if only for the updated safety features like curtain airbags, but I honestly don't feel very comfortable with anything much more recent, for the exact reasons you've described.
PinguTS 9y ago
> Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.) You seem to be out of touch what happens today. Car2Car or Car2Infrastructure is about as aircraft to aircraft communication is used in TCAS. It is about what car (object) is at which position, so that collision may be detected as early as possible to prevent them. Also next generation cars will have their maintenance modes locked down as far as possible. OBDII will be closed down as far as possible. All those dongles, which are hip right now, will be rendered useless. But of course the tuning scene has still a lot of incentives like in the past to thwart all this and to reverse engineer. It will be a cat-and-rat game like it always was.

stcredzero 9y ago

Since organizations do not observe an attacker's failure, the market generally does not reward extreme competence in cyber defense... Blackhats, in stark contrast, are sexy. Bounty hunters are sexy. Why doesn't the government start paying bounties? Paying bounty hunters to run honeypots would convert many black hats into hunters of black hats. This could well create an ecosystem where the more knowledgeable hackers directly prey on the script kiddies for fun and profit. Taking useful idiots out of the ecosystem strikes me as desirable. Such a program would also be useful for recruitment. In a way, this is analogous to such bounties in the transition of the wild west into a more normal society.

im3w1l 9y ago
This would incentivize black hats to frame innocents. And black hats strike me as the type of people that would be able and willing to run with it.
- stcredzero 9y ago
  This would incentivize black hats to frame innocents. I thought of that too. This also happened with bounty hunters. There will need to be safeguards.
lucb1e 9y ago
If you have a rat problem and start paying for rat corpses, people might start breeding rats.
- stcredzero 9y ago
  The analogy falls down a bit, because while rats are born rats, a hacker is first a person. People can witness people becoming examples, and decide not to do that. That said, problems can arise, due to national borders, limited jurisdictions, and differences in access to socioeconomic resources.
roywiggins 9y ago
The last time we paid bounties, we ended up with a bunch of people in Guantanamo who had no reason to be there.

bo1024 9y ago

I think this is a really important issue and glad someone is working on awareness. It extends far beyond cars but those are probably the most 'frightening' example. We need to (1) Admit we have a big problem. (2) Admit it will be difficult to solve and there aren't easy solutions. (3) Start working on solutions. Unfortunately, these steps never seem politically popular. But it will be tough. Except in very rare scenarios (NASA), software security and correctness aren't treated as important in the scheme of things, relative to functionality and features. Nobody wants to invest in actually secure, carefully-audited code. (They might claim they do, but their standards for "secure" and "careful" would be litigation-worthy in any field but software.) For example, I believe in freedom of software and ownership rights, which seems to mean that I believe people should be allowed to run open-source code on their own cars' computers. But this impacts public safety. How do we develop reasonable regulations similar to those for physical "street-lega" modifications?

sparkie 9y ago
I have reason to believe that a competent hobbyist programmer is less dangerous than the electronic engineer, pretend-to-be-programmers who wrote the original firmware on the vehicle. Based on years of working with electronic engineers' code. (Apologies to those of you who are competent in both)
- bo1024 9y ago
  I appreciate your point and am inclined to agree. But my agreement doesn't matter. Your point is only one starting point for a challenging and nuanced discussion over the legality and safety of people installing their own software on their own cars. My broader point is that this (among others) is a discussion we should already be having and eventually must have -- if not amongst "ourselves", then with legislators and laypeople. Yet the state of software security, awareness, and incentives is such that it could be years or a decade until we do...

dfabulich 9y ago

> One of the problems I've had over the past year and a half is how to communicate this idea without: > 1. Sounding like a crank. > 2. Giving ideas to terrorists or hostile foreign governments. #1 is this article's biggest problem. These problems are real, but I think the author sounds like a crank. (Based on the way this article is written, I suspect that the author actually is a crank--obsessed with security, but fundamentally lacking in the relevant skills to do anything about it.) If the author is serious about this, here's what you do. 1. Become/join a non-profit organization. You want to be frequently quoted in the press, but not as "well-known software security expert Zach Aysan" but as "Zach Aysan, president of the Organization for Global Security." 2. Build your reputation by finding and earning credit for security problems. Do ethical reporting, but when the issues get fixed, exhibit them in a flashy way. 3. White papers, not blog posts. This "article" starts with a subtitle "with apologies to Elon Musk", followed by a personal dedication to Zach's father. This is not the tone of a white paper from a think tank. The subtitle of the post should be the thesis statement of the article. "Self-driving cars lack adequate security protections." The first paragraph should be an executive summary of the argument of the post. Each paragraph should support the thesis statement. Have someone read your articles; update your work based on their feedback. Thank them in the footer. 4. Separate "how to" articles from arguments. This article is long because it is both attempting to persuade the reader that we haven't invested enough effort into securing critical systems and also to give a list of proposals. These should be separate articles, with one article arguing why security is important, and another article giving a list of proposals.

lucb1e 9y ago
> #1 is this article's biggest problem I don't know, but I had the same idea years ago, and I later read about an author using the idea in a 2006 book (Daemon, by Daniel Suarez). I'd say this shouldn't be an issue, and it also solves number two: if we can think of it, so can hostile governments* and mad men. That said, the article does (after he gets to the point of "weapons of mass destruction") seem to get a little obsessive and, yeah, crank-y. But that has little to do with the concern itself. * Not "hostile foreign governments" because every government is foreign (and perhaps even hostile) to someone.

mtgx 9y ago

> They rebutted that individual cars are much easier to hack and after they are first used in a terror attack we will get the political will to fix the problem. Was that meant to be a joke? Sure, they can establish a new standard that will apply to all the new car models coming out four years later. But who actually expects hundreds of millions of cars that are already on the market, to receive a software overall with a new architecture? This is why it has always bothered me that almost no one seems to bring this problem to the forefront - certainly not carmakers. They're all too focused on how awesome self-driving technology will be and how it will save us from drunk drivers. Thus, disregarding the fact that once we have 100 million to 2 billion self-driving cars on the road, that will be a huge market for cyber criminals, from ransomware and cryptojacking (hello powerful GPU computer + free solar power charging!) to assassinations. And before anyone says "how much harder it is to hack a car than a PC", consider the fact that most cars today aren't actually connected to the internet. And most of those that are, only have their entertainment systems connected to the internet. Self-driving cars will be able to receive OTA updates that will improve their engine, steering, and brake performance = the OTA software has access to everything. Combine this level of access to the high level of recklessness in the name of profits carmakers seem to be showing today, when they advertise features such as "unlocking your doors through an app". EFF's former chair and someone who worked on Google's Waymo, has some decent ideas about how to protect self-driving cars, if only carmakers would listen: http://ideas.4brad.com/disconnected-car-right-security-plan-robocars http://ideas.4brad.com/disconnected-car-right-security-plan-...

3pt14159 9y ago
Airgapping was my first instinct too, but the problem is we're dealing with state-level actors. Airgapping doesn't work with them. They're patient, well funded organizations. Trying to rely on never having a single type of car (any of which could have a hundred thousand copies on the road) hacked is a fools errand. We need ways of disabling autonomous devices and detecting when they get hacked, not trying to win an impossibly hard game.
- bradtemp 9y ago
  The disabling system becomes another attack surface, and unless it is pretty independent, is itself disabled by a sophisticated attack. But scared as we are of external attack, allowing the government to shut off all cars is like letting Mubarak shut off the internet in Egypt. That's a bigger danger than foreign enemies in many countries.

icefox 9y ago

> It's a bit like the Fermi Paradox. During the late 90's and early 00's I often wondered why with all of the Microsoft hate why someone didn't create a simple virus that did something destructive like just format windows hard drives. There must have been other incentives at play that caused this to never occur. Perhaps in the same logic if you can infect every Tesla it is more valuable to not crash them, but instead scrub the data and sell that back to [insert company] or something. If there was ever something that would cause a formal programming guild to sprout I would be willing to bet that it would form its roots around security.

sp332 9y ago
There were plenty of destructive viruses. Some (e.g. CIH) would erase the MBR of the disk making it unbootable. Others like ILOVEYOU would overwrite user's data directly. Blaster specifically had a message about Windows' poor security and tried to DDoS Windows Update. Sometimes viruses are written to by self-limiting. MyDoom, which caused about 10% of all email traffic for a time, was programmed to deactivate on a certain date. Also once viruses get to a certain level of infamy they get a lot of attention. Blaster was mitigated in just days, so by the time it started its DDoS it was already mostly wiped out.
- marcosdumay 9y ago
  Even more relevant, Internet access wasn't common back then, and the only large-range virus vector was the slow paced sneaker-net.
Piskvorrr 9y ago
From the other virology: a virus that outright kills its host doesn't get very far. A virus that keeps the host limping along, creating copies of the virus in the process, is far more likely to spread...which is, IIRC, precisely what happened in the early 2000s. (And indeed, what would be the incentive for disabling Windows hosts? "W1nd0z3 suxx0rz" would barely count as one, given the lack of general-public alternatives at the time - the afflicted would pay a tech grunt to repave with the same Windows again, been there, done the repaving.)

deleted 9y ago

[deleted]

LinuxBender 9y ago

Unpopular opinion ahead. Having worked in the security industry for a few thousand years (computer years), I can say I would never own a car that can talk to the internet. I plan to move far away from cities very soon for this and several other reasons. People will argue about this and meanwhile the "impossible" will happen, repeatedly. I just replaced the engine and transmission in my non internet vehicle and hope to get another 500k miles.

joshl3253 9y ago
Yes, a self-driving car would probably be hacked in the future, plowing into pedestrians or drive off a cliff. However, what's the odds of that, compared to thousands of deaths caused by incompetent human? "In 2013, 3,154 people were killed in distraction-related crashes." [Source just google it].
deeg 9y ago
I understand your concern but right now 30k Americans die in car accidents. Will (potentially) hackable self-driving cars be any more dangerous than today? If you're that concerned about being involved in a car crash it seems to me that you should never leave the house.
- ori_b 9y ago
  > I understand your concern but right now 30k Americans die in car accidents. And a few thousdand died on September 11th, nearly 20 years ago. The collective fear from that made the world a far worse place than the actual death toll. If there's a terrorist attack that targets internet connected cars, what do you think the collective reaction might be?
  - beat 9y ago
    These are Americans we're talking about. The collective reaction would be to run over people in the street if they wear glasses and nerdy t-shirts.
    - 9y ago

vlucas 9y ago

Easy hack to prevent remote control/hijacking of your car: drive a standard. It will never be able to start or drive anywhere remotely.

eric_h 9y ago
Excellent idea. Too bad it's so much harder to get a standard transmission car in the US these days.
- barsonme 9y ago
  I'd be curious to see the manual vs automatic accident rates per miles driven. I have a feeling automatic are a lot higher. I mean, these days, if somebody's driving a manual either they want to or had no other choice. I wonder if that correlates with driving ability at all.
  - Wehrdo 9y ago
    That would be interesting to look into of one had the data. I would bet insurance companies have investigated it. The only evidence I could find was this study: http://journals.sagepub.com/doi/abs/10.1177/1087054706288103 http://journals.sagepub.com/doi/abs/10.1177/1087054706288103 Although I can't access the article, the abstract says that teens with ADHD had higher attention to driving when using a manual transmission vs an automatic.
    - barsonme 9y ago
      That's funny you mention ADHD. I was diagnosed as a teenager and I've always felt more focused while driving a manual than an automatic—it's not as easy to zone out. I've also heard of doctors "prescribing" manuals for people with ADHD. That's just second-hand, though.

antirez 9y ago

Still cars are already computers but so far nothing like that is happening... In theory the fact of being autonomous should be a protection because you could add some control system that can't be updated and can make choices reagreless of what the other systems are telling to do.

skywhopper 9y ago

I don't think the author is wrong about the threat. Unfortunately, his apparent solution--a government mandated standardized "safety module" to intermediate the Internet-connected bits of the vehicle from the non-connected bits--is likely even more risky. Monocultures can be devastated by discovering a single flaw. And there are guaranteed to be many flaws in any non-trivial computer system. Imagine if a straightforward exploit for Spectre that totally compromised a system leaving a hard-to-detect back door via some simple Javascript had existed on January 1, and had been injected into a few popular websites via ad networks... for that matter, how do we know such a thing didn't happen?

olivermarks 9y ago

A witness to Michael Hasting's 2013 'car crash' describes what he saw https://youtu.be/fweyFCFKcp0 https://youtu.be/fweyFCFKcp0

olivermarks 9y ago
DARPA discussion https://youtu.be/6OfcgJ-pl7Q https://youtu.be/6OfcgJ-pl7Q
gowld 9y ago
"car crash" is the only non-debated aspect of the situation. No scare quotes needed. (The crash wasn't caused by remote control hacking -- that's an absurdly complicated and high-profile way for well-funded professional murderers to murder a single relatively minor person. If we was murdered, it was by some kind of mechanical sabotage or by impairing his mental state with some drug.)

bufferoverflow 9y ago

Cars are different though, they aren't general-purpose computers (even if they are based on them). So you can have a chip that checks the checksum of every file, checks if it's all signed with the correct certificate, and shuts down the car even if one bit is off. That's quite different from your laptop or a smartphone, which can run pretty much any code. It would be very very very hard to hack something like that. As in, steal the signing keys from Tesla hard without being noticed. I wouldn't be surprised if their signing server is air-gapped.

deleted 9y ago
[deleted]
beat 9y ago
Cars are also different in that software updates are harder to pull off. There are millions of cars out there on the road right now that have internet connections, and only minimal security between human-convenience internet like map software, and the embedded systems that do things like steering and brakes. And the average lifespan of a car is what, 15 years? Whatever the vulnerabilities, they're going to be there for a long time. And even if there's a firmware update mechanism available, the manufacturers' abilities to maintain older software will also degrade, like legacy systems do everywhere.
rcxdude 9y ago
Car software is designed with safety in mind. While there is some overlap in techniques with security, they are not the same goal. It is possible to be (and I think most cars are) very safe, but not very secure.

carapace 9y ago

FWIW, Open Kernel Labs was acquired by General Dynamics in September 2012.

deleted 9y ago

[deleted]

vermilingua 9y ago

> What I learned was that there were not only no regulations there were no plans for them either. While I think my MP took my concerns seriously and did what she could, I came to understand that political will lags public outcry This interested me more than the idea of car-bombs, are we moving towards a post-law society? Thinking about it, I haven't seen a single piece of legislation regarding self driving cars, cryptocurrencies, or shared computing in my country; and are transport, currency, and communication not the underpinnings of civilisation?

0xCMP 9y ago

I looked and noticed this post has been submitted several times (not a complaint): https://news.ycombinator.com/from?site=zachaysan.com https://news.ycombinator.com/from?site=zachaysan.com Then I noticed he'd recently published an article on Zero Width characters being used for fingerprinting which got some attention on HN in the last 30 days: https://news.ycombinator.com/item?id=16046329 https://news.ycombinator.com/item?id=16046329

woolvalley 9y ago

Defense is harder than attack, but the attackers can be attacked back. Jail isn't worth a few million dollars vs. working in silicon valley. I think that is also a big reason why many potential black hats do not become black hats. The only 'sustainable' black hats are ones associated with a criminal organization or nation states.

whiddershins 9y ago

Am I the only one who thinks it is ironic that Elon Musk gives so much money to research regarding the existential threat of AI to humanity, while also basing a business on putting AI in devices that can so easily kill humans?

daveguy 9y ago
Pretty sure he has a guilty conscience. He wants to build an autonomous self driving car, which will have to get pretty close to general AI and he wants to put it in multi-ton robots. He knows this could be dangerous and scary. At least he is giving money to research the problem rather than just forging ahead with the potentially dangerous part. Personally I think the robot uprising concern is at least 50 years premature. But the "what if someone could take the controls" concern is a concern we should have been already been considering yesterday.
- Veedrac 9y ago
  The fact you said "50" and not "5000" is enough reason to not consider it premature.

schlowmo 9y ago

While I agree with the author about the undervalued risks of autonomous vehicles in the light of careless security practices his proposed solution is just insane. In my opinion the demand for a government controlled kill switch in every piece of hardware that is somehow able to harm people is much more threatening in so many ways than the insecurity the author is trying to reduce. Just a few: 1. Based on the asumption, that ones current government/state is for the good of all people, what gives him the confidence that this will stay that way? What if your beloved government goes rogue? That's a lot power for an autocratic regime. 2. Why even trust the government in the first place with that kill switch? They are the same people which are careless about infrastructure critical ITSec since decades. 3. An univervsal security module which is highly standardized is a very profitable target. While the author is aware that finding an attack vector of one particular vehicle can mean that all vehicles of that type can be compromised, he doesn't come to the conclusion that the same logic applies to his security module.

3pt14159 9y ago
I’m the author. I was nodding in agreement with you for your first two points. I don’t think I’ve come to a perfect solution, but I want you to know that I actually have many of the same reservations that you have and that I’m open to changing my mind. Here is where I still sit, however: the government had predator drones and will soon have killbots able to take out individuals based on facial recognition software. If we can’t trust them to turn off our cars we’re fucked anyway.
- nickodell 9y ago
  These safety modules sound really complicated. Let's say a security vulnerability has been discovered. An attacker has wormed their way into as many cars as they can. They send a 'go' signal from their command and control servers. Across the world, cars start looking for opportunities to kill their passengers and bystanders. We send the shutdown signal. The safety modules wake up and take over from the compromised computers. What do they do? They could brake. However, the car might be in a turn, on a rainy day. Braking could send the car into a skid and kill its passengers. Maybe they brake, but slowly. However, the car might be behind someone who just suddenly changed lanes and slammed on the brakes. Braking hard might be the right move in that circumstance. Maybe they do something conditional on the sensor input they get. However, if the control computer can do something to 'blind' the safety computer, that doesn't help. For example, can the control computer issue firmware updates to the camera sensors? Can the control computer fill a sensor's bus line until it can't respond? I can't think of any solution to this that doesn't involve duplicating a substantial part of the control computer.

deevolution 9y ago

The human mind is also hackable... it just takes a considerably longer time to hack it compared to installing software on a computer. Id argue that the internet and social media has increased the speed at which the mind can be hacked by a malicious actor. Im not so sure which is worse now, autonomous vehicles or human drivers? I think its much harder to detect when a human has been "hacked" than a fleet of cars. Cars could have subsystems build in for detecting anomolies, axuilary computers and manual overdrive settings. Detecting when a human has been "hacked" requires invasion of privacy, constant surveillance, reliable friends and family, and a whole bunch of other variables. Preventing a human from being hacked might require arduous and expensive changes and experiments in legal systems, incentive mechanism, censorship, education,etc...

deevolution 9y ago
There's going to be casualties no matter what... that seems to be the nature of technological advancement.

Klasiaster 9y ago

First step would be to stop public funding of 0day purchase and development in NSA and co. People believe that more 'cyberweapons' would be good against cyberthreats from 'others' but meanwhile all use the same systems. If these systems do not get fixed because the NSA wants to use its purchased 0days then everybody has a problem. What is needed is joint effort for secure open source solutions. Also, for computer security programmers need to overcome convenience practices and start behaving responsible, but also people in management/politics should not make technical decisions if they can't understand the implications.

matthewowen 9y ago

I know this is somewhat off topic, but what's the deal with "Look no further than the Clinton email breach to see how much a single hacker can change the world."? To my knowledge, there's no evidence that Clinton's email was breached. The DNC, sure, but the Clinton email controversy wasn't based on any actual known breach. It's sort of alarming to see this weird retconning of history.

bdamm 9y ago

Some solutions are strangely misguided. UDP is insecure but TCP is somehow magically secure? Certificates are not to be trusted? Can't agree with all the conclusions but the specific paranoia is well founded.

King-Aaron 9y ago

With the continued reporting of these possible attack vectors, it makes me feel that there's a lot to be said about having a carburettor and points running your car, instead of a computer with network connectivity.

hueving 9y ago

>because obscurity is a valid defensive measure. It's how passwords and authorization tokens work. This is extremely dangerous thinking. Passwords and tokens are not in the 'security by obscurity' category because you can observe how the entire system works, review its source code, read its deployment configuration, and do packet captures of the encrypted valid traffic and still not have a way to gain access to the system. Security by obscurity refers to hiding how the system actually works, and that's a lot harder to keep a secret because you are one compromised device away from revealing all of that and it can't be easily changed. Do not call passwords and tokens "security by obscurity" or claim security by obscurity practices (e.g. Running on non-standard port numbers) is on the same level as passwords/tokens/keys. There is a reason the strongest crypto is using public algorithms and only private keys. Obscuring a system just means that your silly bugs don't get found and exposed early.

Piskvorrr 9y ago
Not a very strong measure, definitely not against a determined attacker, but valid nevertheless. A mosquito net for opsec, so to speak.