6 ms·
The problem with most of these types of "attacks" is that they're about the same as any other messing with roadsigns or what not. There's no end of things you c
by XorNot 5y ago
The problem with most of these types of "attacks" is that they're about the same as any other messing with roadsigns or what not. There's no end of things you could do around the roads to cause crashes, but they're all illegal and ultimately low impact.
This is a lot more interesting though from how it highlights the difference between current machine interpretation and human interpretation of road signs: which is to say, explainable AI is still incredibly important - the ability to slightly discolor the signs and have a blackbox AI read something entirely different means the systems and trainers we have are not cueing off a "safe" set of inputs to deliver their interpretations.
From that context, this is very interesting and important work - and I would argue also points the way to at least some basic road safety standards for self-driving systems (they should be resistant to this sort of non-human perceptible visual distortion).
- jcims 5y agoI do wonder if there could be some kind of 'adversarial training' of large networks, like those of Tesla, in which drivers in an area collude to normalize dangerous behavior in an area (e.g. drive in oncoming lane, stop at an overpass on a major highway, etc). Even with humans in the loop in the training process, there's only so much context they can bring to override data that appears authentic.
- asdff 5y agoI'd me more worried about adversarial employees. Tesla has a reputation of working people to the bone for not much comp compared to the competition. I can't imagine they are connecting their stressed workers to psychologists. If one engineer just cracked one day, they could potentially push an update to all cars that does something terrible.
- jcims 5y agoSure. I'd guess someone could also sneak an easter egg in there to cause the battery packs to self-immolate.
- hirundo 5y ago> ...and ultimately low impact. A suboptimal metaphor in context.
- arthurcolle 5y agochef's kiss
- Sebb767 5y ago> There's no end of things you could do around the roads to cause crashes, but they're all illegal and ultimately low impact. But most of them are easy to spot, at least in the aftermath. If you spayed over a sign, people would notice. If you removed one, a driver would probably not be able to directly notice, but he might notice the situation (i.e. you should not go fast in front of a school, even when the sign is missing) or you could find evidence if an accident happens. The scary thing about these signs is that they could easily go unnoticed, especially if placed well (for example a crossroad with already sun-bleached signs). Neither a driver (who could possibly intervene) or an investigator would be able to spot this, unless they're specifically looking for it exactly. Things like this are, in my opinion, also exactly why people put the bar of trust so high for autonomous vehicles: They might fail in ways totally opaque to us. You can at least somewhat estimate what a person might do [0]; for autonomous cars, this is hard or even impossible. [0] I know someone could be extremely drunk or on drugs and possibly also mistake a stop sign like this, but you'd usually see some warning signs. An autonomous car might go from totally normal to batshit insane without any prior indication or apparent reason.
- Retric 5y ago> from totally normal to batshit insane Self driving cars don’t put that much emphasis on street signs, their not for example going to drive into a brick wall because of an arrow. Alter speed limit signs in front of a speed trap might hit a lot of people but their unlikely to try and take a sharp turn at 70MPH or anything.
- Sebb767 5y ago> Self driving cars don’t put that much emphasis on street signs, They have to. Even the most accurate maps of speed limits have flaws and cannot adapt to dynamic limits. I'm pretty sure that if car makers could avoid the trouble of parsing signs, the would do so. > their not for example going to drive into a brick wall because of an arrow. Teslas first driver death was due to the car not recognizing a white truck. It would be easy to add a fake increase in speed limit, just before a white enough wall, so that the car would not be able to brake in time once the LIDAR picks the obstacle up. To the casual observer it would look like the car just decided to accelerate into a wall.
- Dylan16807 5y agoI feel like there's a categorical difference between actually changing a road sign to say the wrong thing, versus adding some kind of smudge to it that certain computers read in an objectively wrong way. The latter could easily be part of a dumb prank, "doing nothing wrong", but cause serious damage. This piece of tape shouldn't kill anyone: https://www.extremetech.com/wp-content/uploads/2020/02/tesla-tape.png https://www.extremetech.com/wp-content/uploads/2020/02/tesla...
- xwdv 5y agoThe problem is not that you can cause crashes, it’s that you can cause crashes at scale.