5 ms·

> There's no end of things you could do around the roads to cause crashes, but they're all illegal and ultimately low impact. But most of them are easy to spot

> There's no end of things you could do around the roads to cause crashes, but they're all illegal and ultimately low impact. But most of them are easy to spot, at least in the aftermath. If you spayed over a sign, people would notice. If you removed one, a driver would probably not be able to directly notice, but he might notice the situation (i.e. you should not go fast in front of a school, even when the sign is missing) or you could find evidence if an accident happens. The scary thing about these signs is that they could easily go unnoticed, especially if placed well (for example a crossroad with already sun-bleached signs). Neither a driver (who could possibly intervene) or an investigator would be able to spot this, unless they're specifically looking for it exactly. Things like this are, in my opinion, also exactly why people put the bar of trust so high for autonomous vehicles: They might fail in ways totally opaque to us. You can at least somewhat estimate what a person might do [0]; for autonomous cars, this is hard or even impossible. [0] I know someone could be extremely drunk or on drugs and possibly also mistake a stop sign like this, but you'd usually see some warning signs. An autonomous car might go from totally normal to batshit insane without any prior indication or apparent reason.

Retric 5y ago
> from totally normal to batshit insane Self driving cars don’t put that much emphasis on street signs, their not for example going to drive into a brick wall because of an arrow. Alter speed limit signs in front of a speed trap might hit a lot of people but their unlikely to try and take a sharp turn at 70MPH or anything.
- Sebb767 5y ago
  > Self driving cars don’t put that much emphasis on street signs, They have to. Even the most accurate maps of speed limits have flaws and cannot adapt to dynamic limits. I'm pretty sure that if car makers could avoid the trouble of parsing signs, the would do so. > their not for example going to drive into a brick wall because of an arrow. Teslas first driver death was due to the car not recognizing a white truck. It would be easy to add a fake increase in speed limit, just before a white enough wall, so that the car would not be able to brake in time once the LIDAR picks the obstacle up. To the casual observer it would look like the car just decided to accelerate into a wall.
  - lazide 5y ago
    Tesla cars don’t have LiDAR right? That’s why they are so susceptible to this problem. Visual sensors only.
    - asdff 5y ago
      It feels borderline irresponsible that Tesla is visual only. Why not collect another datapoint in the form of lidar? It's human lives at stake, don't tell me lidar is too costly: M robot vacuum uses Lidar and the thing cost 1/10th the price of putting leather seats on most cars today. It's excellent, the vacuum has no issues with my white walls and it paints beautiful maps of my apartment. I'd strap myself to it and go around town right now if it could only support the weight.
      - XorNot 5y ago
        LIDAR surely wouldn't work very well in rain though? Same problem with the RADAR - can't see RADAR transparent objects. I kind of understand where they're coming from: a sensor where under some conditions you have to totally disregard all its input is a really bad sensor to have because your model will get disproportionately good using the "good" input, and then you're still left with "and in a medium drizzle we need to also work off optics only".
        lazide 5y ago
        Not really though right? Sensor fusion is used, and it’s a generally solved problem. We do the same thing - we have sound, touch (which can also pick up changes in air pressure, humidity, temperature, wind), visual, and internal model/balance. We’ve evolved to do that because it’s necessary for survival - sometimes we get dust in our eyes, or there is glare, or it’s too dark to see and we don’t have a light, or we’re picking up on a impending problem with something around a corner (a loud fight, or whistling wind), etc. If visual sensors and LiDAR suck in the rain, radar is great. If radar sucks in crowded road or RF noisy environments, that’s when LiDAR or visual sensors are awesome. Even on the road this is true - rumble strips, mechanical noise, shaking, etc.
- sillycross 5y ago
  > not for example going to drive into a brick wall because of an arrow. If the self driving car misinterpreted a "NO SWIMMING" sign to a "MUST TURN RIGHT" sign, I assume it's going to take the turn and drive into a lake?
  - NavinF 5y ago
    Why would the car follow a sign instead of Google maps?
    - sillycross 5y ago
      Because maps can always be out of date? It's common that due to construction, temporary road signs are placed to re-route traffic. It would be a disaster if the self driving car simply ignores them because they are not on the map, I assume?
- jazzyjackson 5y ago
  > drive into a brick wall because of an arrow. I honestly would push back on this, the car is more likely to recognize the arrows/lane markers and might not understand a brick wall - from what I can tell from watching the FSD ride-alongs, if they car doesn’t understand something it just ignores it.
shagie 5y ago
> Things like this are, in my opinion, also exactly why people put the bar of trust so high for autonomous vehicles: They might fail in ways totally opaque to us. You can at least somewhat estimate what a person might do [0]; for autonomous cars, this is hard or even impossible. This is the subject of the story Car Wars by Cory Doctorow - https://web.archive.org/web/20170105065118/http://this.deakin.edu.au/culture/car-wars https://web.archive.org/web/20170105065118/http://this.deaki...
dheera 5y ago
> But most of them are easy to spot I mean, walking upto a stop sign dangling a projector on a stick is pretty damn easy to notice. It's also much easier to just hold up a bogus actual sign than to go through this mess, and it would be highly illegal as well.
- jazzyjackson 5y ago
  Could etch the pattern onto a mirror that reflects sunlight onto the sign at the chosen time of day that your chosen victim drives by everyday (Disclaimer, I doubt this works because the adversarial image is probably based on a single frame, not the actual object in context)

Retric 5y ago

> from totally normal to batshit insane Self driving cars don’t put that much emphasis on street signs, their not for example going to drive into a brick wall because of an arrow. Alter speed limit signs in front of a speed trap might hit a lot of people but their unlikely to try and take a sharp turn at 70MPH or anything.

Sebb767 5y ago
> Self driving cars don’t put that much emphasis on street signs, They have to. Even the most accurate maps of speed limits have flaws and cannot adapt to dynamic limits. I'm pretty sure that if car makers could avoid the trouble of parsing signs, the would do so. > their not for example going to drive into a brick wall because of an arrow. Teslas first driver death was due to the car not recognizing a white truck. It would be easy to add a fake increase in speed limit, just before a white enough wall, so that the car would not be able to brake in time once the LIDAR picks the obstacle up. To the casual observer it would look like the car just decided to accelerate into a wall.
- lazide 5y ago
  Tesla cars don’t have LiDAR right? That’s why they are so susceptible to this problem. Visual sensors only.
  - asdff 5y ago
    It feels borderline irresponsible that Tesla is visual only. Why not collect another datapoint in the form of lidar? It's human lives at stake, don't tell me lidar is too costly: M robot vacuum uses Lidar and the thing cost 1/10th the price of putting leather seats on most cars today. It's excellent, the vacuum has no issues with my white walls and it paints beautiful maps of my apartment. I'd strap myself to it and go around town right now if it could only support the weight.
    - XorNot 5y ago
      LIDAR surely wouldn't work very well in rain though? Same problem with the RADAR - can't see RADAR transparent objects. I kind of understand where they're coming from: a sensor where under some conditions you have to totally disregard all its input is a really bad sensor to have because your model will get disproportionately good using the "good" input, and then you're still left with "and in a medium drizzle we need to also work off optics only".
      - lazide 5y ago
        Not really though right? Sensor fusion is used, and it’s a generally solved problem. We do the same thing - we have sound, touch (which can also pick up changes in air pressure, humidity, temperature, wind), visual, and internal model/balance. We’ve evolved to do that because it’s necessary for survival - sometimes we get dust in our eyes, or there is glare, or it’s too dark to see and we don’t have a light, or we’re picking up on a impending problem with something around a corner (a loud fight, or whistling wind), etc. If visual sensors and LiDAR suck in the rain, radar is great. If radar sucks in crowded road or RF noisy environments, that’s when LiDAR or visual sensors are awesome. Even on the road this is true - rumble strips, mechanical noise, shaking, etc.
sillycross 5y ago
> not for example going to drive into a brick wall because of an arrow. If the self driving car misinterpreted a "NO SWIMMING" sign to a "MUST TURN RIGHT" sign, I assume it's going to take the turn and drive into a lake?
- NavinF 5y ago
  Why would the car follow a sign instead of Google maps?
  - sillycross 5y ago
    Because maps can always be out of date? It's common that due to construction, temporary road signs are placed to re-route traffic. It would be a disaster if the self driving car simply ignores them because they are not on the map, I assume?
jazzyjackson 5y ago
> drive into a brick wall because of an arrow. I honestly would push back on this, the car is more likely to recognize the arrows/lane markers and might not understand a brick wall - from what I can tell from watching the FSD ride-alongs, if they car doesn’t understand something it just ignores it.

shagie 5y ago

> Things like this are, in my opinion, also exactly why people put the bar of trust so high for autonomous vehicles: They might fail in ways totally opaque to us. You can at least somewhat estimate what a person might do [0]; for autonomous cars, this is hard or even impossible. This is the subject of the story Car Wars by Cory Doctorow - https://web.archive.org/web/20170105065118/http://this.deakin.edu.au/culture/car-wars https://web.archive.org/web/20170105065118/http://this.deaki...

dheera 5y ago

> But most of them are easy to spot I mean, walking upto a stop sign dangling a projector on a stick is pretty damn easy to notice. It's also much easier to just hold up a bogus actual sign than to go through this mess, and it would be highly illegal as well.

jazzyjackson 5y ago
Could etch the pattern onto a mirror that reflects sunlight onto the sign at the chosen time of day that your chosen victim drives by everyday (Disclaimer, I doubt this works because the adversarial image is probably based on a single frame, not the actual object in context)