9 ms·

Nice. So we have some cars that rely on visual information for self-driving. And we have some cars that can project onto the landscape, modifying visual infor

by buro9 4y ago

Nice. So we have some cars that rely on visual information for self-driving. And we have some cars that can project onto the landscape, modifying visual information. What could go wrong! I only have a Volvo, in the UK, and it has one of those systems that tell you the last road sign that it has seen. That information then can optionally control warnings, speed limiter, etc. I typically do use the speed limiter, it works well... until I was behind a car that had a 20mph bumper sticker on the back. Then my car kept insisting that 20mph was the speed limit. I was on a motorway. I also think of that Tesla stop light truck https://www.youtube.com/watch?v=ch7CJEQ2kz0 https://www.youtube.com/watch?v=ch7CJEQ2kz0 , and this https://uk.motor1.com/news/449020/video-tesla-model-3-traffic-sign-recognition-tricked/ https://uk.motor1.com/news/449020/video-tesla-model-3-traffi... , and this https://techxplore.com/news/2020-02-road-self-driving-car-high-speed-mode.html https://techxplore.com/news/2020-02-road-self-driving-car-hi... Plot for a future novel: Project light onto surfaces from a moving vehicle to make another car behave in a way that kills the occupants. No evidence can be found at the scene as to why the car killed the occupants.

micromacrofoot 4y ago
Autonomous Trap 001 is another: https://jamesbridle.com/works/autonomous-trap-001 https://jamesbridle.com/works/autonomous-trap-001
stickfigure 4y ago
How about blinding the (meat) driver with a low-powered laser? If you want to murder someone, there are many options. I wouldn't bet on "no evidence" for the autonomous vehicle. It may record telemetry and video.
- buro9 4y ago
  > It may record telemetry and video. For the novel, probably neither of the above is recorded in high enough resolution. Thinking of subliminal messages inserted into old movies here, just flashing something up for a hundredth of a second, doing it enough that a system detects something... but not enough that video filmed at 30fps really shows it, and that telemetry doesn't really capture it. All the telemetry would show is "car thought X", and the video would appear to be clean. What isn't known from is "why" car thought something.
  - wyre 4y ago
    Old movies are projected at 24fps so subliminal messages like you mention would be noticeable from a 30fps camera.
  - solveit 4y ago
    We have systems log stuff when there's less at stake. You'd have to really go out of your way to make this an unsolvable mystery murder.
    - buro9 4y ago
      That's a shame, I was thinking a modern Crow Road (Iain Banks) with it being an accident that only later you pieced together that a person was in fact a suspect because they'd had these skills and could've done this far out thing.
seri4l 4y ago
Humans also rely on visual information for driving and can easily be tricked. For instance, I don't think there is any security mechanism to prevent the placement of a battery powered green light on top of a red traffic light. In that case, an autonomous car's logs could actually help in discovering the tampering.
- HeyLaughingBoy 4y ago
  No, but the reason there are three (at least) lights is that the designers were well aware that colorblindness exists and the position of the light is also significant.
  - xani_ 4y ago
    It's still kinda shit design to be honest. Could have both color and shape. Make red hexagon, yellow square for example. or really any recognizable shape
    - HeyLaughingBoy 4y ago
      I'm sure that manufacturability is a concern.
      - xani_ 4y ago
        I'm sure it's no harder than making one in shape of arrow lmao
- verbify 4y ago
  Interfering with a traffic signal is a crime in many jurisdictions.
bombcar 4y ago
I am actually slightly surprised that road signs haven't been updated to include something like a "barcode" or similar that would make it very easy for the machine to identify what sign it is. You could "paint" over the entire sign with a variation in the infra-red reflectivity with a QR code, for example. Then the computer would be better at recognizing a sign than a human.
- jaywalk 4y ago
  Signs are already reflective and pretty standardized. I don't think machine-readable codes are really necessary for this.
  - labcomputer 4y ago
    One advantage of something machine-readable, like a QR code, would be to incorporate additional information (more than could be easily consumed by a human). For example, a speed limit sign could include the name of the road it controls, which would help reduce instances of cars suddenly slowing to 35 mph on the freeway when passing a sign on the frontage road. It might also be useful for cases of (this is US-centric) yellow "advisory" speeds posted for, e.g., sharp curves. The machine-readable aspect might indicate how long the advisory speed applies (next 1000 ft) and why (sharp curve, hidden intersection, etc.)
    - xani_ 4y ago
      If you just want to feed computers just put a database with every sign location and data. That way even faking or destroying sign would be ineffective. Would have to be updated during temporary situations (roadwork etc) tho
      - mbreese 4y ago
        > That way even faking or destroying sign would be ineffective In theory you could add an HMAC (or similar) signature to the QR code. It would be bigger, but if cars maintained a DB of state DOT public keys, you could mitigate this somewhat. Then you’d also have to worry about duplicate codes, but at least they wouldn’t be “fake”. Maybe even add lat/long coordinates to the code… again, more data, more pixels required, but it would make sure the signs are roughly in the right place. Which doesn’t seem all that feasible to require construction crews to put the right sign in the right place. Logistically this doesn’t seem all that promising.
googlryas 4y ago
Except in the self driving computer's recordings of its inputs...
- speed_spread 4y ago
  First, display an uproad pattern crafted to cause a buffer overflow. 0wn the autpilot, trash recording. Bonus points for taking over the entertainment system, cueing Death Grips "Guillotine" at max volume as the car leaves the road.
shreddit 4y ago
We are lucky that most people aren’t murderers: https://xkcd.com/1958/ https://xkcd.com/1958/
xani_ 4y ago
> Plot for a future novel: Project light onto surfaces from a moving vehicle to make another car behave in a way that kills the occupants. No evidence can be found at the scene as to why the car killed the occupants. Aside, well, the video the car recorded...
reginaldo 4y ago
Add the "autonomous car trapped by a circle of salt" to your repertoire if you haven't seen it already: https://www.youtube.com/watch?v=thuN2HD6m2s https://www.youtube.com/watch?v=thuN2HD6m2s
adhesive_wombat 4y ago
Most interesting to think about are the stickers[1] that look random but actually trigger some kind of adversarial behaviour, a bit like how tiny human-imperceptible changes to an image can completely flip an image categorisation network's guess. Perhaps most fun to think about in a Dan Brown-style NSA capabilities combined with Peter Watts' Blindsight's near-total-imformation perceptual-cognitive shenanigans kind of way. [1]: https://medium.com/self-driving-cars/adversarial-traffic-signs-fd16b7171906 https://medium.com/self-driving-cars/adversarial-traffic-sig...
bearmode 4y ago
I was actually on a 40mph road earlier today when I passed a side street with a 20mph sign that had been rotated to face the main road. My car was then insisting it was 20 for the rest of the road. >Plot for a future novel: Project light onto surfaces from a moving vehicle to make another car behave in a way that kills the occupants. No evidence can be found at the scene as to why the car killed the occupants. Problem with this is that self-driving cars like teslas will probably have cameras all over them

by buro9 4y ago

micromacrofoot 4y ago

Autonomous Trap 001 is another: https://jamesbridle.com/works/autonomous-trap-001 https://jamesbridle.com/works/autonomous-trap-001

stickfigure 4y ago

How about blinding the (meat) driver with a low-powered laser? If you want to murder someone, there are many options. I wouldn't bet on "no evidence" for the autonomous vehicle. It may record telemetry and video.

buro9 4y ago
> It may record telemetry and video. For the novel, probably neither of the above is recorded in high enough resolution. Thinking of subliminal messages inserted into old movies here, just flashing something up for a hundredth of a second, doing it enough that a system detects something... but not enough that video filmed at 30fps really shows it, and that telemetry doesn't really capture it. All the telemetry would show is "car thought X", and the video would appear to be clean. What isn't known from is "why" car thought something.
- wyre 4y ago
  Old movies are projected at 24fps so subliminal messages like you mention would be noticeable from a 30fps camera.
- solveit 4y ago
  We have systems log stuff when there's less at stake. You'd have to really go out of your way to make this an unsolvable mystery murder.
  - buro9 4y ago
    That's a shame, I was thinking a modern Crow Road (Iain Banks) with it being an accident that only later you pieced together that a person was in fact a suspect because they'd had these skills and could've done this far out thing.

seri4l 4y ago

Humans also rely on visual information for driving and can easily be tricked. For instance, I don't think there is any security mechanism to prevent the placement of a battery powered green light on top of a red traffic light. In that case, an autonomous car's logs could actually help in discovering the tampering.

HeyLaughingBoy 4y ago
No, but the reason there are three (at least) lights is that the designers were well aware that colorblindness exists and the position of the light is also significant.
- xani_ 4y ago
  It's still kinda shit design to be honest. Could have both color and shape. Make red hexagon, yellow square for example. or really any recognizable shape
  - HeyLaughingBoy 4y ago
    I'm sure that manufacturability is a concern.
    - xani_ 4y ago
      I'm sure it's no harder than making one in shape of arrow lmao
verbify 4y ago
Interfering with a traffic signal is a crime in many jurisdictions.

bombcar 4y ago

I am actually slightly surprised that road signs haven't been updated to include something like a "barcode" or similar that would make it very easy for the machine to identify what sign it is. You could "paint" over the entire sign with a variation in the infra-red reflectivity with a QR code, for example. Then the computer would be better at recognizing a sign than a human.

jaywalk 4y ago
Signs are already reflective and pretty standardized. I don't think machine-readable codes are really necessary for this.
- labcomputer 4y ago
  One advantage of something machine-readable, like a QR code, would be to incorporate additional information (more than could be easily consumed by a human). For example, a speed limit sign could include the name of the road it controls, which would help reduce instances of cars suddenly slowing to 35 mph on the freeway when passing a sign on the frontage road. It might also be useful for cases of (this is US-centric) yellow "advisory" speeds posted for, e.g., sharp curves. The machine-readable aspect might indicate how long the advisory speed applies (next 1000 ft) and why (sharp curve, hidden intersection, etc.)
  - xani_ 4y ago
    If you just want to feed computers just put a database with every sign location and data. That way even faking or destroying sign would be ineffective. Would have to be updated during temporary situations (roadwork etc) tho
    - mbreese 4y ago
      > That way even faking or destroying sign would be ineffective In theory you could add an HMAC (or similar) signature to the QR code. It would be bigger, but if cars maintained a DB of state DOT public keys, you could mitigate this somewhat. Then you’d also have to worry about duplicate codes, but at least they wouldn’t be “fake”. Maybe even add lat/long coordinates to the code… again, more data, more pixels required, but it would make sure the signs are roughly in the right place. Which doesn’t seem all that feasible to require construction crews to put the right sign in the right place. Logistically this doesn’t seem all that promising.

googlryas 4y ago

Except in the self driving computer's recordings of its inputs...

speed_spread 4y ago
First, display an uproad pattern crafted to cause a buffer overflow. 0wn the autpilot, trash recording. Bonus points for taking over the entertainment system, cueing Death Grips "Guillotine" at max volume as the car leaves the road.

shreddit 4y ago

We are lucky that most people aren’t murderers: https://xkcd.com/1958/ https://xkcd.com/1958/

xani_ 4y ago

> Plot for a future novel: Project light onto surfaces from a moving vehicle to make another car behave in a way that kills the occupants. No evidence can be found at the scene as to why the car killed the occupants. Aside, well, the video the car recorded...

reginaldo 4y ago

Add the "autonomous car trapped by a circle of salt" to your repertoire if you haven't seen it already: https://www.youtube.com/watch?v=thuN2HD6m2s https://www.youtube.com/watch?v=thuN2HD6m2s

adhesive_wombat 4y ago

Most interesting to think about are the stickers[1] that look random but actually trigger some kind of adversarial behaviour, a bit like how tiny human-imperceptible changes to an image can completely flip an image categorisation network's guess. Perhaps most fun to think about in a Dan Brown-style NSA capabilities combined with Peter Watts' Blindsight's near-total-imformation perceptual-cognitive shenanigans kind of way. [1]: https://medium.com/self-driving-cars/adversarial-traffic-signs-fd16b7171906 https://medium.com/self-driving-cars/adversarial-traffic-sig...

bearmode 4y ago

I was actually on a 40mph road earlier today when I passed a side street with a 20mph sign that had been rotated to face the main road. My car was then insisting it was 20 for the rest of the road. >Plot for a future novel: Project light onto surfaces from a moving vehicle to make another car behave in a way that kills the occupants. No evidence can be found at the scene as to why the car killed the occupants. Problem with this is that self-driving cars like teslas will probably have cameras all over them