3 ms·

These guys had to work really, really hard to find a way to fool the machine vision of recent self-driving software! My take: Maybe the machine vision of recen

by cs702 2y ago

These guys had to work really, really hard to find a way to fool the machine vision of recent self-driving software! My take: Maybe the machine vision of recent self-driving software has become harder to fool than human vision? Human vision is remarkably easy to fool. See https://www.ritsumei.ac.jp/~akitaoka/index-e.html https://www.ritsumei.ac.jp/~akitaoka/index-e.html and https://en.wikipedia.org/wiki/Optical_illusion https://en.wikipedia.org/wiki/Optical_illusion for example. Luckily for all of us, there are no smart people laboring all day long in a lab trying to find new ways to fool human drivers into doing dumb, dangerous things on the road. Human drivers already do that on their own -- no tricks or illusions are necessary.

Dylan16807 2y ago
Optical illusions are cool and all, but can you name one that would make it harder to read a sign? We don't make things where you need to pick up on small differences in alignment or color, and the illusions that cause motion have to be very big to be a distraction.
- cs702 2y ago
  https://www.google.com/search?q=confusing+road+signs&udm=2 https://www.google.com/search?q=confusing+road+signs&udm=2 Illusions aren't even necessary with human drivers, many of whom are shockingly bad -- look around next time you're on the road.
  - Dylan16807 2y ago
    If you put up a big sign that says "potato" it's going to confuse both humans and computers, but that's not fooling vision in any way. The type of issue in the article needs signs that seem to be working to humans, but are secretly failing. Not just "oh they put up too much in a single spot" or "there are two signs that contradict each other".
    - cs702 2y ago
      Given how bad human drivers already are, I wouldn't be surprised if there are lots of ways to fool them that don't fool the machines. Speaking from experience, having used Tesla's "Supervised FSD" v12.3.6, which is far from perfect, I've been surprised by how often it picks up things (signs, cars, pedestrians, etc.) that I have missed on the road. AFAIK, no one is laboring to find how to fool human drivers into misreading road signs.
- bastawhiz 2y ago
  Yesterday I saw someone with their left turn signal on. They were turning left (the wrong way) onto a one way street. There was a one way arrow, a Do Not Enter sign, and an arrow for their lane pointing straight ahead only. I pulled up next to them on my bike and tapped on their window. They were unpeeling a banana. I shook my finger and pointed at the sign. The light turned green and they turned into three lanes of quickly oncoming traffic. I'm honestly not worried about cars being occasionally unable to read a sign. In fact, I know my car already knows where the signs are and what they say before it sees them. I'll always be more scared of human drivers: computers can't get distracted peeling a banana.
  - smegsicle 2y ago
    if someone on a bike shook their finger at me i'd turn towards them
    - bastawhiz 2y ago
      That was sort of the point, we were stopped at a light, I wanted to get her attention and wagging my finger and pointing at the sign was the best I could come up with to try to explain that she couldn't make that turn
- Mountain_Skies 2y ago
  Not an illusion but people still run into the 11'8" (now +8") bridge despite plenty of signage. https://www.youtube.com/watch?v=8qiGP72GFUc https://www.youtube.com/watch?v=8qiGP72GFUc
dogma1138 2y ago
Researching adversarial patterns is important especially for systems you can’t formally verify other than through exhaustive use testing. It’s not just about are they harder to fool than humans but to find the conditions under which they are fooled which are often can be very different. This is needed to both identify edge cases where these patterns can appear in the wild and to make the systems resistant to outside interference.
- cs702 2y ago
  I agree. It seems the edge cases that trip these systems are becoming so rare and convoluted that the return on this kind of research is approaching zero. At some point we just need to collect data from real-world tests.
  - dogma1138 2y ago
    Could be, but in this case it may actually have real world implications. LED wall displays are becoming more and more common, I’ve already seen ones that cause annoying reflections on road signs as they turn the retro-reflectors into a disco ball. I’ve skimmed through the paper and I can’t understand how likely this to occur unintentionally in the wild but their strobing pattern doesn’t seem to be that aggressive so a combination of the animation itself on the sign and LED strobing due to PWM control can quite possibly cause similar interference.

by cs702 2y ago

Dylan16807 2y ago

Optical illusions are cool and all, but can you name one that would make it harder to read a sign? We don't make things where you need to pick up on small differences in alignment or color, and the illusions that cause motion have to be very big to be a distraction.

cs702 2y ago
https://www.google.com/search?q=confusing+road+signs&udm=2 https://www.google.com/search?q=confusing+road+signs&udm=2 Illusions aren't even necessary with human drivers, many of whom are shockingly bad -- look around next time you're on the road.
- Dylan16807 2y ago
  If you put up a big sign that says "potato" it's going to confuse both humans and computers, but that's not fooling vision in any way. The type of issue in the article needs signs that seem to be working to humans, but are secretly failing. Not just "oh they put up too much in a single spot" or "there are two signs that contradict each other".
  - cs702 2y ago
    Given how bad human drivers already are, I wouldn't be surprised if there are lots of ways to fool them that don't fool the machines. Speaking from experience, having used Tesla's "Supervised FSD" v12.3.6, which is far from perfect, I've been surprised by how often it picks up things (signs, cars, pedestrians, etc.) that I have missed on the road. AFAIK, no one is laboring to find how to fool human drivers into misreading road signs.
bastawhiz 2y ago
Yesterday I saw someone with their left turn signal on. They were turning left (the wrong way) onto a one way street. There was a one way arrow, a Do Not Enter sign, and an arrow for their lane pointing straight ahead only. I pulled up next to them on my bike and tapped on their window. They were unpeeling a banana. I shook my finger and pointed at the sign. The light turned green and they turned into three lanes of quickly oncoming traffic. I'm honestly not worried about cars being occasionally unable to read a sign. In fact, I know my car already knows where the signs are and what they say before it sees them. I'll always be more scared of human drivers: computers can't get distracted peeling a banana.
- smegsicle 2y ago
  if someone on a bike shook their finger at me i'd turn towards them
  - bastawhiz 2y ago
    That was sort of the point, we were stopped at a light, I wanted to get her attention and wagging my finger and pointing at the sign was the best I could come up with to try to explain that she couldn't make that turn
Mountain_Skies 2y ago
Not an illusion but people still run into the 11'8" (now +8") bridge despite plenty of signage. https://www.youtube.com/watch?v=8qiGP72GFUc https://www.youtube.com/watch?v=8qiGP72GFUc

dogma1138 2y ago

Researching adversarial patterns is important especially for systems you can’t formally verify other than through exhaustive use testing. It’s not just about are they harder to fool than humans but to find the conditions under which they are fooled which are often can be very different. This is needed to both identify edge cases where these patterns can appear in the wild and to make the systems resistant to outside interference.

cs702 2y ago
I agree. It seems the edge cases that trip these systems are becoming so rare and convoluted that the return on this kind of research is approaching zero. At some point we just need to collect data from real-world tests.
- dogma1138 2y ago
  Could be, but in this case it may actually have real world implications. LED wall displays are becoming more and more common, I’ve already seen ones that cause annoying reflections on road signs as they turn the retro-reflectors into a disco ball. I’ve skimmed through the paper and I can’t understand how likely this to occur unintentionally in the wild but their strobing pattern doesn’t seem to be that aggressive so a combination of the animation itself on the sign and LED strobing due to PWM control can quite possibly cause similar interference.