7 ms·

Figure 7 in the paper [0] shows something really important. The learned adversarial patch is not general at all and has to be held in a specific place with a sp

by rtkwe 8y ago

Figure 7 in the paper [0] shows something really important. The learned adversarial patch is not general at all and has to be held in a specific place with a specific orientation to function. This means it's not going to be generalizable to actual surveillance where you'll be viewed from a lot of different potential angles or say onto a shirt or jacket you can just wear. [0] https://arxiv.org/pdf/1904.08653.pdf https://arxiv.org/pdf/1904.08653.pdf

HeWhoLurksLate 8y ago
Well, I guess that that would be the next step. Another problem is that the minute that AI-proof clothing goes mass market is the minute it stops working (and for that reason, I suppose that you wouldn't want to have a product image, either.)
- rtkwe 8y ago
  I'm not 100% sure a static image on cloth will ever work well for that kind of general countermeasure. So far all adversarial stuff except the dazzle camouflage face paint require pretty precise position or view relative to the object being labeled.
- NotPaidToPost 8y ago
  The minute clothing that is really effective against AI video surveillance is available when AI video surveillance is widespread is the minute it will become a criminal offence to sell or wear such clothing...
- erikpukinskis 8y ago
  Here's a better question: Will the book, "How To Make AI-proof Clothing" stop working the minute it is published, or will it teach generalizable skills. It would seem unfair to pit a static object (prefab clothes) against a dynamic opponent (person detection AI). Better to compare two intelligent opponents.
- mkagenius 8y ago
  > AI-proof clothing goes mass market is the minute it stops working What prevents GAN's from learning a new patch for them ?
  - HeWhoLurksLate 8y ago
    That's the point.
pizza 8y ago
CV Dazzle [0] is/was a style of makeup that defeated facial recognition, and it seemed to work from multiple angles. Unfortunately it was released something like 5 years ago so I'm certain it won't withstand today's state of the art techniques. Looks like it was never really tested to work against deep learning-based techniques. That said, I think somebody could very easily come up with an adversarial makeup camouflage recommender. Imagine an interactive visualization of the regions of confidence of a facial recommendation system, hooked up to your webcam. You could have a visual overlay of where to apply make up to most damage the classifier's confidence of your identity. [0] https://cvdazzle.com/ https://cvdazzle.com/
- Zaphods 8y ago
  That CV Dazzle project is fascinating. The link to the warship Dazzle camouflage is also hilarious in the sense that that is what people have to turn to in 2019. It reminds me of the comic book The Private Eye by Brian K Vaughn. In that story the cloud "bursts" and online privacy evaporates overnight so everyone turns to intense camouflage to protect their identity. (https://en.wikipedia.org/wiki/The_Private_Eye https://en.wikipedia.org/wiki/The_Private_Eye)
- RugnirViking 8y ago
  On the website it talks about openCV and how to fool that, as well as the general techniques used. I still think from the tips shown that it would be very effective today. For exaple, it talks about how the most distinct features of a face are the nose and eyes, and that hiding or obscuring them works well. Newer technology likely gets much better at spotting noses and eyes, but taking them entirely out of the picture probably helps. Also the asymmetry I can imagine helps a lot.
- rtkwe 8y ago
  Does face detection today use the same kind of trained neural nets that YoLo or others use? Watching this video [0] from their site it seems it's using a much more static algorithm trying to detect particular features which is what the CV Dazzle makeup aims to disrupt. As far as I understand one of the benefits of the neural net models is that they're less reliant on defined features than the older CV models. [0] https://vimeo.com/12774628 https://vimeo.com/12774628
weaklearner 8y ago
eh some adversarial inputs are very robust to perturbations/orientation. see the famous turtle video.
- rtkwe 8y ago
  That was pretty impressive though it's still on a rigid surface which gives you a fixed relationship between the shape and the pattern.
  - weaklearner 8y ago
    that's a good point about the rigid surface.
mcv 8y ago
From what I understand, all of these methods to fool image recognition systems have to be very specifically tailored to a very specific image recognition system, which makes them completely useless for any kind of practical use. To really avoid image recognition, you need to create patterns that disrupt the actual patterns that these systems look for. For facial recognition, make your eyes and jawline hard to find, like Dazzle makeup does. To defeat full-body recognition systems, wear clothes with heavy patterns that make it hard to distinguish where your arms, legs and head are. Even then, it's unlikely to work from all angles and against all backgrounds. You'd need something that adapts to the background you're likely to be seen against.
- colordrops 8y ago
  And they will also use gait detection and cross reference with other data. It's a lost cause.
  - wang_li 8y ago
    Put on a burka and ride a hoover board.
    - heck 8y ago
      Had to create account to upvote this one :)
    - bdamm 8y ago
      Yeah, it seems Islam tradition solved identity recognition ages ago. It certainly makes business negotiations difficult; the burka wearer has the advantage of emitting no body language. An advantage if they are being pitched to, that is.
    - frizer 8y ago
      And camouflage coloured burka.
  - mcv 8y ago
    To defeat gait detection, I suspect the only options are really long, wide skirts, or a pebble in your shoe.
317070 8y ago
It is already reasonably robust though: https://www.youtube.com/watch?v=MIbFvK2S9g8 https://www.youtube.com/watch?v=MIbFvK2S9g8 It is not magic, but I can imagine the method being extended to be more robust to rotation/shearing (by improving the metric being optimized for).
- Krasnol 8y ago
  I can't believe he didn't move it higher or lower!!
  - rtkwe 8y ago
    He did once but it broken the patch and YoLo correctly labelled his torso so he put it back down. They say directly in the paper that it's quite sensitive to the position of the patch relative to the bounding box of the detected object.
- rtkwe 8y ago
  I wouldn't call having to keep it mostly level and centered at hip height robust in the slightest. Check out around 0:35 in that video where he starts moving it around once he rotates it just a little out of orientation (both directly towards the camera which seems very important and also the slight <20 degree rotation) that breaks the patch and YoLo figures out it's a person. Or again at 0:55 approx where it's moved slightly to the side it again breaks pretty immediately.
asdfasgasdgasdg 8y ago
More to the point, you won't have access to the actual surveillance network's AI, so you won't be able to train your patch adversarially in a reasonable amount of time.

by rtkwe 8y ago

HeWhoLurksLate 8y ago

Well, I guess that that would be the next step. Another problem is that the minute that AI-proof clothing goes mass market is the minute it stops working (and for that reason, I suppose that you wouldn't want to have a product image, either.)

rtkwe 8y ago
I'm not 100% sure a static image on cloth will ever work well for that kind of general countermeasure. So far all adversarial stuff except the dazzle camouflage face paint require pretty precise position or view relative to the object being labeled.
NotPaidToPost 8y ago
The minute clothing that is really effective against AI video surveillance is available when AI video surveillance is widespread is the minute it will become a criminal offence to sell or wear such clothing...
erikpukinskis 8y ago
Here's a better question: Will the book, "How To Make AI-proof Clothing" stop working the minute it is published, or will it teach generalizable skills. It would seem unfair to pit a static object (prefab clothes) against a dynamic opponent (person detection AI). Better to compare two intelligent opponents.
mkagenius 8y ago
> AI-proof clothing goes mass market is the minute it stops working What prevents GAN's from learning a new patch for them ?
- HeWhoLurksLate 8y ago
  That's the point.

pizza 8y ago

CV Dazzle [0] is/was a style of makeup that defeated facial recognition, and it seemed to work from multiple angles. Unfortunately it was released something like 5 years ago so I'm certain it won't withstand today's state of the art techniques. Looks like it was never really tested to work against deep learning-based techniques. That said, I think somebody could very easily come up with an adversarial makeup camouflage recommender. Imagine an interactive visualization of the regions of confidence of a facial recommendation system, hooked up to your webcam. You could have a visual overlay of where to apply make up to most damage the classifier's confidence of your identity. [0] https://cvdazzle.com/ https://cvdazzle.com/

Zaphods 8y ago
That CV Dazzle project is fascinating. The link to the warship Dazzle camouflage is also hilarious in the sense that that is what people have to turn to in 2019. It reminds me of the comic book The Private Eye by Brian K Vaughn. In that story the cloud "bursts" and online privacy evaporates overnight so everyone turns to intense camouflage to protect their identity. (https://en.wikipedia.org/wiki/The_Private_Eye https://en.wikipedia.org/wiki/The_Private_Eye)
RugnirViking 8y ago
On the website it talks about openCV and how to fool that, as well as the general techniques used. I still think from the tips shown that it would be very effective today. For exaple, it talks about how the most distinct features of a face are the nose and eyes, and that hiding or obscuring them works well. Newer technology likely gets much better at spotting noses and eyes, but taking them entirely out of the picture probably helps. Also the asymmetry I can imagine helps a lot.
rtkwe 8y ago
Does face detection today use the same kind of trained neural nets that YoLo or others use? Watching this video [0] from their site it seems it's using a much more static algorithm trying to detect particular features which is what the CV Dazzle makeup aims to disrupt. As far as I understand one of the benefits of the neural net models is that they're less reliant on defined features than the older CV models. [0] https://vimeo.com/12774628 https://vimeo.com/12774628

weaklearner 8y ago

eh some adversarial inputs are very robust to perturbations/orientation. see the famous turtle video.

rtkwe 8y ago
That was pretty impressive though it's still on a rigid surface which gives you a fixed relationship between the shape and the pattern.
- weaklearner 8y ago
  that's a good point about the rigid surface.

mcv 8y ago

From what I understand, all of these methods to fool image recognition systems have to be very specifically tailored to a very specific image recognition system, which makes them completely useless for any kind of practical use. To really avoid image recognition, you need to create patterns that disrupt the actual patterns that these systems look for. For facial recognition, make your eyes and jawline hard to find, like Dazzle makeup does. To defeat full-body recognition systems, wear clothes with heavy patterns that make it hard to distinguish where your arms, legs and head are. Even then, it's unlikely to work from all angles and against all backgrounds. You'd need something that adapts to the background you're likely to be seen against.

colordrops 8y ago
And they will also use gait detection and cross reference with other data. It's a lost cause.
- wang_li 8y ago
  Put on a burka and ride a hoover board.
  - heck 8y ago
    Had to create account to upvote this one :)
  - bdamm 8y ago
    Yeah, it seems Islam tradition solved identity recognition ages ago. It certainly makes business negotiations difficult; the burka wearer has the advantage of emitting no body language. An advantage if they are being pitched to, that is.
  - frizer 8y ago
    And camouflage coloured burka.
- mcv 8y ago
  To defeat gait detection, I suspect the only options are really long, wide skirts, or a pebble in your shoe.

317070 8y ago

It is already reasonably robust though: https://www.youtube.com/watch?v=MIbFvK2S9g8 https://www.youtube.com/watch?v=MIbFvK2S9g8 It is not magic, but I can imagine the method being extended to be more robust to rotation/shearing (by improving the metric being optimized for).

Krasnol 8y ago
I can't believe he didn't move it higher or lower!!
- rtkwe 8y ago
  He did once but it broken the patch and YoLo correctly labelled his torso so he put it back down. They say directly in the paper that it's quite sensitive to the position of the patch relative to the bounding box of the detected object.
rtkwe 8y ago
I wouldn't call having to keep it mostly level and centered at hip height robust in the slightest. Check out around 0:35 in that video where he starts moving it around once he rotates it just a little out of orientation (both directly towards the camera which seems very important and also the slight <20 degree rotation) that breaks the patch and YoLo figures out it's a person. Or again at 0:55 approx where it's moved slightly to the side it again breaks pretty immediately.

asdfasgasdgasdg 8y ago

More to the point, you won't have access to the actual surveillance network's AI, so you won't be able to train your patch adversarially in a reasonable amount of time.