4 ms·

So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the a

So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the algorithm to account for a t-shirt. This is cute, at best. It's also then super easy to say that the individual wearing the shirt is likely to try to usurp monitoring. In practice this type of thing will likely make you a more prevalent target for monitoring along the lines of "what do you have to hide?". Not that I agree at all with large-scale monitoring or think anyone should prove that they don't have something to hide. Only that it paints the target on your back.

jjeaff 7y ago
Ya, I immediately thought of the IlIlIl licence plate xkcd. https://xkcd.com/1105/ https://xkcd.com/1105/
crooked-v 7y ago
> Very, very easy to adjust the algorithm for a t-shirt. The operative point here is not 'a shirt', but a visual pattern that tricks deep learning-style classifiers into wildly misidentifying something. There's no 'very easy' way to counteract that other than retraining on a new dataset or switching entirely away from a deep learning system.
- dTal 7y ago
  Surely all you have to do is train a "trippy shirt" detector, and run it in parallel?
- Kalium 7y ago
  As I understand it, adversarial designs generally work on one specific recognition system. So working around this attack would be very achievable with three or more recognition systems and a consensus check. This particular paper is based around attacking YOLOv2.
  - polishTar 7y ago
    I think these types of adversarial attacks are even easier to foil than that because they're specific to one particular set of weights. Even really really small changes in the training data or model could invalidate the attack if I understand correctly.
    - andrewflnr 7y ago
      I know there has been work in generating adversarial images that work against multiple models. That kind of thing is probably only going to get better, to say nothing of particular sets of weights in a single model.

jjeaff 7y ago

Ya, I immediately thought of the IlIlIl licence plate xkcd. https://xkcd.com/1105/ https://xkcd.com/1105/

crooked-v 7y ago

> Very, very easy to adjust the algorithm for a t-shirt. The operative point here is not 'a shirt', but a visual pattern that tricks deep learning-style classifiers into wildly misidentifying something. There's no 'very easy' way to counteract that other than retraining on a new dataset or switching entirely away from a deep learning system.

dTal 7y ago
Surely all you have to do is train a "trippy shirt" detector, and run it in parallel?
Kalium 7y ago
As I understand it, adversarial designs generally work on one specific recognition system. So working around this attack would be very achievable with three or more recognition systems and a consensus check. This particular paper is based around attacking YOLOv2.
- polishTar 7y ago
  I think these types of adversarial attacks are even easier to foil than that because they're specific to one particular set of weights. Even really really small changes in the training data or model could invalidate the attack if I understand correctly.
  - andrewflnr 7y ago
    I know there has been work in generating adversarial images that work against multiple models. That kind of thing is probably only going to get better, to say nothing of particular sets of weights in a single model.