5 ms·

Colour me skeptical. There are multiple ways to capture features and the shirt may fool one set of algorithms but I highly doubt they'll fool them all.

jadell 7y ago
Like any good security protocol, this wouldn't be the only line of defense. A combination of adversarial clothing, makeup, hair style, and accessories would be used. And constantly evolving, making countermeasures harder. Security is always reactionary, you can't defend against an attack you've never seen before.
- notus 7y ago
  > Security is always reactionary, you can't defend against an attack you've never seen before Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.
  - danShumway 7y ago
    Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail. Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?
    - notus 7y ago
      I think it was implied that I meant heuristics that humans have added themselves. The point of it all is to allow models to make generalizations about things it hasn't seen before. This can be done with a combination of supervised and unsupervised techniques.
      - danShumway 7y ago
        > heuristics that humans have added themselves I don't think this is a meaningful distinction. Who cares whether the new heuristic is being added by a machine or a human? You still need to keep feeding the neural network data to learn from, and it will still choke when it sees novel data that doesn't align with the heuristics it developed. That's the entire reason adversarial AI works. The reason the Trippy T-shirt makes you invisible to some current AI systems is because it exploits the heuristics they've built using data that these systems are unfamiliar with and haven't learned to process yet. If it was possible to build an AI system that could defend against novel attacks, the Trippy T-Shirt wouldn't be able to fool them.
  - jadell 7y ago
    If you train your security on parallel lines, and I come in with circles, I've just defeated your security. Machine learning only learns how to categorize things into predetermined categories. If I come in with a novel category it's never seen before, the best it can do is guess, and most likely, it will be worse at guessing than random chance.
    - notus 7y ago
      Except nobody would train just on parallel lines. They use a wide array of different types of data so the model can make generalizations about things it hasn't seen before. > Machine learning only learns how to categorize things into predetermined categories. This is just one type of machine learning called classification, there are others like regression and clustering which can be combined to create more robust models. Look at the technology behind Cylance's product which identifies files as malicious or not pre-execution. They are not just using classification.
MarkMMullin 7y ago
This has pretty broad applicability across a wide range of algorithms. The common failure mode when the machine fails to recognize the otherwise normal real face and body indicates that the whole face/skeleton relationship has fallen apart. Defeating this is interesting, as we have enough trouble just trying to recognize faces. To add to this "yes these are faces too but they are not faces too" is probably going to drive some researchers to drink. At the end of the day, this is a common flaw in a lot of deep learning systems, they're very brittle.
notus 7y ago
Exactly, this is mostly a gimmick. It works in specific situations, but isn't robust and won't stand the test of time.
undersuit 7y ago
Yeah, it's a static T-shirt, all these systems are just one network update away from a fix. We need dynamic clothing like the SmartShroud from "The Light of Other Days".
aasasd 7y ago
Yup, I have the same problem with DPI circumvention software. We know that it fools more-or-less widely available open and commercial DPI suites. Does it fool systems that are made specifically for deployment on ISPs as mandated by the government? Who the hell knows, aside from those working with the systems. The system may even not shut down your requests, but mark you as a weirdo to keep an eye on.

jadell 7y ago

Like any good security protocol, this wouldn't be the only line of defense. A combination of adversarial clothing, makeup, hair style, and accessories would be used. And constantly evolving, making countermeasures harder. Security is always reactionary, you can't defend against an attack you've never seen before.

notus 7y ago
> Security is always reactionary, you can't defend against an attack you've never seen before Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.
- danShumway 7y ago
  Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail. Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?
  - notus 7y ago
    I think it was implied that I meant heuristics that humans have added themselves. The point of it all is to allow models to make generalizations about things it hasn't seen before. This can be done with a combination of supervised and unsupervised techniques.
    - danShumway 7y ago
      > heuristics that humans have added themselves I don't think this is a meaningful distinction. Who cares whether the new heuristic is being added by a machine or a human? You still need to keep feeding the neural network data to learn from, and it will still choke when it sees novel data that doesn't align with the heuristics it developed. That's the entire reason adversarial AI works. The reason the Trippy T-shirt makes you invisible to some current AI systems is because it exploits the heuristics they've built using data that these systems are unfamiliar with and haven't learned to process yet. If it was possible to build an AI system that could defend against novel attacks, the Trippy T-Shirt wouldn't be able to fool them.
- jadell 7y ago
  If you train your security on parallel lines, and I come in with circles, I've just defeated your security. Machine learning only learns how to categorize things into predetermined categories. If I come in with a novel category it's never seen before, the best it can do is guess, and most likely, it will be worse at guessing than random chance.
  - notus 7y ago
    Except nobody would train just on parallel lines. They use a wide array of different types of data so the model can make generalizations about things it hasn't seen before. > Machine learning only learns how to categorize things into predetermined categories. This is just one type of machine learning called classification, there are others like regression and clustering which can be combined to create more robust models. Look at the technology behind Cylance's product which identifies files as malicious or not pre-execution. They are not just using classification.

MarkMMullin 7y ago

This has pretty broad applicability across a wide range of algorithms. The common failure mode when the machine fails to recognize the otherwise normal real face and body indicates that the whole face/skeleton relationship has fallen apart. Defeating this is interesting, as we have enough trouble just trying to recognize faces. To add to this "yes these are faces too but they are not faces too" is probably going to drive some researchers to drink. At the end of the day, this is a common flaw in a lot of deep learning systems, they're very brittle.

notus 7y ago

Exactly, this is mostly a gimmick. It works in specific situations, but isn't robust and won't stand the test of time.

undersuit 7y ago

Yeah, it's a static T-shirt, all these systems are just one network update away from a fix. We need dynamic clothing like the SmartShroud from "The Light of Other Days".

aasasd 7y ago

Yup, I have the same problem with DPI circumvention software. We know that it fools more-or-less widely available open and commercial DPI suites. Does it fool systems that are made specifically for deployment on ISPs as mandated by the government? Who the hell knows, aside from those working with the systems. The system may even not shut down your requests, but mark you as a weirdo to keep an eye on.