5 ms·

I think these things are clever, but wonder about adversarial images aka, the turtle that registers as a gun https://www.theverge.com/2017/11/2/16597276/google

I think these things are clever, but wonder about adversarial images aka, the turtle that registers as a gun https://www.theverge.com/2017/11/2/16597276/google-ai-image-attacks-adversarial-turtle-rifle-3d-printed https://www.theverge.com/2017/11/2/16597276/google-ai-image-...

tgb 9y ago
Why is that a concern? Obviously you can fool the system into thinking you have a gun, but I don't think casino's will have any qualms in banning fake-guns.
- mark-r 9y ago
  Based on their description of the tech, you can trigger the system with something that doesn't look anything like a fake gun. It just has to have similar resonances.
  - 24gttghh 9y ago
    I think the false-negatives of such a system, which weren't entirely addressed ("will get better with machine learning"), would be worth more to look into than things not guns being seen as guns.
    - mark-r 9y ago
      False negative and false positive rates are correlated, you can generally trade off one for the other - although I don't know how easy that is with machine learning. The false negative rate doesn't have to be perfect to be a deterrent - would you try to sneak a gun into a casino if you knew you had a 50/50 chance of being caught?
      - 24gttghh 9y ago
        People sneak and accidentally take guns on a plane all the time and don't get caught, but I'm not sure what the 'catch-rate' is for that.
  - clort 9y ago
    The article mentions machine learning. So, it triggers, they search you, they find a thing that is not a weapon, they update their database and comp you something for disturbing you. Of course, they could as tgb suggests just ban anything that triggers their limited tests but that would impact income once word got around
    - pavel_lishin 9y ago
      But until that happens, a casino is consistently getting false negatives. What happens when the trinket for sale as you're exiting Cirque du Soleil looks like a machine gun to the system? Suddenly, you've got dozens of people held up at the entrance. At best, that's a customer service annoyance; at worst, a bad actor could use this to their advantage.
      - nl 9y ago
        Then they put a sign up saying "please remove trinket from pocket". This really doesn't seem like a huge problem.
        apple4ever 9y ago
        Sure it does. Who's going to go because they constantly get stopped?
  - gozur88 9y ago
    Which implies it shouldn't be too hard to sneak in a gun if it doesn't look like a gun to the scanner.
  - tgb 9y ago
    Well, that's a concern (for the profitability of the casino) but isn't what I was replying to. The parent post was suggesting that there could be objects designed to look like guns to this system. I have a hard time imagining why anyone would do that.
ubercore 9y ago
I'm far from an expert, but my understanding was that those kinds of adversarial attacks would be prohibitively difficult without full access to the learning system used by that product. If so, there's a vector if you can hack the company and train your own model to generate the adversarial examples to fool it, but it's not a trivial thing
- chillingeffect 9y ago
  I think the so-called carousel attack would apply. Assuming the penalties for bringing in a gun are low, "oops, I forget, sorry," all you need to do is send in a train of people with different types of attacks and note which make it through.
  - LyndsySimon 9y ago
    > Assuming the penalties for bringing in a gun are low A quick check (ETA: of Nevada law) shows that "gunbuster" signs don't have force of law. Assuming you leave if and when asked, there's absolutely no legal repercussion to ignoring them.
    - wl 9y ago
      This varies from jurisdiction to jurisdiction. It's a crime in Texas for a licensed concealed carrier to carry a gun onto property where there are signs prohibiting the carry of guns. http://codes.findlaw.com/tx/penal-code/penal-sect-30-06.html http://codes.findlaw.com/tx/penal-code/penal-sect-30-06.html
      - LyndsySimon 9y ago
        Of course - I should have specified, I checked Nevada law. That said, Texas law requires very, very specific signage. The last time I was in Austin I saw lots of them, but none that met the legal requirements.
        tombrossman 9y ago

tgb 9y ago

Why is that a concern? Obviously you can fool the system into thinking you have a gun, but I don't think casino's will have any qualms in banning fake-guns.

mark-r 9y ago
Based on their description of the tech, you can trigger the system with something that doesn't look anything like a fake gun. It just has to have similar resonances.
- 24gttghh 9y ago
  I think the false-negatives of such a system, which weren't entirely addressed ("will get better with machine learning"), would be worth more to look into than things not guns being seen as guns.
  - mark-r 9y ago
    False negative and false positive rates are correlated, you can generally trade off one for the other - although I don't know how easy that is with machine learning. The false negative rate doesn't have to be perfect to be a deterrent - would you try to sneak a gun into a casino if you knew you had a 50/50 chance of being caught?
    - 24gttghh 9y ago
      People sneak and accidentally take guns on a plane all the time and don't get caught, but I'm not sure what the 'catch-rate' is for that.
- clort 9y ago
  The article mentions machine learning. So, it triggers, they search you, they find a thing that is not a weapon, they update their database and comp you something for disturbing you. Of course, they could as tgb suggests just ban anything that triggers their limited tests but that would impact income once word got around
  - pavel_lishin 9y ago
    But until that happens, a casino is consistently getting false negatives. What happens when the trinket for sale as you're exiting Cirque du Soleil looks like a machine gun to the system? Suddenly, you've got dozens of people held up at the entrance. At best, that's a customer service annoyance; at worst, a bad actor could use this to their advantage.
    - nl 9y ago
      Then they put a sign up saying "please remove trinket from pocket". This really doesn't seem like a huge problem.
      - apple4ever 9y ago
        Sure it does. Who's going to go because they constantly get stopped?
- gozur88 9y ago
  Which implies it shouldn't be too hard to sneak in a gun if it doesn't look like a gun to the scanner.
- tgb 9y ago
  Well, that's a concern (for the profitability of the casino) but isn't what I was replying to. The parent post was suggesting that there could be objects designed to look like guns to this system. I have a hard time imagining why anyone would do that.

ubercore 9y ago

I'm far from an expert, but my understanding was that those kinds of adversarial attacks would be prohibitively difficult without full access to the learning system used by that product. If so, there's a vector if you can hack the company and train your own model to generate the adversarial examples to fool it, but it's not a trivial thing

chillingeffect 9y ago
I think the so-called carousel attack would apply. Assuming the penalties for bringing in a gun are low, "oops, I forget, sorry," all you need to do is send in a train of people with different types of attacks and note which make it through.
- LyndsySimon 9y ago
  > Assuming the penalties for bringing in a gun are low A quick check (ETA: of Nevada law) shows that "gunbuster" signs don't have force of law. Assuming you leave if and when asked, there's absolutely no legal repercussion to ignoring them.
  - wl 9y ago
    This varies from jurisdiction to jurisdiction. It's a crime in Texas for a licensed concealed carrier to carry a gun onto property where there are signs prohibiting the carry of guns. http://codes.findlaw.com/tx/penal-code/penal-sect-30-06.html http://codes.findlaw.com/tx/penal-code/penal-sect-30-06.html
    - LyndsySimon 9y ago
      Of course - I should have specified, I checked Nevada law. That said, Texas law requires very, very specific signage. The last time I was in Austin I saw lots of them, but none that met the legal requirements.
      - tombrossman 9y ago