7 ms·
Coat makes wearers invisible to AI security cameras
- superchroma 4y agoI am inclined to treat "research" into gimmicks that undermine assumptions made by one particular trained AI as attention seeking spam by this point. There is no through-line from this discovery to a greater outcome, and we just see this article over and over from different groups. Nevermind the fact that authorities do not disclose what AI implementations they use to detect people in footage.
- dpflan 4y agoPretty clearly this clothing needs to be digitized (electrified fabrics or screen-like or curvable/bendable screens somewhere) to actually useful and update-able. Using last year's design could mean this year you get caught... A digitized pendant or pin seems to accomplish the same idea and easy to take with you.
- nomel 4y ago> A digitized pendant or pin seems to accomplish the same idea and easy to take with you. Citation needed, for each neural network.
- constantcrying 4y ago>A digitized pendant or pin seems to accomplish the same idea and easy to take with you. They don't even resolve at the distances the camera detect people, what effect could they possibly have?
- kortex 4y agoI experimented with these sorts of adversarial patterns a few years back. It was straightforward to develop a dazzle pattern which messed with your bog-standard Resnet, but they didn't remotely generalize well. Just using a different architecture or sometimes even a different pre-trained model was enough to thwart it.
- feet 4y agoThat's the thing I don't get, why are people making such a big deal about these adversarial patterns when they might not even work on any models that are actually in use?
- deleted 4y ago[deleted]
- runemadsen 4y agoA much more thorough investigation of these ideas was done by Adam Harvey 11 years ago: https://ahprojects.com/cvdazzle/ https://ahprojects.com/cvdazzle/
- gnicholas 4y agoSeems like it wouldn't take much to update an algorithm so that wearers of these jackets are again identified as people. The next step, of course, would be to flag wearers of these jackets as especially suspicious, since they are trying to evade detection.
- jgalt212 4y agosort of like using the Tor Browser.
- anonporridge 4y agoProbably, but perhaps at the expense of a huge increase in false positives. It's a similar reason we humans tend to get scared of rustling in the night, when the majority of the time there's nothing dangerous out there. Because predators are evolved for stealth and camouflage, our brains overcorrect pattern matching to try to detect them, but most of our "detections" are false positives.
- coder543 4y ago> Probably, but perhaps at the expense of a huge increase in false positives. "Perhaps" is a doing a lot of heavy lifting in your comment, and I find the possible outcome you describe to be very unlikely after looking at the pictures in the article. Personally, I'm confident that these anti-AI patterns don't work consistently across different person detection models, even though the article doesn't even ask the question, let alone dig into the answer. The article doesn't present independent evidence that these work at all, let alone against more than just a single toy model built for PoC purposes. It's an idea that gets clicks, and the oddly-specific "$71" (just an unnecessarily specific conversion from 500 yuan) also helps with attracting clicks. This article is basically just clickbait, in my opinion, not anything substantial.
- falcolas 4y agoThis is based (or independently developed alongside) CV Dazzle, which makes it harder for algorithms to identify "human" features by obscuring the edges they rely upon. The original Dazzle camouflage was effective against human eyes too, so there's a pretty high bar for making an algorithm dazzle-proof.
- brk 4y agoThis wouldn't do anything useful in real-world scenarios. Source: I've been heavily involved in AI security cameras for the last decade+.
- Steltek 4y agoSo... how does this fail and what would be effective these days?
- bfeynman 4y agothese fail for a variety of reasons. The dead simple answer is that people who are trying to sell these are doing it off of a gimmick because to truly do this as the commenter suggested is nearly impossible if you are trying to offer an actual solution. Compression often adds noise that is hard to capture, the patterns can change on model updates, you can easily update models to capture these sequences. It relies on both a dumb model and dumb consumer for this work.
- brk 4y agoIn short, there are too many different kinds of approaches to AI in surveillance cameras to have a universally effective camouflage. In many cases there is simply not enough pixels on target for these patterns to render in a way that makes them at all distinguishable. You also have to account for the fact that you may wind up viewing a person from any range of a 360 arc/angle, so the pattern would need to be around the entire jacket. Most algorithms are looking for more of an overall target size and proportions, plus things like target location relative to an artificial horizon or ground plane. In some cases, this might work to reduce the overall classification confidence, but is unlikely to truly make the person "invisible". Also, thermal cameras are hardly used anymore. They have been stuck at relatively low resolutions (D1 / 640x480), and modern sensors have really good low-light imaging. Because thermal cameras are still very costly, and because they never produce a "good" image with any identifiable detail, they have become really really rarely used overall. Even so, a few patches on a jacket that show small regions of high thermal contrast are unlikely to fool any systems. I doubt that these researchers had access to current state of the art perimeter protection analytics products. The most likely tested on lower end easily available consumer based products. It is hard to say what would be effective overall that is practical. Many systems ultimately fail on people crawling, some will detect this, but often at the trade off of many false alarms, so it is usually not enabled. However, crawling around is not really that practical. Large groups of people moving very closely together are harder to detect, particularly if they are all dressed very similarly. But, I wouldn't call this a reliable evasion technique.
- WheelsAtLarge 4y agoI've heard that an infrared light will make an object invisible to a camera. Anyone know if it's true? That would, surely, make sure it's invisible to the camera and the AI. Note: Got my answer: https://www.wikihow.com/Blind-a-Surveillance-Camera https://www.wikihow.com/Blind-a-Surveillance-Camera It's no very practical.
- mindcrime 4y agoThe thing is, the answer is really "sort of" or even "maybe". Some of it depends on the camera in question. As I recall the genesis of this idea was rooted in the fact that at a certain point in time, many security cameras (especially cheaper ones) had little or no IR filtering, and a bright IR light source could cause a "flare" effect that would effectively mask other parts of the field of view. But it's hard to know in advance if this will or will not apply to any given camera. Also, it might be able to, for example, hide your face or mask your car's license plate... but it doesn't make you invisible. In fact, just the opposite... using this technique makes you acutely visible, but just (if all goes well) unrecognizable. Or if you masked the entire field of vision, you might be effectively "invisible" but it would be obvious to anyone watching the camera output that something weird is happening. So you'd be making yourself conspicuous if there's a live operator watching. So yeah... it does kinda work, but definitely of questionable (but probably non-zero) practicality.
- brk 4y agoActually, all security cameras tends to have an IR filter by default, otherwise colors will be off. Better surveillance cameras will have a "movable cut filter", meaning a mechanism to remove the IR filter from the light path to the sensor to allow for better low-light images. In this mode, the camera reverts to black and white images so you don't get the color shift from the ambient IR light. Using some average 5mm IR LEDs in a flashlight setup during daylight hours would do nothing most of the time. At night you might be able to cause problems with some cheaper cameras, better units with good Wide Dynamic Range specifications would be able to handle most of these kinds of disruptor devices. You'd need some really powerful IR LEDs, like an array of OSRAM IR LEDs (https://ams-osram.com/products/leds/ir-leds https://ams-osram.com/products/leds/ir-leds) to create a strong IR floodlight that would cause the camera to be blown out. Also it is common these days for cameras to send alerts on problems with massive image disruption, so you'd have to hope you're trying to disrupt a very cheap system with nobody receiving event notifications (which is admittedly still very common).
- jameshart 4y agoI'm curious whether $71 is supposed to be 'surprisingly cheap' for a surveillance-defeating measure, or 'surprisingly expensive' for a coat.
- SEJeff 4y agoA really good north face winter jacket (The kind you'll want in somewhere like Canada, Chicago, Northern US, etc) are $200-$350. A women's stylish (non-winter) coat can be $500+. So no, $71 is not that expensive for a coat.
- wskish 4y agoBased on my experience with vision models (my previous company has several thousand models in production), basic CNNs are great at detecting camouflage patterns that they have seen before in training. This type of strategy would have the exact opposite of the intended effect once models were updated with these patterns in the training set. I guess that is an opportunity for subscription "camouflage as a service".
- chickenpotpie 4y agoI think this opens up an interesting cat and mouse game however. Once the model is trained on this shirts, the shirts could just be left around town and all of a sudden everything is a human. Now they have to refine their algorithm even more and the people refine their methods even more and so on and so on until there's too many cases for the model to function effectively.
- coder543 4y agoNo, none of this is how ML models work either, unless you're leaving humans in those shirts lying around town. The model would not be trained against "just shirt" == "human". It would just be more samples from the surveillance cameras of actual humans walking around being labeled properly. (The huge assumption here is that the shirt actually worked in the first place, which would only happen against a specific model, and the article doesn't provide any useful insights into anything.)
- chickenpotpie 4y agoThat's not necessarily true without knowing what model they're using though. If they're doing dimensionality reduction, the model could learn that if the shirt is present, than the human part of the image really isn't important because the presence of the shirt is a 100% accurate indication for people.
- coder543 4y agoPerson detection models draw an outline around each person in frame. What you describe would completely break that, regardless of the model. The model has to keep more of the context or else the bounding rectangles would be all over the place.
- ballenf 4y agoNext step would be programmable patches of IR emitters/absorbers that change in real time during use.
- jkepler 4y agoOr beanie caps that use a IR laser to target and blind surveillance cameras like in Doctorow's "Pirate Cinema" novel.
- jeroenhd 4y agoThere are guides all over the internet (https://urbanarmor.org/portfolio/miss-my-face/ https://urbanarmor.org/portfolio/miss-my-face/ seems rather effective) but I doubt such clothes will gain any popularity any time soon. I'm also wary of the impact all the IR light emitted from some of these powerful LED lights on the human eye; remaining flood-lighted by invisible light, especially late at night, seems rather risky to me. All efforts I can find seem rather amateurish. Very little light diffusion on the IR LEDs, or even entire LED strips wrapped around oneself as a way to get a bigger spread.
- jay_kyburz 4y agoThe final step is we all travel around in the exact same black cube on wheels. Can't get bometrics or your unique gait. Or just not leave home. I've always thought that surveillance footage gathered using our tax money should be available to everybody. I thought if everybody can access them, they would understand how we exposed we all are as soon as you leave the house.
- astrange 4y agoWhy is it surprising that other people can see whose car is whose when that's the point of having a license plate on the back?
- jay_kyburz 4y agoThere is a difference between knowing who owns a car, and knowing exactly where is and has been. Imagine a public database where you can type in a license number and get a log of every sighting of the car in the last few weeks. Somebody has this data, but far as I know its not public.
- meindnoch 4y agoCool! I’m gonna add it to the training set.
- i_like_apis 4y agoIf you like this there is also https://adversarial-designs.shop/ https://adversarial-designs.shop/ They sell mugs that show up as dogs/birds/toasters/nothing ... shirts that are stop signs, stickers that are toasters, etc. Great secret santa gifts or stocking stuffers for ML nerds. No affiliation, I just think it's cool :)
- bogwog 4y agoIf I wear that on the side walk, and confuse a self driving car enough that it causes an accident, does that make me an asshole?
- hoosieree 4y agoChaotic neutral application of Postel's law.
- tlavoie 4y agoPresumably the self-driving car would do a controlled stop for the fake stop sign, so the asshole should be the human driver who rear-ends them for stopping in the middle of the block.
- jeroenhd 4y agoSelf driving cars are still required to have a human at the wheel so I don't see much of a problem. Self driving cars need to be If you get hit yourself, though, don't expect sympathy from the courts or your insurance company.
- mxuribe 4y agoThese seem a tad pricey, but very, very cool!
- constantcrying 4y agoYou can usually beat adversarial examples by training against them, which makes the whole thing a cat and mouse game, which in the end only strengthens the AI.
- pcurve 4y agoI wonder if there would be repercussion from the Chinese government for wearing this. The fact that people would attempt to create something like this and publicize its finding means maybe there's a bit of hope for more free China?
- constantcrying 4y ago>I wonder if there would be repercussion from the Chinese government for wearing this. Only if they are really stupid. Adversarial attacks are easily beatable and if anything this only improves the AI. These attacks exploit the specific structure and training of a neural network, they do not make you "invisible to AI".
- 988747 4y agoWhat you really want for avoiding security cameras is more high-tech solution (and I know I might be crossing into science-fiction territory here): A device that looks like forehead flashlight, but actually has a camera and some computer vision AI (or some other way of detecting security cameras), and a laser beam that it can use to blind those cameras. Or you need something like this: https://futurism.com/the-byte/watch-invisibility-cloak-military-use https://futurism.com/the-byte/watch-invisibility-cloak-milit...
- astrange 4y agoWhy isn't the security guard curious that someone's just blinded their camera?
- coder543 4y agoYou're being downvoted, but I have no idea why. You're completely correct. Anyone who believes this $71 coat will make them "invisible to AI" doesn't know how machine learning works.
- IvyMike 4y agoI feel like we need to give a hat tip to Philip K. Dick's Scramble Suit from "A Scanner Darkly". http://www.technovelgy.com/ct/content.asp?Bnum=997 http://www.technovelgy.com/ct/content.asp?Bnum=997
- deleted 4y ago[deleted]
- wallstprog 4y agoSee also "Zero History" by William Gibson (2010) (although the "invisible t-shirt" idea apparently came from Bruce Sterling).
- jrd259 4y agoSee the scramble suit in Philip K. Dick's A Scanner Darkly
- commieneko 4y agoReminds me of the "ugly t-shirt" in William Gibson's _Zero History_.
- IndySun 4y agoI was surprised by this part of the story... "The coat won first prize in a contest sponsored by Huawei Technologies". The same Huawei between a rock and a hard place (on paper) i.e., much feared in the West, a success story for China. And to evade the AI security of both?
- jay_kyburz 4y agoIf you were making tech to track people, how would you incentivize people to think outside the box and come up with ways to circumnavigate your tech? You run a competition and test your tech against the entries. When your tech fails, know what you need to fix.
- IndySun 4y ago>...run a competition and test your tech... In a typical capitalistic, privately owned and run company, yes.
- rqtwteye 4y agoThis is a losing battle. Basically they are providing training data for the surveillance companies. It's pretty scary that soon we will have more powerful surveillance than Orwell imagined in "1984".
- acdha 4y agoThe fact that research was published openly by Chinese students and favorably recognized by Huawei strongly supports that interpretation. That's more or less the polar opposite of what you'd do if you were trying to subvert mass surveillance systems rather than improve them.
- stuckinhell 4y agoUntil the algorithm gets updated. The AI is evolving faster everyday. I work on some ai adjacent work, and we have translation and drawing ai internal service that are basically multiple AI's working together seamlessly. Ideally to automate the localization of one our products across 150 countries, even it boosts our productivity 20% it's going to be a huge win.
- randcraw 4y agoThese coats should change the pattern they generate every second, like Rorschach from Watchmen. That'd not only be cool, but should guarantee anonymity since the current photo of you doesn't match the last photo of you.
- vorpalhex 4y agoJust need a giant soft e ink display.
- Maursault 4y agoWhat I want is small device I can wear under a ball cap running a ML model trained on detecting pronounced and obscured mounted cameras, with a small reticulating arm, an accurate guidance system and fast-targeting 30W laser. Let them detect all they want with cooked image sensors.
- hypertele-Xii 4y agoYour laser will assuredly shoot people blind in the eyes. Don't do this.
- Maursault 4y agoThat's not how ML works. Trained properly, it could never mistake an eye for a mounted camera. Also, you missed the descriptor, "accurate." Get in on it early. Everyone will be wearing one of these.
- hypertele-Xii 4y agoI can feel a "no true ML fallacy" brewing. 'Properly trained ML' is 'accurate' until it makes a silly mistake and somebody loses an eye. Then it's "still in beta" with Perfect Accurate Function just around the corner, right behind Full Self Driving.
- gershy 4y agoI wonder about the law here: you wear one of these and get hit by a self-driving car - are you at all culpable? :)