9 ms·
Hey! Sriram here, author of this work. I'd be glad to answer any questions. There's also a short talk I gave about this here (https://www.youtube.com/watch?v=t4
by frizensami 5y ago
Hey! Sriram here, author of this work. I'd be glad to answer any questions. There's also a short talk I gave about this here (https://www.youtube.com/watch?v=t4Txdhlji4k https://www.youtube.com/watch?v=t4Txdhlji4k) if that's helpful!
- starwind 5y agoDo you have any plans release this as an app I could install on my phone?
- frizensami 5y agoYes, we do! There are a couple of user-facing annoyances at the moment, one of which I mentioned in another comment (hacks to align the color and ToF images). Hopefully these API limitations are removed soon, or we find better workarounds.
- Iolaum 5y agoAwesome, if there's a waiting list or a website that we can join or follow to be notified of a release please share.
- frizensami 5y agoSadly no project-specific page yet, but I'll definitely update on my page at https://sriramsami.com/research/ https://sriramsami.com/research/ (I think there's an RSS feed?) when it's active.
- specto 5y agohttps://sriramsami.com/feed.xml https://sriramsami.com/feed.xml
- iamwil 5y agoIt's easy to set up a mailing list. Just sign up for mailchimp, or any one of the many newsletter products like substack or tinyletters, just to collect the email addresses. Not many people regularly read RSS anymore, sadly.
- iamwil 5y agoWhere do we sign up to be notified when this is released?
- jmnicolas 5y agoIf you do release it, is it possible to not depend on Google services please? I am almost a year in my degoogled life!
- frizensami 5y agoI would love to, but I'm not sure how yet without a major re-write. The augmented reality code we completely rely on is part of Google Play Services for AR. I definitely do understand the benefit of degoogling, so maybe it can be a community effort once I opensource it.
- robotresearcher 5y ago> Google Play Services for AR. Google’s branding is awful.
- lrem 5y agoGoogle Play Music All Access -> YouTube Red Man, we're so bad at naming, that we keep joking about it... And trying to satire that with internal codenames (the ones engineers give without adult supervision)... And somehow those often seem better than the external names :(
- bradknowles 5y agoSo does this mean the app will not be available on iPhone?
- frizensami 5y agoIt could also possibly work on iOS, but there's a catch we have not yet explored. We're relying on something called the "confidence" score in the ToF camera API, which has 8 possible values (3 bits) in Android, but from what I last saw, has only 3 possible values total in iOS. It's not clear how this lack of info exposed by the APi will affect accuracy. This is something we'd have to separately test, and ultimately make a wholly different app for if it's feasible on iOS.
- david_allison 5y agoWould you consider open sourcing the app as-is? You've got a community of people who're willing to spend time polishing.
- frizensami 5y agoIt's definitely something I'm considering. The only hard blocker now is a hashmap that checks for our test phones' unique ID and applies a fixed transform to align both cameras. Right now, any other phones will cause an immediate crash. Minimally I think we need to disable this and maybe include a small UI to let users put in the alignment parameters (just a scale + offset).
- frizensami 5y agoAlso I appreciate the interest, thank you. I'm heading back from the conference over the next few days, so I should be more free to take a look at it soon.
- david_allison 5y agoFeel free to shoot me an email (in profile) if you'd like this UI done. No need for anything else other than the source code. It'd be a massive benefit to society to make this widely available.
- raylad 5y agoiOS too?
- mensetmanusman 5y agoGreat work! Do you have a link to where you can buy these types of time of flight lasers/sensors? Curious about the additional hardware cost versus sensitivity.
- frizensami 5y agoThank you! While this work operates on ToF sensors that are already present in smartphones (e.g., Samsung S20+/Ultra), a Microsoft/Azure Kinect should also be a valid option because it has a depth camera as well. It has a higher resolution and bit-depth as well. We initially intended to compare against the Kinect, but it doesn't fit the use case (something that you can have on you at all times). However, it could be a cheap choice for a different kind of deployment (automated hidden camera detection with robots, perhaps?)
- tootie 5y agoSo I had no idea phones have ToF sensors these days. Do most phones have them or only the high-priced flagships? What are the actual intended uses for them?
- frizensami 5y agoHigh priced flagships have them but there are also a few midrange phones like the Huawei P30 Pro that have them too. The trend seems pretty positive towards more ToF sensors in phones, especially because Apple has had them for two iPhone Pros in a row now. They're basically for augmented reality applications because they sense depth. Placing objects at the right size and scale in the augmented view is much easier and more accurate with the ToF sensors, for instance.
- genewitch 5y agoAny phone that has sub 200ms focus times has to have some sort of TOF sensor, right? The Asus Zenfone 2/3 laser, the Huawei monochrome sensor on the honor series, the portrait and/or macro sensors on the Xiaomi mi note 10, whatever the original Zenfone had, and so on. Not all of them are lasers, which might be important.
- 123pie123 5y agoI do like this what technology advancement would be needed to increase the detection rate and reduce the false positives?
- frizensami 5y agoRight, a few things would be very useful: - Increasing the resolution of ToF cameras (right now images are around 320 x 240) --> reflections from hidden cameras can then be more detailed, whereas now it's only 1 or 2 pixels each. - Increasing the bit-depth of ToF images - right now every pixel is only 3 bits (8 colors). It's very hard to differentiate bright hidden camera reflections from everything else, so we had to do a lot of work for that. - API improvements in conjunction with augmented reality libraries, e.g., a) allowing Android devs to enable the flashlight when AR apps are running b) more raw access to the ToF sensor if possible
- debt 5y ago320 x 240, is this the resolution of the depth data provided by the LiDAR sensor?
- frizensami 5y agoYep, exactly.
- pnw 5y agoSamsung started marketing their ISOCELL Vizion 33D camera in 2020 with 640 x 480 resolution. So it's likely we'll see better ToF resolution announced in some phones in the next year or two. Great project btw!
- frizensami 5y agoThank you! Actually, my understanding when I started this project was that I would get a 640 x 480 image (IMX516 sensor). However, I could only get a 320 x 240 image from the sensor through the Android API, so that was a bit of an oddity.
- 5faulker 5y agoThe next project would be creating another device to detect these spy camera detectors
- stinos 5y agoA hidden camera which detects ToF sensors
- filoeleven 5y agoAh yes, like the radar detector wars! > many early "stealth" radar detectors were equipped with a radar-detector-detector-detector circuit, which shuts down the main radar receiver when the detector-detector's signal is sensed, thus preventing detection by such equipment. https://en.m.wikipedia.org/wiki/Radar_detector#Radar_detector_detectors https://en.m.wikipedia.org/wiki/Radar_detector#Radar_detecto...
- pishpash 5y agoWell if the hidden camera can be shut down by turning on a hidden camera detector, then the hidden camera detector becomes a hidden camera jammer! All the better.
- 14 5y agoI was wondering if cameras started using anti-reflective glass would this prevent detection? Some of the glass I’ve looked up say they reflect less than 1% of light. How sensitive is your app could it detect 1% reflections? Thanks
- nomel 5y agoIt's lens + sensor stackup reflection. Camera sensors are much more reflective than a lens.
- frizensami 5y agoYep, the combination is highly reflective. I cited another work (by a physics research group) in a comment that modelled and tested this effect comprehensively.
- 14 5y agoWould you speculate if anti reflective glass on the outside of the camera and a layer of one way tint on the inside that would allow light through but prevent it from reflective back out? I’m just thinking what we will expect in the future to sell the “undetectable spy camera”.
- frizensami 5y agoThat's a great question. I'm not totally sure if this is possible. If it was, I think one-way mirrors would use this technology. For now, it seems like they only work because they only let 50% of the light through + there's a brightness differential between both sides. Perhaps someone more informed about this could chime in.
- mnw21cam 5y agoYeah, there's no magic material that can violate the second law of thermodynamics, because it's the second law of thermodynamics. One-way mirrors are just partial mirrors with less light on one side than the other.
- ticklemyelmo 5y agoCould this be applied to longer-range applications? Concert / theater cameras, camera / binocular observers, counter-sniper detection?
- frizensami 5y agoWhile the broader technique should work for those applications, the platform (smartphone ToF sensors) probably won't. Smartphone ToF sensors have a pretty limited range. We were only able to detect cameras within 1 metre of the smartphone because we're really using the hardware for something it's not intended for. For the applications you suggest, there are some existing military-looking devices out there that use multiple lasers to find sniper scopes, for example. My basic searching shows at least https://www.ldsystems.us/product/sniper-optics-detector/# https://www.ldsystems.us/product/sniper-optics-detector/#, though I'm sure there's more.
- deegles 5y agoWhat are the properties of the cameras that don't get detected? Wouldn't bad actors just use the same app to check if their cameras are detectable at installation time?
- frizensami 5y agoSo far the biggest issue is if the camera is angled oddly away from the user (basically, outside the 20 degree observable FoV). Another issue would be if they manage to install a larger camera that returns larger reflections (which we filter out). Regarding using the app to check, I guess that applies for the existing handheld detectors as well. It's definitely something that intelligent attackers can try to plan for, but we havent tested the adversarial robustness of the system right now. That would be a very interesting direction for us as well.
- frizensami 5y agoBut at least for the larger camera case, we can just increase one of the filtering thresholds for reflection size, which is already doable on the UI
- djmips 5y agoWould your system work against flat cams? https://youtu.be/BdgwO_i5p54 https://youtu.be/BdgwO_i5p54
- frizensami 5y agoThats very interesting, first time hearing of it. Not sure, really depends on what kind of reflection it generates when light hits it, and because of the unique design I'm not sure what that will be.
- megablast 5y agoDoes it work against something that doesn’t exist yet??
- genewitch 5y agoLensless cameras have been around for a long time, the chips and masks that are completely flat is a new idea, I guess. Single pixel cameras are lensless, for instance.
- pishpash 5y agoAt some point if you are light-emitting you're going to be captured. May need to turn yourself into a blackbody, i.e. hide under sheets.