9 ms·
Options for Enhancing Video of a Burglary?
My home was burglarized last week. My neighbor's security camera captured two 30s video clips of the individual, one at my front door prior to forcing it, and one with him leaving with a cloth grocery bag containing property he was taking.
The video shows his face, but it's not as clear as I would like. I don't know how to quantify its quality except to say its "not bad" but "not great either."
I would welcome any suggestions for extracting and improving still images from the two clips. I'd prefer to outsource this to someone that has experience in this kind of work.
I don't have a lot of confidence that the local police are going to invest a lot of time and effort in this, only because they are overloaded.
Any recommendations for providers or other actions to take would be appreciated.
- fortran77 7y agoSometimes, if you have multiple shots where the subject hasn't moved much, you can combine them to get accurate subpixel resolution (i.e., resolution higher than the camera has). However, this works best to subjects that are "fixed" like license plates. Here's a "practical" way to do it: https://www.dpreview.com/articles/0727694641/here-s-how-to-pixel-shift-with-any-camera https://www.dpreview.com/articles/0727694641/here-s-how-to-p... And here's an example of a research paper about it: https://arxiv.org/pdf/1801.04590.pdf https://arxiv.org/pdf/1801.04590.pdf The local police will spend no time and effort on it. And any video you produce/enhance will not be able to be used as evidence by them.
- jes 7y agoThank you for this. I don't know much about this topic, but it seems reasonable to think that multiple frames ought to be useful in estimating subpixel values. I agree with your take on the suggestion that that police won't spend time on it, and that any altered images wouldn't be admissible as evidence. They did spend considerable time taking fingerprints in my home. And even the original video is damning. I am hopeful that better quality images can be used to get the name and possible location of the suspect. I'm offering a reward of $1,000 to the first person that gives Kirkland PD information that leads to them interviewing a credible suspect. I doubt that any of our property will be recovered. I'd like to see this individual off the streets. Perhaps someday, he will get treatment that would help him.
- ttul 7y agoEven if the photos can't be used as evidence, they may be useful in identifying a suspect enough that a warrant could be executed to search his apartment or whatever. For commercial crime cases in my city, the police almost never count on the courts to convict people of theft; their primary focus is on getting stuff back to its rightful owners if possible.
- ttul 7y agoThis is what I was thinking as well. Superresolution techniques aren't going to surface information that was not already in the image. However, combining multiple sequential images of the same subject can have this effect. The problem is that I'm sure the subject moves considerably from one image to the next. You will have to isolate his face in each frame and transform the face grabs so that they line up perfectly before the images can be combined by averaging. It's worth a shot!
- CoolGuySteve 7y agoYeah, it would be a pretty good weekend project. Detect faces in each video frame, skew/stretch so that the area of each detection is the same, apply this super resolution technique in OpenCV.
- himinlomax 7y ago> And any video you produce/enhance will not be able to be used as evidence by them. It may not be good evidence in itself, but if the person is recognized, they could be investigated and more evidence found. It's not unlikely they'd be in possession stolen property, that their clothes match the video, that they left cell phone records in the vicinity and so on.
- cj 7y agoI've always assumed those scenes in TV shows where they pull up blurry photos / video and "enhance" them 10-fold was just hollywood. Can this actually be done to any degree of usefulness?
- s_kilk 7y agoCorrect, if the information simply isn't present no amount of trickery will conjure it up.
- himinlomax 7y agoWhile you are correct for a still image, there is more information in a movie. That could potentially be exploited similar to the way a long exposure can cancel noise. This normally requires a static subject, but some AI technique may be able to normalize the position of the objects so that they are amenable to be averaged.
- bordercases 7y agoOr the degree to which entropy gets reduced still isn't sufficient to åfford admissible amounts of certainty on its own.
- SiempreViernes 7y agoWell, these days there is plenty of trickery that will conjure up plausible details to put in place of the information that isn't there. It won't be true information though.
- bordercases 7y agoSometimes enriching possibilities is good enough to break out of an investigative minima but other restrictions prevent you from recapitulating that evidence in your line of thinking. That's why parallel construction is helpful.
- jaclaz 7y agoNot as much as you see on TV, of course, particularly the "zoom in" that you see there, but there are specific enhancers that sometimes do what appear like "miracles". If you want to test one, for "motion blur" (let's say the licence of a car driving away), get this (last freeware version): https://github.com/Y-Vladimir/SmartDeblur/downloads https://github.com/Y-Vladimir/SmartDeblur/downloads And test it on the images here (actual test images from a reknowned Commercial vendor of a suite that is also for videos): https://articles.forensicfocus.com/2014/10/08/can-you-get-that-license-plate/ https://articles.forensicfocus.com/2014/10/08/can-you-get-th...
- nlh 7y agoThis is a totally random shot, but I wonder if Pixelmator Pro's new ML Super Resolution could be of help here: https://www.pixelmator.com/pro/machine-learning/ https://www.pixelmator.com/pro/machine-learning/ You could grab individual frames of the video and run them through this and see what the results look like? (no connection to Pixelmator - just played with the software and it looked pretty darn cool)
- drcode 7y agoThere is a reason Google's initial DNN project was called "deep dream"- It's because it essentially uses "dreaming" to generate new content based on a source database of unrelated images. It is making up new image details (highly convincing and realistic new image details) and hence this tech is completely inappropriate for handling criminal evidence.
- LeifCarrotson 7y agoIt's definitely not appropriate for suggesting that a certain person is in the video. Instead, what it can do is help (or poison) the imagination of a viewer, by inserting what a person matching the few details that are in the image/video would look like with average other details added. Maybe your person has a darker pixel on the one side of their face you see: might be a mole, a face tattoo, or a five-o'clock shadow, or maybe they've just got a bit of mud on their face from falling in the dark. DNN will 'helpfully' provide that large facial moles and tattoos are less common than facial hair in their database, and most of their models wipe mud off their face before having their picture taken, so it will give your suspect a bit of realistic salt-and-pepper growth. Or maybe it has a bunch of interesting pictures of a guy with some facial hair and face tattoos that it can fit to the video. Suddenly you find you've been burglarized by Post Malone. It's only useful if you're trying to get an idea of what an average situation looks like because you can't imagine what you're seeing in the noise.
- chaostheory 7y agoBy your logic, criminal sketches would be deemed inappropriate for criminal investigations as well. They’re both imperfect, but useful at the same time
- keanzu 7y agoWhere does enhancement stop and tampering with evidence (18 U.S.C. § 1519) start? With sufficiently powerful machine learning the video can be deepfaked to have Elvis robbing you. https://www.law.cornell.edu/uscode/text/18/1519 https://www.law.cornell.edu/uscode/text/18/1519
- colejohnson66 7y agoIs it still tampering if you provide the original and the “enhanced” one?
- dordoka 7y agoIt will probably not be considered tampering, but at the same time, it will probably not be accepted as a proof officially.
- brk 7y agoI work in the video surveillance space. The short version is that you are unlikely to be able to do anything worthwhile with those videos. If you want to contact me privately I can take a look at them and maybe offer some specific suggestions.
- jes 7y agoThank you. I would welcome any suggestions you might have. If you'll contact me at john.sambrook@protonmail.com I'll send you the two clips. I don't see how to find your email and I'm reluctant to ask you to disclose it publicly.
- wizzwizz4 7y agoMight want to obfuscate that address a little – perhaps replace the @foo.bar with "on ProtonMail's dotcom domain" or something, if it's worth it to stay off unsophisticated scraper lists.
- dylan604 7y agoIs this worth the trouble? I'm seriously asking. Seems like a ripe arena for a cat&mouse game similar to adblockers. The script kiddies writing email scraping tools surely have regex searching for "[at|@] * dotcom" type of concepts. Maybe I'm too cynical, but I pretty much assume that any email address is "compromised" (as in no longer spam free not pwnd) as soon as it is created. It may not be that bad, but I honestly believe the first time the email address is used as an ID then it is.
- lunchables 7y ago>Maybe I'm too cynical, but I pretty much assume that any email address is "compromised" (as in no longer spam free not pwnd) as soon as it is created. It's not binary, as in, either "compromised" or not. There are lots of different spammers using different methods. Posting your email address on a public website will dramatically increase the amount of spam you get.
- m3kw9 7y agoIf you have the time stamp you could also reference other areas where he could have walked and get more footage. But the faster you go and get these cottage the better as some are not kept after a few days
- sansnomme 7y agoYou might want to consider reaching out to ClearView AI. Ethics aside, they probably can help you identify the burglars. With regards to the comments suggesting super-resolution, just remember that the Second Law of Thermodynamics prevents recovery of information that does not exist. Neural networks and ML can give you a good guess at what the face may look like, but you won't necessarily get more information out of it and if done incorrectly, might even get a false positive. What neural networks are useful for here is to extract the feature vectors from the face and perform facial recognition on them.
- tpmx 7y agoThe key breakthrough here (for the typically crappy footage) will probably involve a) using many frames to ID the perp and perhaps b) working at a native H.264 I/P frame level with the neural networks, rather than at a decoded framebuffer level.
- mgamache 7y agoWhy would the encoded frame be better vs the decided? Deblocking? Other filters? I would think temporal filtering might help? Not an expert here.
- CharlesW 7y agoA company called MotionDSP makes a tool called "Forensic" that costs $250/month: https://www.youtube.com/watch?v=u_B5PUYaIIA https://www.youtube.com/watch?v=u_B5PUYaIIA
- jes 7y agoWow. That seems like a pretty compelling tool. I may license for a month and see what it can do. Thank you for taking the time to help me with this.
- behringer 7y agoI was able to save a group photo with this technique once: https://www.gimp.org/tutorials/Smart_Sharpening/ https://www.gimp.org/tutorials/Smart_Sharpening/
- samstave 7y agoIt would be interesting to save a frame to a modern iPhone, edit it in the photos app AND then apply this technique to the output of the iPhone edited pic. I have been able to get a lot more detail from a photo in a dark setting on the photos editing app on iPhone.
- ravenstine 7y agoThe tools provided by Topaz Labs are incredible for enhancement and have free trials.
- teddyh 7y agoYou’re assuming that getting a clear face view would result in: 1. Identifying the person 2. The authorities finding this person 3. The arrest of this person 4. A conviction of this person and possibly also 5. You getting your things back. In my cynical experience, every single one of these five assumptions is extremely unlikely to actually happen. Note that each one of them depends on all the previous ones.
- bborud 7y agoThat's not what OP asked and probably not the answer he/she really wants to read. This being hacker news I had hoped the top comment would be something that actually might help someone who wants to create better stills from video. I see this on a lot of forums: people not answering the question but feeling that they just have to give their piece of advice that OP is having the wrong problem. Stop it. It isn't helpful. Especially to others who look for answeers to the same question (but perhaps for entirely different reasons).
- teddyh 7y agoThere are already tons of answers here in the comments about solving the problem as stated. I’m just pointing out that solving that exact problem might not be what they actually wanted. Realizing this before you go down a long, arduous, and possibly expensive, path to solve the problem, is valuable. See: https://en.wikipedia.org/wiki/XY_problem https://en.wikipedia.org/wiki/XY_problem
- stronglikedan 7y agoBut this isn't just a Q&A forum. It's also a discussion forum, and OP's post contributed to the discussion. It's good info, and can save the poster a lot of time, headache, and expectation.
- bborud 7y agoThe OP didn’t ask for legal or procedural advice. The question was a rather straight forward technical one. However, it came with context, which was a mistake because it drew focus away from the actual question.
- throwlaplace 7y agoPut contact info in your profile or here so I can contact you. I do research in exactly this area (super resolution for target detection/identification).
- lisk1 7y agoTopaz Labs have some amazing photography enhancing tools that can help. They are claiming they use AI for the enhance. Topaz Studio have this feature ClearAI for me even on standard settings is doing an amazing job to clear the photos. For batch processing you have to use it as a Photoshop plugin , the frames of interest from the videos have to be extracted as separate jpeg photos.
- ubercow13 7y agoUsing AI to enhance CCTV footage for the purpose of identifying criminals seems somewhat unfair to the individuals whose photos were used to train those AI models.
- deleted 7y ago[deleted]
- theincredulousk 7y agowow small world - I saw you post this on Ring. Hello from the greater Seattle area :)
- theincredulousk 7y agoWow small world! I saw you post this on Ring. Hello from the greater Seattle area :)
- spdustin 7y agoI wonder if the “stacking” software often used in amateur astrophotography might help. Or possibly using the Super Scale feature in Blackmagic Design’s DaVinci Resolve?
- c3534l 7y agoYou can't get information from a photograph that isn't in that photograph in the first place. Any software technique that purports to fill in those blanks, like a neural network, is drawing a face in based on what it knows about faces in general. The face you get may look realistically like a face, but it won't look like the face of the person who robbed you, except perhaps by pure coincidence.
- jolmg 7y agoI think the idea is to merge the multiple bad images available from the videos to make one good image. They're not asking to improve the quality of the image using just one image.
- theincredulousk 7y agoVideo frames can be composited to improve clarity the same way it is done in Astrophotography. And your assertion that "any software technique...won't look like the face" is incorrect. Again using multiple frames, it's entirely possible to infer what clearer underlying detail would result in the less-sharp pixel values in the video, given the training data establishing the relationship. It's a specific technique for reconstruction, not just drawing in a realistic but synthetic replacement like Photoshop's content-aware fill.
- katyusha 7y agohttps://www.ghacks.net/2019/10/23/videocleaner-is-an-open-source-video-enhancement-tool-for-forensic-purposes/ https://www.ghacks.net/2019/10/23/videocleaner-is-an-open-so...
- theincredulousk 7y agoNot going to out OP's location by linking, but the actual video was posted in Ring (so publicly available). OP, if you want experts to help, maybe consider just posting the link.
- hackpert 7y agoYou might want to try out some stacking based resolution improvement methods, like the ones used in astronomy. They take data from multiple "sources" and combine them to produce a high resolution picture. For example, check out https://www.astronomie.be/registax/ https://www.astronomie.be/registax/
- akeck 7y agoIt may or may not work, but try applying the below project to a few frames to see what you get. Make sure to preserve the original footage on the original media, though. https://github.com/alexjc/neural-enhance https://github.com/alexjc/neural-enhance
- wizzerking 7y agoOpenCV has a bunch of image processing algorithms that I searched for, but don't see mentioned CLAHE Contrast Limited Histogram Equalization _ I have used to good effect to get better contrast Neural Network DeNoising https://papers.nips.cc/paper/4686-image-denoising-and-inpainting-with-deep-neural-networks.pdf https://papers.nips.cc/paper/4686-image-denoising-and-inpain... https://arxiv.org/pdf/1807.08176.pdf https://arxiv.org/pdf/1807.08176.pdf I'm sure github has some repos on this
- throwaway_tech 7y agoBeyond the video...Depending on what was stolen, if it is likely to be sold, then monitor craigslist, offer up and FB garage/yard sale groups in your area.
- pintxo 7y agoSecond that. Actually got 4 tires/rims back that way (found them on ebay)
- andrewtbham 7y agoThis is not a practical recommendation by any means... but fast ai has a lecture on image restoration with deep learning. https://www.fast.ai/ https://www.fast.ai/ https://youtu.be/9spwoDYwW_I?t=2913 https://youtu.be/9spwoDYwW_I?t=2913
- mackman 7y agoOh god this is a terrible idea when talking about identifying a suspect. You’re basically making an educated guess to construct evidence that could then be used to falsely accuse someone. Please don’t do this.
- andrewtbham 7y agoYeh that is a good point. It would likely be a composite of training data rather than the suspect.
- jes 7y agoHere are links to the two videos: https://www.dropbox.com/s/jkyxlkh0aa5d24m/IMG_5402.mp4?dl=0 https://www.dropbox.com/s/jkyxlkh0aa5d24m/IMG_5402.mp4?dl=0 https://www.dropbox.com/s/oxgwpe6bn39p0re/IMG_5404.mp4?dl=0 https://www.dropbox.com/s/oxgwpe6bn39p0re/IMG_5404.mp4?dl=0
- rs23296008n1 7y agoFirst video shows he touched a few things. Perhaps fingerprints off those? Those clothes would mark that guy depending on locale. Perhaps other cameras saw him as well? Also, now you know your camera isn’t doing the best, add other camera(s) where it would have helped. Eg to capture face-on angle as you approach door. Consider upping the resolution of the cameras you add. On a more general note, rethink about how this guy found your place as a suitable target. Mitigate what you can. He knew he had plenty of time. And he seemed quite comfortable being there.
- throwaway_rock1 7y agoOne observation is that he had a different way of walking, particularly the way his foot landed. Combine this with the fact that he was comfortable being there means that he is from around your place
- microdrum 7y agoEven if you can see his face clearly, how will you find him? It's just a face.
- giarc 7y agoIf police put out the clearest image possible, you still have to rely on those that know him to be willing to identify him. Friends of criminals aren't usually the type to turn each other in.
- piracy1 7y agoYou can enhance the overall image quality using: https://github.com/alexjc/neural-enhance https://github.com/alexjc/neural-enhance on every frame. https://letsenhance.io/ https://letsenhance.io/ For faces: https://www.geoffreylitt.com/2017/06/04/enhance-upscaling-images-with-generative-adversarial-neural-networks.html https://www.geoffreylitt.com/2017/06/04/enhance-upscaling-im...
- solveit 7y agoI would strongly recommend against neural enhancing for identifying faces. There's a good chance the face will get "enhanced" into somebody in the training set that kinda looks similar to the blurry photo.
- mister_hn 7y agoYou can try with ffmpeg to upscale video and improve quality. On GitHub there are some tools that do a pretty decent work upscaling up to 4K
- The_mboga_real 7y agoAny enhanced imagery of our video contact is illegal and good day to you, good sir! But thanks a bunch for your email address, too.
- bilater 7y agoI can try and take a shot at it using my Pic.Hance model
- rotterdamdev 7y agoA better camera next time sadly. I prefer indoor cameras, better light conditions help make better videos.
- jerome-jh 7y agoWith a video player/editor, locate the I-frames (or simply the best looking ones, playing frame by frame), extract them, then use _any_ photo editor to sharpen, adjust contrast/luminosity, enlarge, etc, and print them. It is not rocket science and will give you something to show to the police/whoever, in about an hour of work. Before investing real money on this, be aware of the legal value of these videos: do we really see the burglar forcing the front door? Taking your property? Is your neighbour entitled to film that area? The answer strongly depends on your local laws.