8 ms·
Making Blurry Faces Photorealistic Goes Only So Far
- owenshen24 6y agoThis seems pretty reasonable; I don't know if anyone was claiming otherwise throughout these upsampling results?
- jszymborski 6y agoUpsampling used to be pitched in papers as a way to add plausible information to images so they don't look so degraded. Recently, however, there's been a lot of work pitching upsampling for "deblurring" faces, which seems like a great way for LEO to run the programme until they hit on a face that most likely looks like a person of interest. Hell, you can even make it explicitly so that the net takes two inputs, the blurred image and a suspects image, and generate a plausible upsample that is similar to the suspect. That sounds like bad faith science that no judge would accept, but perfectly legitimate technologies like DNA testing has historically been abused this way in the courts.
- kevin_thibedeau 6y agoIt's only a matter of time until AI generated sketches are trotted out in court to show someone's "obvious" guilt.
- rasz 6y agothis is not upsampling
- owenshen24 6y agoThat's true; I was too loose with my terms.
- tyingq 6y agoDepends on the blur used, right? Straight pixelation is very lossy, but gaussian is less so.
- qayxc 6y agoNot really. The images aren't pixelated as an effect - it's just a representation of the actual number of pixels taken from the ground truth. You can just as well take these pixels and apply any kind of blur filter - you still wouldn't retain any more information. If you go from say 1k x 1k pixels down to 100 x 100 pixels, you end up with 1% of the original information, no matter what you do.
- contravariant 6y agoI think they may have been thinking of just blurring rather than actually lowering the amount of pixels. A Gaussian blur doesn't really remove as much information as you might expect. Although it's very sensitive to noise.
- qayxc 6y agoBut that's something else entirely then - the very idea of upsampling is based around taking a lowres picture and turning it into a hires version.
- contravariant 6y agoBoth pixelation and blurring are used to anonymize faces, which this very article is about, so I can see the relation.
- not2b 6y agoIt's information theory. No matter what blur technique you use, there's only so much information in the pixels. You can't produce more information by magic, or by technology so advanced that it is indistinguishable from magic (h/t Arthur Clarke).
- tyingq 6y agoWell, I'm suggesting that, for example, pixelation where you change 10x10 blocks of pixels to 1 10x10 block of a single color...is very lossy. A blur of a known algorithm (gaussian, for example), can be somewhat reversed. It's not as lossy.
- speedgoose 6y agoI would say the technology has some issues. For example when you don't include enough black people in your training dataset, probably by simply not thinking about it, it can make the algorithm a bit racist. Example : https://twitter.com/Chicken3gg/status/1274314622447820801 https://twitter.com/Chicken3gg/status/1274314622447820801
- naravara 6y agoFunnily enough, this is also true for humans: https://www.nytimes.com/2015/09/20/nyregion/the-science-behind-they-all-look-alike-to-me.html https://www.nytimes.com/2015/09/20/nyregion/the-science-behi...
- scrooched_moose 6y agoThe Oprah one is probably worse: https://twitter.com/Truttle1/status/1274361095285886982 https://twitter.com/Truttle1/status/1274361095285886982
- 1-6 6y agoThanks for the morning laugh.
- cycrutchfield 6y agoI mean, it's amusing, but the face is not aligned the same as the training data so what would you expect?
- GuiA 6y agoAs a computer scientist? Garbage. As a layperson? Magic, because that’s how these things are marketed by startups and journalists.
- tsumnia 6y agoAgreed, while the algorithm is novel, it is vulnerable to introducing false assumptions about the original image. For example, if you were to use this as a method for determine suspects from low quality security camera footage, you may generate suspect images that are biased or completely false.
- elwell 6y agoWaiting for the CSI episode: "Ok, zoom in... alright, now enhance that... now upsample using AI trained on criminal database... "
- Chazprime 6y agoThey already did that in No Way Out, way back in 1987: https://youtu.be/0zLL_XdqxmQ https://youtu.be/0zLL_XdqxmQ
- stabbles 6y agoInverting a non-invertible operator impossible, study finds.
- jaredtn 6y agoSure the theory is nice, but can I see some empirical evidence? :)
- PragmaticPulp 6y agoEveryone in the ML field understands that ML-assisted upscaling doesn't produce output that accurately represents the original full-resolution image. It produces output that humans perceive to be realistic-looking, or free of traditional upscaling artifacts. While this is obvious to anyone familiar with the technology, it's difficult to explain to casual observers. The image output looks real. It feels like it could be real. It's free of traditional scaling artifacts that would trigger suspicion. Without additional explanation, it's easy to see why casual observers would assume the hallucinated upscaled version is an accurate representation of the original image. Historically, police sketches and blurry surveillance images are obviously low quality enough that people inherently know they're approximations. The problem with these ML hallucinated upscaled images is that they look and feel real enough that they bypass people's suspicions. We can try to present them as "Here's what the suspect might look like", but when they look like a full-resolution photograph, people will simply assume that it's exactly what the suspect looks like.
- acheron9383 6y agoExactly, and then it is up to your defense attorney to somehow dispute that the AI just made up a face, that just happens to look like yours, and then explain it to 12 laymen who likely know nothing of how ML or computers work. The company that develops this tech will want to market themselves as some sort of oracle to the people and LEO. Junk forensic science sticks in courts forever, we should be careful before we assent to more.
- wmf 6y agoOne obvious problem is creating only one output face when there are many possible faces that match the low-res version. A tool that turns a low-res face into, say, 16 maximally-different upscaled versions doesn't suffer from the same level of false certainty.
- notdonspaulding 6y agoDoesn't it? You arbitrarily picked 16 possible photorealistic faces out of a total solution space of what? Millions? Wouldn't the balance of probability be on someone in the general population of humans more closely resembling one of your 16 candidate images than any of your candidates resembling the Ground Truth image? Doesn't the problem get both better and worse as you scale your N up from 16? That is, it would be better because one of your candidates is more likely to match the Ground Truth, but it would be worse because you've also widened your net for catching false positives?
- m3kw9 6y agoTry a grid of 2x2. Is called ambiguity the more you hide it. The idea is so simple on why it wouldn’t work even on trained models.
- robertlagrant 6y ago> For starters, Rudin said, “We kind of proved that you can’t do facial recognition from blurry images because there are so many possibilities. So zoom and enhance, beyond a certain threshold level, cannot possibly exist.” Least needed "proof" ever.
- steerablesafe 6y agoIt's also not a proof.
- _trampeltier 6y agoIt might be less true for pixeled videos. I know from some IR camera companys, they use very small movements from the camera to calculate the picture in a higher resolution. And I think, the first picture from a black hole use also a kind of this technologie
- etaioinshrdlu 6y agoThe increasing politicization of the machine learning field makes me grow tired. It makes me want to work on things without political implications. Something like electrical engineering. Does anyone else feel this way?
- BurningFrog 6y agoSo since this algorithm can't be used to reliably recover faces from blurry photos, it won't be. It can still be useful for entertainment purposes.
- aantix 6y agoSlightly offtopic - who has the best video upscaling algorithm out there?
- hirundo 6y agoInstead of blurring faces in photos of protests, Google Street View, etc., blur them and then upscale them, so they don't have the jarring blur effect but still anonymize.
- ithkuil 6y agoCan such a model be used to compare a real picture of a suspect with a low res picture and produce an objective measure of similarity (i.e. not only raw pixel similarity but a similarity measure that actually takes into account how actual human faces downscale, possibly resilient to minor rotations?)
- ttul 6y agoWith a large enough database of potential faces (see VKontakte), you might be able to use this tool to upscale blurry images and match them to a short list of candidates. Other intelligence could then lead you to the actual person in the blurred image. Scary implications.