4 ms·
GAN Prior Embedded Network for Blind Face Restoration in the Wild
- fabbari 5y agoThis is not the first time I see "(not our best model due to commercial issues)" in a model release. Am I right thinking the the fully trained model may have been sponsored by a provate entity and it's being kept for a commercial release?
- sillysaurusx 5y agoTo be completely candid, the situation around model release is so fucked that it’s anyone’s guess. The logic never holds up. “Ah yes, if we release our best model, then maybe someone will come build an entire business around it and out-compete us.” OpenAI started the trend by refusing to release GPT-3. But GPT-3 makes millions, so at least they can say “hey, that’d be like giving away X million.” It’s all so tiring and depressing. I usually have something positive to say, but my original hypothesis of “npm for ML” seems like a decade away. The culture here is starting to look alarmingly like closed-source gamedev rather than open-source unix. “How could we possibly justify releasing our code? We wrote it. We paid N developers $X, so that would be like giving away $Y. Plus someone might take the code and build a business and out compete us…” But I’ll never have that mindset. Subreddit simulator was the only model I never released, and it was because I forgot to check with the creator whether he’d be cool with releasing the model after I trained it. He had different concerns, but I can’t remember them clearly so I don’t want to paraphrase. But I clearly remember they were bogus in exactly the same way, just a different domain (cultural vs commercial). But hey, at least they released “not our best model.” That’s something. It is indeed better than “no model at all.”
- robertlagrant 5y agoFully agree. Walled gardens are springing up. On the other hand, it took the software industry a long time to go from the early days of mainframe programming to the wonder of apt-get. Perhaps we're being a bit impatient.
- deleted 5y ago[deleted]
- kbaker 5y agoWould really like to see some original pictures, blur, then the model output. Pretty impressive results but it seems like it might suffer heavily from overfitting. Or maybe that's the point. I always come back to this article: https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-added-ryan-goslings-face-to-this-photo/ https://petapixel.com/2020/08/17/gigapixel-ai-accidentally-a...
- sdflhasjd 5y agoI'm not too fond of the term "restoration" because it implies that it's recovered the original face and not just generated one that looks similar. This is one of the reasons I don't want the police using facial recognition.
- robertlagrant 5y agoEntirely agree. This is where some of the AI "controversy" has come from. A blurred Obama photo got "restored" into a white guy's face, which was never going to be understood well by the general public. The nomenclature needs to improve for this sort of thing to make sense, and it not be used for the wrong purposes.
- fpgaminer 5y agoThe term "restoration" is already used to refer to the restoration of artworks. Oftentimes that work can't recover the original, and various fixes and approximations must be made. Yet the term restoration is still used, because that's what one is doing. There are, though, plenty of terms of art (pun intended) for _how_ a painting is restored and modern restoration involves many tricks to ensure that those changes are both non-destructive and reversible. Perhaps digital restoration will begin to adopt similar terms of art for succinctly describing how an image has been restored. Lucky for us, digital images can easily carry metadata that describe what's been done to them, including perhaps a copy or hash of the original image. Side note: https://www.youtube.com/channel/UCvZe6ZCbF9xgbbbdkiodPKQ https://www.youtube.com/channel/UCvZe6ZCbF9xgbbbdkiodPKQ is a great channel for learning about modern art restoration.
- Sanzig 5y ago
- coolspot 5y ago> Alibaba > Blurred face reconstruction So many papers on pose and face reconstruction appear to be coming from China. Are they developing that for mass surveillance and to identify every single Uighur in CCTV footage?
- JulianMorrison 5y agoIf they do, they will end up identifying everyone as "generic Uighur AI created face #1". A digital faceplant.
- yorwba 5y agoAlibaba uses facial recognition for their "pay with your face" feature. Though I doubt this particular GAN is relevant for that. Most likely just some researchers playing around and their employers let them in case they come up with something useful as a side effect.
- anitil 5y agoThere was some controversy here (Sydney) when it came out that one of our local universities (UTS) had been working on facial recognition that was then used for exactly this purpose [0]. [0] https://www.abc.net.au/news/2019-07-16/australian-unis-to-review-links-to-chinese-surveillance-tech/11309598 https://www.abc.net.au/news/2019-07-16/australian-unis-to-re...
- JulianMorrison 5y agoIt's worth remembering that this isn't restoration. This is imagination. It's the construction of an imagined face that fits the blur. You literally can't get more information than the blur itself encodes. So everything more specific, is a plausible lie.
- sillysaurusx 5y agoPhotographs themselves are plausible lies. They’re no more real than this. If that sounds absurd, remember how much information about the world is lost when projected into 2D RGB. For example, one could imagine arguing vehemently “… but you can’t see depth anymore! You can’t possibly understand their facial geometry from a 2D projection. Color is totally different; that’s not how our brains work.” The lie is that we feel photographs are more real than the plausible lies our minds tell us each day. That’s reductive. But it’s not merely a dumb philosophical debate. I believe the outputs here are more real than the blurry source images. So real that you can imagine having a conversation with them. People are so focused on the authoritarian abuse of power / dystopia angle that I think this is often overlooked. If these are lies, are you sure the truth is useful? Depends what you use it for. But I’m aiming for “star trek holodeck.” We can do it; all the pieces are there. This is a step towards it, and everyone on the Enterprise understood that the holodeck was delightful fiction. I’m bearish about AGI, so I’m surprised to hear myself say this: I give a >10% chance that within our lifetimes, someone might fall in love with one of these convincing simulations. Legitimate, actual love, as real as your love for your SO. People will cry their eyes out if these lies are deleted, since it will feel to them like someone died. I think it’s best to get comfortable with those ideas sooner than later. Because from where I’m sitting, that paragraph seems a lot less fictitious than a Mars colony, at least over the next 30 years.
- pedalpete 5y agoWell, your memory is also more fungible than you think. Photographs at least capture a realistic portrayal of what the lens saw. Your mind adds and removes features without your knowledge.
- deleted 5y ago[deleted]
- abeppu 5y agoThese results are good, but looking at them only reinforces my belief that really good generation of images of real complex subjects requires more than training on images (e.g. relationships between images and 3d structure, and understanding relationships between parts of structures, and how items rather than image parts co-occur). E.g. in the first example, where her collar bone should be, are we looking at a garment or skin? We know those are not the same; the model does not. In the second example, the collar of the boy's garment looks odd -- and perhaps like it would be challenging to put on/take off. The model doesn't have an understanding of what a garment is, how it relates physically to the wearer, etc. Glasses should be symmetric most of the time. A light patch on one side of a person's head is perhaps more likely to be due to lighting conditions than really asymmetric and uneven white hairs. These are all things that a human knows from operating in a world full of humans wearing stuff, moving in 3d spaces, etc.