2 ms·
> On a related tangent, this also showed up recently: (NVIDIA’s inpainting). A lot of people I know -- intelligent people who are familiar with machine learning
by dahart 9y ago
> On a related tangent, this also showed up recently: (NVIDIA’s inpainting). A lot of people I know -- intelligent people who are familiar with machine learning and image manipulation -- were confused as to how this approach was "recovering" data.
Realistic image inpainting & synthesis has been going on for decades, so I’d guess the main confusion is due to reading the title of Fortune’s article, rather than the paper’s title “Image Inpainting for Irregular Holes Using Partial Convolutions”. BTW, Kudos to Fortune for actually linking to the paper. I felt like it was pretty obviously inpainting, and suggesting arbitrary training data just from watching the video, so maybe the confusion was from reading the PR title only, and not diving any deeper?
Here’s my favorite inpainting paper, partly because the author is a friend, but also because it’s able to hallucinate written text, which most inpainting algorithms since then haven’t been able to do. It’s not a neural network though, and the training data comes from the single input image itself. http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung.html http://graphics.cs.cmu.edu/people/efros/research/EfrosLeung....
> it’s more and more dangerous... I feel like the same blind spot would be attacked with “seeing in the dark” as well.
It’s possible, yes, but it does depend on what the authors did, how the network was trained, whether they allow reconstruction from pure noise, etc.. I would agree that this paper title is a bit provocative, and suggests assuming the output is realistic. The problem might be the title, and not the technique.
While it is important to understand that NNs are hallucinating output with training data, it’s also a good idea to reflect on the history of analog & digital photography & photoshop, and recall that this slippery slope of danger against fake realism has been warned against multiple times before. There are lots of legitimate uses for inpainting (movies, ads). As someone who’s worked in film, I’m excited about the possibilities that NNs bring in terms of new techniques and reduction of labor.
- sandworm101 9y ago>> I’m excited about the possibilities that NNs bring in terms of new techniques and reduction of labor. Reduction of labour is nice. Elimination of labour is problematic. To photoshop something someone has to actually photoshop it. We don't set our cameras to automatically photoshop things as we take them. The techniques in the OP are dangerous because they could be employed in situations where the photographer doesn't realize. A camera/robot trying to edit something automatically fabricating a false reality in the eyes of humans. Imagine a camera used to capture evidence of crime. You don't want such a thing filling in details on its own.
- TeMPOraL 9y agoIsn't Google Photos / Google Camera App doing that, automatically or semi-automatically? I recall they had a feature that could magically merge multiple photos into one that has a combination of details from the source images.