4 ms·
How can it possibly know what was in the white areas of the library? Is there a residual image? Seems impossible that it guesses correctly.
by BatFastard 9y ago
How can it possibly know what was in the white areas of the library? Is there a residual image?
Seems impossible that it guesses correctly.
- deleted 9y ago[deleted]
- azeirah 9y agoIt doesn't guess "correctly" at all. Zoom in on the image, and focus on the filled-in areas, they look really blurry. It just doesn't look very bad from a birds-eye view.
- eutectic 9y agoProbably this is a consequence of least-squares training; the 'average' solution is extremely unlikely.
- knolan 9y agoSomewhat similar to content aware fill in Photoshop [0]. The untrained network can latch onto frequent patterns and match them to holes in the data. Why doesn’t it paint everything white? Are these actually transparent images or are they somehow tagged? [0]https://helpx.adobe.com/photoshop/using/content-aware-patch-move.html https://helpx.adobe.com/photoshop/using/content-aware-patch-...
- jmmcd 9y agoYes, for the inpainting, the parts to be painted (big white deleted areas) are supplied as masks, so it doesn't try to match them.
- BatFastard 9y agoBut its generating unique content in those areas...
- jmmcd 9y agoYes, every "run" of the network is generating pixels in those areas, but they're not being compared against the white (deleted) pixels. On the final run of the network, they're still not being compared against anything, except by us, visually, when we look at those pixels.
- theoh 9y agoIt seems like it should be similar in capability to a wavelet-based approach to e.g. image inpainting. In other words the neural network architecture essentially establishes a basis of features onto which the image is projected. Gabor wavelets are known to occur in the human visual system as the 'image elements' in a similar setup -- Gabor wavelets are kind of optimal, but clearly the features that this nn architecture uses are pretty effective too.
- alexlarsson 9y agoIt cannot know, but it can extend what’s around it to something that “looks believable”. I don’t fundamentally think it is different from asking an artist to complete the image. You’ll get an image, but no actual information about what was under the white.
- nametube 9y agoIt doesn't it repeats/generates pixels that minimize E, so the behaviour heavily depends on the choice of E. for inpainting E is the difference between their generated x and the true x but they mask out the error in the white areas. The model gets good results by just copying nearby pixels.
- Zedmor 9y agobecause your brain runs on same CNN software i.e. doing pattern matching, not image matching.