3 ms·

I think one of the main reasons GANs are used for images is that the perceptual distance is not well defined, and the discriminator can tease whatever statistic

I think one of the main reasons GANs are used for images is that the perceptual distance is not well defined, and the discriminator can tease whatever statistical difference the real and fake data (e.g. blurriness), thus in practice learning a decent distance function. What perceptual distance function did you have in mind?

sillysaurusx 7y ago
Peter Baylies’ encoder is the best I’ve seen: https://github.com/pbaylies/stylegan-encoder https://github.com/pbaylies/stylegan-encoder (He has a great twitter account too: https://twitter.com/pbaylies https://twitter.com/pbaylies) It’s incredible. It’s far more stable than the projector that ships with stylegan2. I think it was originally Putzer’s work, and pbaylies honed it to perfection. He has a neat “latent estimate” too, where a resnet model predicts a closer latent starting point to an arbitrary photo. This cuts down on fitting time considerably. The theory is, that encoder is so good, you can match faces identically. And in fact, the readme in this submission shows Harry Potter’s face, so it’s certainly good enough to provide a very close match to existing images. Random tangent, but it’s also possible in practice to get close to Dota 2 hero icons: https://twitter.com/theshawwn/status/1182208124117307392?s=21 https://twitter.com/theshawwn/status/1182208124117307392?s=2... Unfortunately in principle it seems like there isn’t a fully automatic solution to do that yet; those were generated by me carefully tuning various latent sliders and using pbaylies’ encoder to coax the face closer to the target. You can see the workflow here: https://twitter.com/theshawwn/status/1185278057063632897?s=21 https://twitter.com/theshawwn/status/1185278057063632897?s=2... Point being, you can get an identical match with the encoder for images it was trained on, or you can get pretty close to arbitrary targets by giving artistic control to humans. Perhaps one day an ML model will learn that mapping automatically. There is some impressive work in that direction floating around. Ultimately, you’re probably right that this might fail to learn smooth interpolations in latent space for one reason or another. But it seems worth implementing and seeing what the failure mode is, if any.

sillysaurusx 7y ago

Peter Baylies’ encoder is the best I’ve seen: https://github.com/pbaylies/stylegan-encoder https://github.com/pbaylies/stylegan-encoder (He has a great twitter account too: https://twitter.com/pbaylies https://twitter.com/pbaylies) It’s incredible. It’s far more stable than the projector that ships with stylegan2. I think it was originally Putzer’s work, and pbaylies honed it to perfection. He has a neat “latent estimate” too, where a resnet model predicts a closer latent starting point to an arbitrary photo. This cuts down on fitting time considerably. The theory is, that encoder is so good, you can match faces identically. And in fact, the readme in this submission shows Harry Potter’s face, so it’s certainly good enough to provide a very close match to existing images. Random tangent, but it’s also possible in practice to get close to Dota 2 hero icons: https://twitter.com/theshawwn/status/1182208124117307392?s=21 https://twitter.com/theshawwn/status/1182208124117307392?s=2... Unfortunately in principle it seems like there isn’t a fully automatic solution to do that yet; those were generated by me carefully tuning various latent sliders and using pbaylies’ encoder to coax the face closer to the target. You can see the workflow here: https://twitter.com/theshawwn/status/1185278057063632897?s=21 https://twitter.com/theshawwn/status/1185278057063632897?s=2... Point being, you can get an identical match with the encoder for images it was trained on, or you can get pretty close to arbitrary targets by giving artistic control to humans. Perhaps one day an ML model will learn that mapping automatically. There is some impressive work in that direction floating around. Ultimately, you’re probably right that this might fail to learn smooth interpolations in latent space for one reason or another. But it seems worth implementing and seeing what the failure mode is, if any.