12 ms·
Generating custom photo-realistic faces using AI
- choot 8y agoI am looking for a way to generate photos of me after plastic surgery. The AI should generate a face which will be most beautiful and trusted in all cultures. It will be using my present face and morph things from there according to the limitations. Using least incisions in least risky areas of face. After that we get a surgeon to do perform plastic surgery as per this AI. I am looking for ideas on how to accomplish it.
- deleted 8y ago[deleted]
- VikingCoder 8y agoInteractive demo here: https://www.kaggle.com/summitkwan/tl-gan-demo https://www.kaggle.com/summitkwan/tl-gan-demo
- simlevesque 8y agoHN effect in full effect
- transpy 8y agoIt's very interesting and kinda creepy. The controls behave randomly, if you decrease "age" it might start changing skin color, etc.
- e_ameisen 8y agoActually, you can click on the name of the attribute to "lock" it. That way you can change one without changing the other!
- transpy 8y agoExcellent
- subcosmos 8y agoOh yeah! I wrote a similar algorithm and constantly suffered similar problems. Constraining the latent space can be a major pain. Its hard to visualize but I think a lot of latent features may exist in non-continuous latent "pockets", meaning that the desired direction of the latent vector is dependent on your position in space.
- transpy 8y agoI´m having a blast changing all the parameters. Really uncanney valley feelings. Some women come up very attractive, looking like Jessica Alba. How can I react to a face of a person that don't even exist? On a second thought, I don´t mean that the controls behave randomly. I understand they affect related parts of the system. If you increase "baldness" on a woman's face, it will obviously increase the "male" factor and the "gray hair" factor. I understand that this faces are being generated from a continuous space. Fascinating.
- modeless 8y agoLooking like Jessica Alba is not a coincidence. This model was trained on CelebA, a dataset of Hollywood celebrity face photos. Jessica Alba is almost certainly in the training set.
- subcosmos 8y agoVery very rad! I've got a similar algorithm https://bit.ly/2ELVG50 https://bit.ly/2ELVG50 To use, upload a photo with the #ageme command. This is beta though, so will take a while to return. The other thing this bot does (the #showage command) runs instantly. Example output: https://bit.ly/2PXKTWu https://bit.ly/2PXKTWu Your model seems to produce much higher quality output. I think I played a penalty by putting the users face into latent space, which is a difficult optimization to perform.
- VikingCoder 8y ago(It's not mine, I just found the link in the article)
- IshKebab 8y agoIt would be cool to integrate a face tracker to measure features, then you could learn the latent vectors for things like eye position and then you could drag the eyes around.
- jatsign 8y agoNext workers put out of jobs by AI: Models.
- nprateem 8y agoAlso photographers (especially stock image photographers).
- teddyh 8y agoAs foreseen by Looker (1981), written and directed by Michael Crichton.
- wyldfire 8y agoI remember enjoying this one as a kid. I guess I was too young to have understood the implications of the plot, I can only remember the LOOKER hypnosis gun from this film.
- gambler 8y agoNot sure what this has to do with modeling. On the other hand, this could be useful for generating NPC/character portraits in RPGs.Fat chance this will ever happen, though. No one wants to deal with Python mess of libraries and no one seems to care about putting neural net stuff in self-contained, reusable packages.
- calebh 8y agoUnity has been investing heavily in neural nets for AI. I could envision this also being used for art asset generation. The core TensorFlow API is written in C++, not Python.
- jpmoyn 8y agoThis is really cool. My first thought of using it would be seeing how i look with a beard haha.
- weinzierl 8y agoCan anyone explain how to do that (in theory). I guess one needs to train a model from a dataset that contains a picture of oneself and many images of other people, some with beards (possibly CelebA-HQ). According to the README this would take about 2 weeks on a NVIDIA Tesla V100. Given that we already have a model from CelebA-HQ would it be possible to use that as a pre-trained model and just train it a bit more with an additional image? If possible would that speed things up enough to do it on cheaper hardware in less than a day?
- Geee 8y agoI'm definitely not an expert, but I think the way to go is to use an algorithm that can find a specific face in the latent space. If the latent space has enough dimensions (trained with a big enough set) it should already contain your face. The problem is to find it. Actually I think this could do the job: https://arxiv.org/abs/1702.04782 https://arxiv.org/abs/1702.04782 https://github.com/simoroma/RecoverGANlatentVector https://github.com/simoroma/RecoverGANlatentVector
- summitkwan 8y agoThanks for the reply! I just want to add a little more details. As long as the training dataset is diverse enough, the trained GAN generator should be capable of generating images that are similar to the training dataset. Therefore there is no need to retrain the GAN. However, to "embed" a given face image to the latent space, we need to either use the optimization based method https://arxiv.org/abs/1702.04782 https://arxiv.org/abs/1702.04782 mentioned in "Geee"'s post, or train a encoding network. The encoding network would be much easier to train than the decoding/generator network because we do not need to use adversarial loss. I would love to include this functionality into my model once I got a job offer and have some spare time :)
- jaster 8y agoVery impressive work ! On a side note, the section "A word about ethics" was a welcome addition. I found this: > If we can generate realistic looking faces of any type, what are the implications for our ability to trust in what we see. resonated with another article posted on HN (https://news.ycombinator.com/item?id=18309305 https://news.ycombinator.com/item?id=18309305) talking in part about the ethical impact of engineered addiction. I guess very soon we will be able to generate "super-attractive" (as in "superstimuli") faces for virtual personas, according to targeted demographics and purpose (advertisement, youtube videos for kids, political messages and so on). Rather disconcerting...
- nestorD 8y agoImagine having Tinder's dataset for your target audience and, for each potential customer, generating advertisement with faces designed in order to attract that particular person. That's a disturbing thought.
- heronymus 8y agoThat's not even far fetched, the ever expanding possibilities of targeting individuals through the big data evolution, will clearly lead to some nightmare like this. My hope is that capitalism will be rendered obsolete through cultural and technological progress, before we arrive there. Stuff like this always reminds me on YTCracker's cyberpunk Album "Introducing Neals" "...big data and your love live after this..." https://ytcracker.bandcamp.com/album/introducing-neals https://ytcracker.bandcamp.com/album/introducing-neals
- mamon 8y agoThe problem is that technical progress is much faster than cultural one. Human nature is basically the same it was 2000 years ago: people in power want more power. And while the totalitarism of early 20th century had eventually fallen, the totalitarism powered by constant surveillance, Big Data, Machine Learning has a potential to be so effective we won't be able to escape it. Some elements of that are already implemented in China (think: Great Firewall).
- lossolo 8y agoWell, I didn't know changing skin tone would melt your lip, make you bald and change your gender. https://imgur.com/a/vc5dz0L https://imgur.com/a/vc5dz0L AI magic.
- imbusy111 8y agoThis is gold.
- subcosmos 8y agobeauty alchemy?
- bitL 8y agoCan't wait until such latent-space variable changes will be outlawed...
- femto113 8y agoI think "generated" may be too strong a word for what this system seems to be doing. It feels more like it has found a bunch of averages among a large enough set of similar pictures of some fairly homogenous celebrities. When enough of those averages are put on a gradient along a particular axis you get transformations that seem magically smooth. In the example you gave I think it didn't have enough source photos along that axis. I'll further guess that this works best for celebrities that are generally considered attractive because attractive faces are already averages. [1] https://www.ncbi.nlm.nih.gov/pubmed/15376799 https://www.ncbi.nlm.nih.gov/pubmed/15376799 "Images of faces manipulated to make their shapes closer to the average are perceived as more attractive."
- bitL 8y agoThe point is that the image has been transformed into a latent space encoding and interpolating those latent-space variables creates those mind-blowing effects (imagine just moving from one value to another via a simple convex combination). How are those latent variables constructed and what do they represent (or if they could be described easily) is completely up to non-linear optimization process running on a huge number of dimensions.
- beaconstudios 8y agoI have to wonder - is anyone working on using generative AI to create 3d models for games? Presumably they are but I'm surprised I've not seen more about it because that's a huge use case for this sort of tech. I only tinker with game development for fun but I can see the value in, e.g., having an app that can generate various kinds of models using these feature sliders. Combine that with style transfer to copy an aesthetic from one model to another and that could cut down on the asset creation time for video games considerably.
- ebrewste 8y agoI have no specific info on this, but I can only imagine a lot of work being done here, none of which will be made public until it is close to release and the marketing benefits of showing it outweigh tipping off their competitors.
- cycrutchfield 8y agoI have heard of video game companies experimenting with GANs for texture synthesis. I don’t see why you couldn’t also try to do some simple 3D modeling, although that may be a much harder problem.
- bitL 8y agohttp://3ddl.stanford.edu/CVPR17_Tutorial_Overview.pdf http://3ddl.stanford.edu/CVPR17_Tutorial_Overview.pdf
- kelvin0 8y agoCreative directors and other Artists in video games (my experience) tend to like to 'stamp' the visuals of a game with a certain aesthetic, from their own inspiration. Generating 3D models (offline, not in game) would be a great start, but in the end it's only a 'rough' draft and will likely go through the same amount of work to get to a final 'approved' visual state. Unlikely in the near future, highly probable in the longer term. Imagine writing a script (movie) and having the AI generate the characters ... storytelling to a new level.
- sangnoir 8y ago> I have to wonder - is anyone working on using generative AI to create 3d models for games? It's not AI, but MakeHuman[1] has parametric features for human meshes[2], and you can hit "Random". It also has sliders for age and gender parameters. I last used it a while ago, I'm sure I'm underselling it. 1. http://www.makehumancommunity.org/ http://www.makehumancommunity.org/ 2. https://www.youtube.com/watch?v=iDhb-6FcqeU https://www.youtube.com/watch?v=iDhb-6FcqeU
- chrisco255 8y agoSometimes I think the idea that we live in a simulation is preposterous...and then I see things like this, and wonder.
- samstave 8y agoOccasionally I have the notion we exist in a holographic reality is incredulous...when I am presented with examples such as this, and reconsider.
- deleted 8y ago[deleted]
- _greim_ 8y agoThere's a transition where a quality signal becomes easy-to-fake, during which time humans are "fooled", but then we become sensitized. It's no longer a quality signal and we subsequently refine our aesthetics toward harder-to-fake signals. For example, wood veneer over particle board, tile-patterned linoleum, etc. In light of that, I wonder what will happen to the concept of beauty as a result of this?
- TheRealPomax 8y agoNot sure where you were going with this: your examples are examples of fake-detection in real life. If you show a photograph with wood veneer over particle board, and it's a high quality wood veneer, then no: no one can tell it's not wood unless they're at the shoot and have the opportunity to examine it. Same for linoleum: as long as the photo's lit properly, you can't tell. And the same is true for what's presented here: as "fake photos", once you hit photorealistic, you're done. There is no "until humans start seeing the pixels".
- _greim_ 8y ago> once you hit photorealistic, you're done Maybe once, but not over repeat occurrences. Then you have an arms race between consumers tuning their aesthetic preferences to certain quality signals, producers trying to exploit those preferences without actually delivering quality, consumers re-adjusting their preferences after getting burned, etc. A closer comparison might be dating profile pics, which as I understand dating "shoppers" quickly learn to distrust, at least for certain angles or types of shots. This AI enhancement stuff would presumably cause some rapid evolution in that particular arms race.
- ben_w 8y agoSuch an arms race is likely until either the AI or the customer brains hit their limits. The in-principle limits of a synthetic mind vastly exceeds anything biologically possible — it’s like comparing wolves to hills, both in size and speed, and silicon has the advantage both ways. The only thing keeping us safe is that we don’t yet know how our minds work.
- moneil971 8y agoInteresting but also lends itself to bad actors and faking photos.
- callesgg 8y agoSeam more like it is very good at blending images than creating new ones. Still very cool but not as cool as it initially appeared when I saw it.
- sp332 8y agoCheck out the first image in the Results section. It's not blending between two images, it's taking one image (in the middle row) and changing different characteristics. It's the understanding of what these characteristics are that's interesting. Taking them all together, they have created a "feature space" which is a multidimensional space where every point corresponds to a different kind of face.
- summitkwan 8y agoThank you for the clarification! The latent space is so well behaved that that I can find linear relationship between the latent vector and the feature labels, and then use the regression slop as the "understanding" of the space
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- romwell 8y agoOK, I am not well-versed in machine learning, so I need to ask: How is it really different, in an essential way, from using Eigenfaces[1] for the same task? There's a paper from year 2000 that does exactly that[2]; you can see the results on page 5. Now, the results in the linked article are way more impressive than the Eigenfaces-generated random faces, but from skimming the text, it seems like the principles are the same: dimensionality reduction from a high-dimensional space into a space where the axis are "meaningful" components (in Eigenfaces, the reduction is done using SVD). Eigenfaces is a linear approach; there's also a multilinear version, TensorFaces by Vasilescu et al[3][4]. I wonder if similar quality images can be obtained using this approach. [1]https://en.wikipedia.org/wiki/Eigenface https://en.wikipedia.org/wiki/Eigenface [2]https://link.springer.com/content/pdf/10.3758/BF03207802.pdf https://link.springer.com/content/pdf/10.3758/BF03207802.pdf [3]http://alumni.media.mit.edu/~maov/mica/mica05.pdf http://alumni.media.mit.edu/~maov/mica/mica05.pdf [4]http://web.cs.ucla.edu/~dt/papers/eccv02/eccv02.pdf http://web.cs.ucla.edu/~dt/papers/eccv02/eccv02.pdf [5]https://en.wikipedia.org/wiki/Convex_set https://en.wikipedia.org/wiki/Convex_set
- skinner_ 8y agoIn both cases, there's a low-dimensional latent space, and there's a pixel space of images. In the case of Eigenfaces, the correspondence between the two is linear. That's not a bad approximation, but "make him smile" or "make him bold" is obviously a very nonlinear operation. Eigenfaces approximates it with something linear, and that's just not very good. The justification for all the added nonlinearity in the mapping is that this operation really becomes linear in the latent space.
- mlboss 8y agoAs you said the result is way more impressive. That is the main theme when you try to compare deep learning based approaches to traditional ML approaches especially in unstructured data(images, speech, text). Also, deep learning based approaches don't require too much fiddling with features. Main work involves coming up with NN architecture and loss function.
- quadcore 8y agoDeep learning will do eigenfaces of eigenfaces of... it's eigenfaces all the way down. Actually it's not all the way down, there is only a few hidden layers.
- ninetax 8y agoI wonder if you apply this sort of model to music. Go from a description of a song to a song that fits that description. I know there are APIs with great descriptors of songs out there.
- summitkwan 8y agoThat would be super cool idea and I have thought about it! To deal with music, we may want to use RNN, LSTM or WaveNet instead of GAN as the backbone generator and use my tl-GAN idea to find the "feature" axes for controlled generation. I would love to contribute to such projects.
- golemiprague 8y agook, so where is the porn
- xamuel 8y agoIt's fun to create paradoxical faces by alternating between increasing two contradictory vectors. For example, alternating between +bald, +wavyhair, +bald, +wavyhair, ... eventually seems to produce unsettling faces where it's hard to tell where the hair begins and the skin ends.
- summitkwan 8y agoNice try, you managed to push to the limit of this algorithm!
- zaroth 8y agoI think it's a bit misleading to claim that AI is "generating" these images, as if from whole cloth. Images of real people have been scored along the adjustable metrics, and then as you click the +/- adjustments, the source images are very craftily "blended" to produce a finite spectrum of results. Yes, no output image is itself a "real" human (although I wonder if you tweak the settings just right to exactly match one of the input images if it would just spit it back out?). It does not appear to me that we are seeing an AI that has learned what humans look like and can now generate arbitrary fictional humans. The magic trick is exposed by lossolo's comment below. [1] EDIT: Apparently I'm just agreeing with what femto113 said 3 hours ago down-thread [2] [1] - https://news.ycombinator.com/item?id=18310461 https://news.ycombinator.com/item?id=18310461 [2] - https://news.ycombinator.com/item?id=18311377 https://news.ycombinator.com/item?id=18311377
- thomasahle 8y agoIn the recent Deep Mind paper on Big GANs, they discuss this point and compare their generated images with the most similar images in the training set. They certainly makes it seem like the network has learned something real and can generate new images that are not just blends of previous ones.
- browsercoin 8y agoyeah the progress is scary. it will possible at some point to have our AI generate highly addictive binge worthy tv shows real time. imagine if you could watch breaking bad season 9 for example, Walter White Jr. breaks bad
- Yreval 8y agoThis is an incorrect description of what the algorithm is doing. Two networks--a "generator" and a "discriminator"--play a minimax game where the generator maps random vectors into images, and the discriminator attempts to distinguish between these images and real images of celebrities. When the discriminator is (close to) optimal, image-gradients based on the features it uses to predict are passed to the generator, so that the generator can learn the distribution in feature space that makes up human faces. > Images of real people have been scored along the adjustable metrics, and then as you click the +/- adjustments, the source images are very craftily "blended" to produce a finite spectrum of results. There is no reason to believe the source images are actually embedded in the latent space. The descriptors that you can edit in the gui don't necessarily span an orthogonal basis in that latent space, so some of them are correlated, which is why editing one value can change others. Additionally, there is no a priori reason to believe that the manifold of "human face-like images" of 628x1024 is 512-dimensional, so there are areas of the space that still don't map well to real images. The network's ability to cover this space is limited by the number of unique training images it sees, how long it is trained, and its architecture.
- ChuckMcM 8y agoThose are some really amazing results. I'm wondering when this shows up as an aid to Police sketch artists.
- summitkwan 8y agoThanks for mentioning this! I think other than using this model for enhancing Photoshop, Instagram, etc. Police sketching would be an awesome application of it
- fernly 8y agoI appreciate the author acknowledging -- if belatedly -- the inbuilt essential bias of the training set, > Training on photos of celebrities of Hollywood means that our model will be very good at generating photos of a predominantly white and attractive demographic... If we were to deploy this as an application, we would want to make sure that we have augmented our initial dataset to take into account the diversity of our users. Could this system generate credible faces of people of color? If it has a "gender" axis, could it have a "melanin" adjustment axis? Or various ethnic axes?
- remontoire 8y agoYes
- PeterisP 8y agoYes, iff it's given relevant training data that includes a wider mix of ethnicities.
- leblancfg 8y agoInteresting that the top level layers of that network correlates to what is essentially gene expression. To think of it, seems feasible to train a network with whole genome as input, and 3D scanned morphology as output. Again, top level weights in the network would end up encoding salient gene variances that cause morphological differences. High cost to generate the data, but the first thing I’d do if I was setting up a designer baby startup.
- nkoren 8y agoAmazing work! But I find myself amused and distracted by the spurious correlations. Eg.: eyeglasses are apparently the domain of balding old men, not beautiful young women: https://imgur.com/a/99WUJ6J https://imgur.com/a/99WUJ6J
- red75prime 8y agoDegree of having eyeglasses on doesn't make much sense anyway. So it has to be some spurious correlation for the network to interpolate it anyway.
- Corrado 8y agoOn a related note, I've long been fascinated by how easily we can tell a male from a female, even from a distance. For example, driving around and looking at people walking along the road or following people in the mall, I can tell, with remarkable accuracy, if someone is male or female. Is it the clothes, or gate, or size? Is it hair style or style of shoes? And at what age does this female/male separation begin. Generally, I can't assess new born people very well but by the age of say 5 years old it becomes much easier. This software seems to answer some of these questions by doing things like presenting males with stronger jawlines or females with higher cheekbones. Does this same thing happen with other parts of the body (barring the obvious)? I know that females usually have wider hips and as a result walk differently than males. Are there other examples of differences that we intrinsically know but don't consciously realize? NOTE: I'm speaking in traditional gender roles. Obviously I can't tell if someone is gender dysmorphic from a distance.
- defanor 8y ago> Are there other examples of differences that we intrinsically know but don't consciously realize? Probably everything from https://en.wikipedia.org/wiki/Sex_differences_in_human_physiology https://en.wikipedia.org/wiki/Sex_differences_in_human_physi....
- hoseja 8y agoCould this be used for non-intrusive photo anonymization? Just replace faces with randomly-generated-but-plausibly-looking ones.
- IshKebab 8y agoNice idea! You could do that but this current method has no way of constraining the face to fit with an existing body. I'm sure there's a way to do that though.
- summitkwan 8y agoHi, I am the author of this blog post, feel free to ask me any questions about this work! Thank you for your comments!
- ohiovr 8y agoI'm not impressed with the demo gif I've seen basically the same results from the 90s era software elastic image. (Motion Image morphing)