7 ms·
Progressive Growing of GANs for Improved Quality, Stability, Variation [video]
- visarga 9y agoGenerated images start at 0:43 in the video. Link to paper here: http://research.nvidia.com/sites/default/files/pubs/2017-10_Progressive-Growing-of//karras2017gan-paper.pdf http://research.nvidia.com/sites/default/files/pubs/2017-10_... Source: NVIDIA
- yeldarb 9y agoI’ve seen this going around twitter and the video looks cool but I don’t really understand what’s going on. Can someone explain it in layman’s terms?
- partycoder 9y agohttps://www.youtube.com/watch?v=Sw9r8CL98N0 https://www.youtube.com/watch?v=Sw9r8CL98N0
- kyleschiller 9y agoNeural Networks are really good at identifying patterns in data. As a classic example, if you wanted to predict housing prices, you could build a data set that maps features about houses (square feet, location, proximity to Caltrain, etc) onto their actual price, and then train a network to recognize the complex relationship between features and pricing. Training happens by feeding the network features, letting it make a guess about the price, and then correcting the guess (backpropagation). Convolutional Neural Networks work similarly, but with images. Instead of giving a CNN discrete features, you'll usually just use the pixels of the image itself. Through a series of layers, the CNN is able to build features itself (traditionally things like edges, corners) and learn patterns in image data. For example, a CNN might be trained on a dataset that maps images onto labels, and learn how to label new images on its own. This video uses Generative Adversarial Networks (GANs) to actually generate new images. In this case, you have two networks "competing" against each other. One network is a traditional CNN trying to identify is an image is "real" or computer generated, and the second network tries to generate new images to trick the first network. We've been able to generative fairly realistic small images before (usually 64x64), but doing it this well on high-resolution (1024x1024) images is unprecedented. http://kvfrans.com/generative-adversial-networks-explained/ http://kvfrans.com/generative-adversial-networks-explained/
- modeless 9y agoA neural network is fed hundreds of thousands of images of celebrity faces. From this it learns the typical characteristics of human faces, and can then be used to draw new faces with random combinations of those characteristics. This work is interesting because it is the first one that has been this successful at high resolution, so the output images are large and detailed rather than postage stamp sized.
- Cacti 9y agoTypically in a neural network, you train a single network against a single loss function that is known in advance. For example, an autoencoder is (usually) a neural network that has a chokepoint somewhere, and is trained to reconstruct the input image. Since there is a chokepoint (a layer that is significantly smaller than the input), it learns to compress the input and reconstruct it. Sort of like a lossy image compression. To train it, and tell how well it does, we can just measure the output against the input (difference between then reconstructed image and original input image). This tells us how well the network does, and gives us a well known loss function we can use in advance. But what if we don't have a loss function? Or we don't know it? (for example, how do we even measure "what makes a face a celebrity-like face?") In that case we can train it against another network that is itself trained to differentiate between a "real" input and a "fake" input. The new network takes an image as input, and outputs a probability that the input is real or fake. We don't know the loss function, but by alternating which batch of images this network gets (fake or real), we can tell how well it does (it should estimate the real oens are real, and the fake ones fake). By training these two networks in tandem, we can use the information from the new network (the discriminator) to tell the old network (the generator) how to generate new, better images. This way we don't really need to know the loss function in advance, between the discriminator serves as our loss function. That is the general idea. In practice, it's fairly non-trivial to get these two networks to work together nicely... often one will get much better than the other, which prevents the other from learning. In this particular paper, they are using a technique to expand the size of the images to much larger than you would normally be able to.
- eltoozero 9y agoA GAN is a Generative Adversarial Network[0], the video is like animated Deep Dream[1] stuff but way more refined. I don't like the horizontal sliding transition because I'm way focused on the bizarre iterations of the various targets. Gonna have to update our camouflage patterns again to combat computer vision... [0]: https://en.m.wikipedia.org/wiki/Generative_adversarial_network https://en.m.wikipedia.org/wiki/Generative_adversarial_netwo... [1]: https://en.m.wikipedia.org/wiki/DeepDream https://en.m.wikipedia.org/wiki/DeepDream
- goombastic 9y agoInteresting. In another 300-500 years, I am pretty sure we will start simulating sensory experience and ultimately the past. I am not sure if I am a toy simulation of the past from the future, right now.
- davesque 9y agoInteresting thought.
- mindcrime 9y agoapropos: https://en.wikipedia.org/wiki/The_Thirteenth_Floor https://en.wikipedia.org/wiki/The_Thirteenth_Floor
- visarga 9y ago> I am not sure if I am a toy simulation of the past from the future, right now. Think about this: what is the only generation that lives in times when massive recording, storage and communication is possible, but was not yet influenced by AI? Ours. When AI will want to simulate a "natural" human society, it has only us as templates to train its humanGAN on. Even our past comments on reddit and Twitter have been used many times to train dialogue systems - we're being uploaded with each post we make.
- has2k1 9y ago> In another 300-500 years instead of "In the future", triggers me. Do you know something we do not? I am resigned to the notion that everything is up for grabs. > I am not sure if I am a toy simulation We need "reality discriminants". The trippy thing is if they could exist and their output is not necessarily boolean. There would be a threshold point at which beings can exist along the simulated-real spectrum, were by they can understand the output of the "reality discriminants", yet they are not real.
- subcosmos 9y agoThe way these GAN algorithms work is precisely by building a network that discriminates reality from fake. That's why they become so good at this!
- v64 9y agoHoly crap that video. It makes me think of the scramble suit from A Scanner Darkly[1] [1]: https://www.youtube.com/watch?v=BWne23FfKW8 https://www.youtube.com/watch?v=BWne23FfKW8
- heroprotagonist 9y agoWell, I guess the propaganda bots will be a little more convincing now that they can have unique profile photos. Would be nice if I could use this to convince Facebook that some fictional image is myself, though.
- kpil 9y agoOh, damn. I've already had enough of fake personas, but now there is virtually no limit.
- asciimo 9y agoLooking at synthetic celebrities is unsettling. They're all familiar. They're obviously celebrities, but it's impossible to remember their names.
- kpil 9y agoExactly as the actual celebrities then... I think i recognized one of them.
- geraltofrivia 9y agoMeanwhile, my Neural translation models don't converge. Sigh.
- duskwuff 9y agoI love how using training data from the Internet has resulted in the GANs believing that a picture of a "cat" often contains text -- or, at least, shapes resembling text -- at the top and bottom of the image. (Visible on the right side of the screen starting around 4:20.)
- dnautics 9y agoI thought the cats looked like abominations.
- pault 9y agoThe interpolating animals look so much like my dreams it's unsettling. I often wake up when things get too "unrealistic" and people or animals are changing shape too rapidly. Whenever I see something come out of a NN that looks like something that came out of my brain I get a little future shock.
- platz 9y agoDid you see the the horror faces it generated for humans? Good for Halloween!
- avaer 9y agoWonder how long until this is co-opted by the porn industry and what the law will have to say about it. Is it illegal to own a digital brain that can think up illegal porn?
- dnautics 9y agoIn the us, child pornography with digital actors has already been ruled to be legal.
- subcosmos 9y agoNote to all: Please don't do this. Lots of missing kids out there and the agents working in this area don't need anymore distractions.
- subcosmos 9y agoLove how this was downvoted. Way to stay sane HN... I don't think that any conversations focused at further promoting child abuse imagery should be encouraged in this community, particularly when such misapplication of these technologies would profoundly derail existing investigations. But fuck it. Downvote away!
- IshKebab 9y agoI would say that it probably is. If you trained a network using illegal data (e.g. cp images) then not only did you have to have that data once, which is of course illegal, but the data itself is at least partially encoded in the network weights which I think should make it illegal.
- yorwba 9y agoI guess you might be able to get around this if you train only on legal content and can interpolate into content that would be illegal if a real recording. However, I'm not sure whether there are any other applications for this specific interpolation scenario that would lead to it being developed, as the effort required to make it work is likely much higher.
- olalonde 9y agoI wonder if something like this could be combined with 3D animation to produce super realistic computer-animated films.
- IshKebab 9y agoNice idea.
- davesque 9y agoSome of the generated celebrity images seem pretty much indistinguishable from reality. Does reality exist any more? In the future, it may get harder and harder to know.
- NTDF9 9y agoThis+VR = porn industry dream (also propaganda industry dream)
- a1371 9y agoThe progress in this field is astonishing. It wasn't even a few years ago that the typical demonstration in a paper on GAN would have been an array of very small images. I recall often seeing numbers between 64x64 and 256x256. At the time, the argument was raised that the resolution can hardly be increased because it is our brain that tries to identify objects and faces in the photos to figure out what they are representing. Then here we are, with indistinguishable 1024x1024 recreations and trippy latent space interpolations. I know not every researcher or entrepreneur has the resources of NVIDIA to train for this many days, but let's not forget, that part needs to occur only once. It makes me wonder about the day that a GAN manages to bankrupt stock photography services.
- rjtavares 9y ago> I know not every researcher or entrepreneur has the resources of NVIDIA to train for this many days, but let's not forget, that part needs to occur only once. It's not like they trained this on a GPU farm. According to the paper [1], they "trained the network on a single NVIDIA Tesla P100 GPU for 20 days". [1] http://research.nvidia.com/sites/default/files/pubs/2017-10_Progressive-Growing-of//karras2017gan-paper.pdf http://research.nvidia.com/sites/default/files/pubs/2017-10_...
- 317070 9y agoYes, but they probably did not get it right on first try. You usually need hundreds of iterations to get it right. This does not imply 2000 days of GPU, but it quickly pushes this outside of reach of consumer hardware.
- a1371 9y agoWhere I live that thing alone is $17,000.
- bitL 9y agoP100 is essentially 1080Ti, so you can grab 1080 Ti and have the same speed. V100 might be the expensive version, up to 10x faster.
- argonaut 9y agoWorth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/924013332549513217 https://twitter.com/nalkalchbrenner/status/92401333254951321....
- moultano 9y agoSeems like you could validate using random points in the latent space, and crudely verifying that the nearest neighbors in image space aren't similar.
- 317070 9y agoThat is true, but if you don't look at them as a machine learning method, but rather as a computer graphics method, then it is quite impressive. It has the added benefit of being allowed to overfit as long as the average human does not find out. If you optimize for psychovisual metrics, GANs are fine.
- tw1010 9y agoBut if you don't see it as a machine learning method, and don't care if the things the GAN spits out are just memorized photos, that means you don't actually care about the synthesis parts of the GAN? Thus the only reason to get excited about it is the interploration stuff; which significantly reduces how interesting it is in my eyes.
- mannigfaltig 9y agoActually, GANs reach state of the art in anomaly/outlier detection and drug/molecule prediction, so there is certainly more to it than just artistic applications: https://openreview.net/forum?id=S1EfylZ0Z https://openreview.net/forum?id=S1EfylZ0Z https://www.ncbi.nlm.nih.gov/pubmed/28703000 https://www.ncbi.nlm.nih.gov/pubmed/28703000 http://pubs.acs.org/doi/abs/10.1021/acs.molpharmaceut.7b00346 http://pubs.acs.org/doi/abs/10.1021/acs.molpharmaceut.7b0034...
- aleju 9y agoFor people interested in the details, I wrote a small summary of the paper: https://github.com/aleju/papers/blob/master/neural-nets/Progressive_Growing_of_GANs.md https://github.com/aleju/papers/blob/master/neural-nets/Prog... (Prior knowledge of GANs is recommended, otherwise its probably hard to understand.)
- stablemap 9y agoThe authors have a GitHub repo containing code and links to the paper and more outputs: https://github.com/tkarras/progressive_growing_of_gans https://github.com/tkarras/progressive_growing_of_gans
- rayuela 9y agoWritten in Theano, of course! R.I.P.