10 ms·
How an A.I. ‘Cat-And-Mouse Game’ Generates Believable Fake Photos
- ducampopinus 9y agoGreat overview. I especially appreciate that they linked straight to the paper instead of a popsci/buzzfeed regurgitation of the results.
- minimaxir 9y agoFor those looking for a well-commented you-don't-need-a-PhD-to-understand implementation of GANs + variants (using Keras), I recommend the examples in this repo: https://github.com/eriklindernoren/Keras-GAN https://github.com/eriklindernoren/Keras-GAN
- blaze33 9y agoThe paper was published in October and is named: Progressive Growing of GANs for Improved Quality, Stability, and Variation http://research.nvidia.com/publication/2017-10_Progressive-Growing-of http://research.nvidia.com/publication/2017-10_Progressive-G...
- visarga 9y agoAnd the one hour movie of celeb faces generated by the Progressive GAN (ProGAN) is here. https://www.youtube.com/watch?v=36lE9tV9vm0 https://www.youtube.com/watch?v=36lE9tV9vm0 The most amazing AI video I have ever seen, actually. I spent hours staring into it - it works great as background for many pieces of music, you can think of it as the AI version of the burning log video.
- coldcode 9y agoScary to think where this will be in 10 years. Perhaps even video evidence will be hard to believe any more. How do you convict someone if this technology is mature?
- eat_veggies 9y agoThere was a post here a few days ago on amateurs (people who had never even touched python before, let alone tensorflow) who were using deep learning to generate fake celebrity porn. The results are actually pretty believable: https://news.ycombinator.com/item?id=16040463 https://news.ycombinator.com/item?id=16040463
- i_cant_speel 9y agoIs this safe to view at work?
- XOPJ 9y agoYes, first link is to a 4 day old post on HN, which links out to another SFW writeup. The title of the article (AI-assisted fake porn are being used by people on Reddit for self-completion) is worth keeping off the monitor if you don't want someone to quickly scan the word 'porn' on your monitor, but there are no NSFW photos or content.
- adventured 9y agoAnd or how do you prove someone's innocence if you can generate a believable, fake, crime video. Supporting counter evidence will become that much more important.
- matrixtransform 9y agoThere's going to need to be some kind of blockchain-style tech that allows the source and veracity of video to be determined.
- andrewla 9y agoA blockchain in this case would just be a timestamping service, to prove the minimum time that elapsed since the image existed. That’s only useful if the time to create a fake is significantly more than the resolution of the chain - 10 minutes for bitcoin, 15 seconds for ethereum. But that I only makes sense if you know ahead of time that the information will be valuable, and it only proves the age if there is sufficient hash power on the network, so a “private” blockchain would not be viable.
- IIAOPSW 9y agoOne application I imagine for this is a future of game development similar to the movie inception where an "architect" designs the layout and setting and then all the details are filled in ad hoc by the computers "imagination". Today its faces that feel familiar but aren't real. Tomorrow its whole cities that feel familiar but aren't real. The cities are filled with people you swear you've seen before. Perhaps the details are tailored to you personally based on the corpus of photos you've posted online.
- hellbanner 9y agoThis is happening. https://www.youtube.com/watch?v=1Ea57XERywM&index=6&list=PLcZ-fMpNVsjUQ_eKlGrOH_OOgjxKzT84w https://www.youtube.com/watch?v=1Ea57XERywM&index=6&list=PLc... "Narrative Dungeon Design"
- grondilu 9y ago"semantic image manipulation" is getting better and better : https://www.youtube.com/watch?v=XhH2Cc4thJw https://www.youtube.com/watch?v=XhH2Cc4thJw
- hellbanner 9y agoSee also: https://news.ycombinator.com/item?id=16040463 https://news.ycombinator.com/item?id=16040463 -- fake porn faceswap generated by AI
- oillio 9y agoGood article and great tech. However, I don't know if I believe the results are as good as they claim. Many of the pictures look a bit off to me, like they all have dead eyes. Maybe celebrities generally look like that anyway, so it is being true to form. :) In particular, I think this guy is missing a pretty significant part of his head: https://static01.nyt.com/newsgraphics/2017/12/26/ai-faces/8e890dbb781d60b011c9be1f9a169ed2583a0dda/finished/7.png https://static01.nyt.com/newsgraphics/2017/12/26/ai-faces/8e...
- atourgates 9y agoThe hair was the giveaway to me. I stared at the two "which one is real" images for a couple minutes thinking, "They both have that fake looking wavy hair, I thought for sure neither was real." Cheap trick NYTimes. Cheap.
- nerfhammer 9y agolook at their foreheads, there's still like blurry half-generated wavey hair texture on their skin.
- GuB-42 9y agoIn most cases hair gave it away for me, especially at the contours. The picture you linked is an extreme example. However, give it to a good Photoshop artist, like most celebrity pictures are, and I'm sure these issues will be fixed in no time.
- visarga 9y agoThe inventor of GANs, which have been considered the most interesting idea in ML in the last decade, is Ian Goodfellow. I met him on reddit a few years ago. I was supposed to get private ML tutoring from him, just around the time Andrew Ng opened the first Coursera course. I didn't get lessons because I gave up and eventually took the MOOC. But it's amazing to know we share the same forums and sometimes exchange a comment or two. The great idea about GANs is that they replace one of the most hard to understand parts of a neural net - the "Loss function" - with another neural net, thus making the loss function learnable. This opens up the door for a kind of unsupervised learning that was impossible to make work before. GANs are very very important also because they are almost like reinforcement learning (actor + critic = RL, generator + discriminator = GAN), and RL is supposed to be the way to AGI. The most famous problem of GANs is instability during training and mode collapse - which is like a student learning especially for an exam (and not in general) thus optimising for the test instead of the real thing.
- saycheese 9y ago>> “QUESTION: Look at the two photos below and see if you can figure out which person is real.” >> “ANSWER: Sorry! This was a trick question. Both images were generated by computers.” Not really a trick question when even if you know they’re both fake that the only way to be right (confirm you are right) is to be wrong.
- pluma 9y agoYeah, I thought both looked a tiny bit off. I think it has to do with the reflection in the eyes which is a tiny bit inconsistent, among other things.
- disantlor 9y agomaybe so (they fooled me) but you were already prepped to scrutinize them. To the point others have made, we’ll soon need to be constantly prepared to assume fakery. the technology of fakery is rising the meet the “everything is fake news” moment
- logfromblammo 9y agoI immediately picked the right image, because I saw whisker stubble on the left, and I already knew that image-generation AIs seem to have a thing for painting whisker stubble all over anything even remotely resembling a male face. Surprise! Guess I should have considered the possibility of a trick question.
- schaefer 9y agoI’d also like to comment on the “ha, fooled you!” tactic used in this article, where the authors asks the reader to choose the photo of the real person from two given photos and then reveals that, gasp, both are computer generated. Whenever I run into this often-used tactic in papers and talks, I can’t help but feel – no, the author didn’t just convince me of their point. Instead they convinced me that they don’t value being trustworthy. Often I will just stop reading the article right then. Or if I do continue I will become unforgivingly skeptical of any claim that doesn’t provide a citation that is independently verifiable. Use of the tactic feels particularly peculiar in an article which itself grasps towards the implications of a future in which photos and videos are no longer trustworthy, a future in which personal reputation will be more meaningful.
- m3kw9 9y agoSo Cat and Mouse is layman’s term for Adversial network?
- fjsolwmv 9y agoNo, is NYT's made up term. "Arms race" is the traditional term
- 33W 9y agoI found the Obama video at the end very interesting. It would be a neat next step to map non-Obama audio to the generated video. For example, pull audio from an Obama impersonator.
- visarga 9y ago> pull audio from an Obama impersonator We can impersonate voices with neural nets. We can clone timbre and style, and this tech is being used commercially by Baidu at the very least (keyword: Deep Voice 3).
- EGreg 9y agoMr. Hwang believes the technology will evolve into a kind of A.I. arms race pitting those trying to deceive against those trying to identify the deception. That's like a chess game. We have seen AlphaGo and other MCTS implementations take the "trying to detect the deception" into account. By the time the image is generated, it would have already been factored in.
- visarga 9y ago> the technology will evolve into a kind of A.I. arms race Ha! Google Brain organised a competition on "Adversarial Attacks and Defenses" within the NIPS 2017 conference. Reminds me of Harry Potter learning magic attack and defence arts at Hogwarts.
- fjsolwmv 9y agoWhere does AlphaGo try to detect deception? What is deception in perfect information games?
- floofyreal 9y agohttps://www.youtube.com/watch?v=XaQu7kkQBPc https://www.youtube.com/watch?v=XaQu7kkQBPc Imagine automated system for danger recognition on for example airport. These kind of deception attacks could make problems with these systems. Imagine if suddenly 10,20,100 airports all around globe would recognize weapons, bombs or any other dangerous items? I can imagine panic and huge news headlines badmouthing AI. People don't trust AI. These kind of errors could only prolong proper integration, which in many ways could enhance the way we live.
- hokkos 9y agoGreat PR but none of the faces look human, with the weird position of the nose, the strange curly thing around the hairs, there is still work to do to trick the most fundamental tool of the brain, recognize a fellow human face.
- olivermarks 9y ago“Believe nothing you hear, and only one half that you see.” ― Edgar Allan Poe
- matte_black 9y agoI would love this as a service for generating fake users.
- sago 9y agoI don't understand the images associated with that article. They purport to show the progressive refinement of the output over a series of days. But the figure changes dramatically from image to image, all the way to the end of the run. At the very least it seems the output is not stable: a human has to decide when to stop the Wheel of Fortune. It looks more like a series of images taken from different training sets or parameters, for the NNs I'm used to. Caveat: I've done a lot of ML, but not GANs specifically. Is this common? How do you solve the 'where to stop' problem if the output is so unstable?
- pk78 9y agoI am not sure if "unstable" is the word I would use. Sure, even after training for days the GAN produces not-so-realistic images, but the rate at which it generates those images gradually decreases over the training period and the images get more "realistic". >How do you solve the 'where to stop' problem if the output is so unstable? Looking at the discriminator loss would be a good start for that.
- sago 9y agoIt's not the quality I was referring to. Look at the main image sequence. The images from 0 to, say, Day 5 show the kind of progressive refinement I expected: the network is improving its image over time. Each image is a refinement of the previous. But compare the images from Day 5 the end. Eye colour is changing and then changing back. As is the background. And the hair colour. The position of the parting. Whether the mouth is closed or showing teeth. Day 16 is not an intermediate point between Day 9 and Day 18. If it runs for another couple of days, would we get another version like Day 16? That's what I mean by instability.
- pk78 9y agoAh, I understand what you are saying. The instabilities could be explained by the batches sampled during those training days and the generator's input. Training a GAN is not very straightforward and even minor changes in batch sampling could produce vastly different generated images.
- fjsolwmv 9y agoThe faces are pretty good but the ears and craniums are awful, presumably because of a dependency on neighboring pixels and getting confused by diverse backround images. Why ruin their work by including he garbage parts in the presentation/claims? And why not leart foreground and background separately, and mask them together?
- oliv__ 9y agoThese are pretty damn good but it seems to me like the program is sort of over optimizing the pictures: in my mind, the pictures from the 5th to the 7th day are the most realistic. After that, it feels to me like the "realness" slowly degrades.
- booleandilemma 9y agoI’d be really worried if I was a photo model.
- bcaulfield 9y agoWe've made good looking people obsolete.
- lerie82 9y agoExcellent images and the more I think about what this could be used for the creepier it gets. One day, truly, software will be very dangerous.
- ByThyGrace 9y agoAnd it makes you wonder, what is the current legal framework preventing this kind of tool to be misused? How does it differ across countries/unions? How can such a thing be enforced to begin with? Are companies/labs/universities/individuals themselves the only thing standing between fair play and massive misuse of realistically generated media?
- bobajeff 9y agoWatching the progress of this system reminds me of that scene at the beginning of The Thing where The Thing almost but not quite mimicks one of the humans.
- IgorPartola 9y agoVery interesting. A this kind of tech something that is mere mortals can play with?