6 ms·
I used to fancy myself a starving artist (emphasis on "starving"). These pictures suffer from a common newbie-artist problem! One of the concepts of early sta
by jvanderbot 7y ago
I used to fancy myself a starving artist (emphasis on "starving").
These pictures suffer from a common newbie-artist problem!
One of the concepts of early stage drawing I noticed is "symbol drawing". This is where a person knows how an eye should look, and where an eye should go on the face, and so draws an eye on a face-shaped oval. They repeat this for the other eye, a nose, a mouth, etc. They are often sad with the results.
A person who practices incorrectly will try to do photorealistic eyes, noses, mouths, etc but use the same 2D composition technique. No matter how realistically they can render components, their composition is off because they are ignoring the structure of a face that causes subtle differences in the shape of each of the components when projected onto a sheet of paper.
Our mind is so capable of 3D modelling that when we look at a face we don't notice the changes in perspective leading to changes in component shape and shading. The great artist skill is to build up those 3D to 2D projections and avoid the 2D composition (except in stylistic choices).
I suspect that the networks rendering these faces, and the underlying biological phenomena they are designed against, are "symbol" recognizers, rather than "structural" recognizers.
- dehrmann 7y agoI've always struggled with drawing organic 3D things, and this is probably part of why. > A person who practices incorrectly So how does one practice correctly?
- RenRav 7y agoLooks almost perfect to me, it's just the eyes and hands that consistently seem messed up.
- rightbyte 7y agoThe eyes look really creepy somehow. It's like it is two different eyes.
- mraudiobook_com 7y agoI bet any amount that side by side you could not reliably tell the difference between real and these generated faces
- spyder 7y agoThey are many models that are capable of generating 3D faces even from single photos. For example: https://www.youtube.com/watch?v=qX8AIRsFmTA https://www.youtube.com/watch?v=qX8AIRsFmTA or use the rotation of a real face as a guide: https://www.youtube.com/watch?v=p1b5aiTrGzY https://www.youtube.com/watch?v=p1b5aiTrGzY it's not hyper-realistic but it's getting better and better.
- savanaly 7y agoAt many points in my life I have tried to do sketching and recognize my problems in your description of symbol drawing. Since you sound knowledgeable about it could you possibly recommend something that expands on that topic, and gives advice for learning to draw that is centered on that concept?
- jvanderbot 7y agoLearn to sculpt! For me, a few hours making faces from clay was all it took to teach my mind the mapping from 3d to 2d. This forces you to think from 2d to 3d by taking an image you have in your mind and forcing you to build 3d support for it. Then, do still life sketches of what you sculpt, no matter how bad. This seemed to reinforce the other direction for me. That and play Descent 2.
- paulnpace 7y agoIs my observation correct that this is most apparent when the face is not perfectly perpendicular to the display? I feel like even some tiny fraction of a degree, and I sense something is not quite right, while the images that seem most believable are perfectly perpendicular to the display. Is it also difficult to get the face to align with the teeth? Nearly all of their pearly whites feel eerily disconnected to me.
- lonelappde 7y agoThat seems to be an artifact of 3D model being non-existent, so all the projected angles are wrong unless the output angle matches the common input angle.
- jvanderbot 7y agoThat was my impression as well! It looks like those projects where a kid did the initial sketch and a skilled artist did a photo-real rendering.
- jvanderbot 7y agoFurthermore, I'd go so far as to say that the reason these faces "feel" weird sometimes is that your mind is reconstructing the structure of the fictitious persons' bodies and warning you that they have bones that are misshapen and lumps under their eyes, and teeth that point strangely, etc. A biological response might be "misshapen eye ... likely parasite detected, avoid this person" or "mismatched emotional cues ... likely brain damage / person unpredictable, avoid"
- nitrogen 7y agoGetting glasses for the first time at an adult age can have a similar distorting effect on facial perception for the first few months.
- defen 7y ago> likely parasite detected, avoid this person Definitely the case for this guy https://cdn.vox-cdn.com/uploads/chorus_asset/file/19216620/000023.jpg https://cdn.vox-cdn.com/uploads/chorus_asset/file/19216620/0...
- falsedan 7y agoThere no parasite that causes facial deformities, and mismatched emotional cues are a fundamental part of humour/acting. It makes as much sense to rationalise the unbidden disgust at unexpected facial structure with disease justifications as it does for the far more likely “person appears to have been damaged during youth/juvenile period by violent/poor nutrition & is of a low social class”.
- lonelappde 7y agoThanks for leading a vanguard of anti-racism against deepfakes. Moving your left eye, nose, mouth, and right ear independently of each other is fundamental to acting? "Unexpected" is a milder term than "never before seen on a human seen, nor interpolated".
- falsedan 7y ago
- dpcan 7y agoI think your point is that people will recognize these faces as not being real, or maybe at least feel like they are "off" in some way, but can't put their finger on it... That being said, if you scroll through thousands of these, what you are saying does happen often, but even more often, the faces look completely real and I would have no problem believing they were anything but. Maybe I missed your point though....
- cstrahan 7y ago> [...] the faces look completely real and I would have no problem believing they were anything but. I suppose that depends on who the viewer is. For example, everyone at work was blown away with how hard it was to pick the real face here: http://www.whichfaceisreal.com/index.php http://www.whichfaceisreal.com/index.php However, I went through about 50 in under a minute and got every one correct. Programmers spend their time thinking in abstractions and models, rather than appreciating things as they are, as an artist must (see Drawing on the Right Side of the Brain -- https://www.drawright.com/theory https://www.drawright.com/theory). I wouldn't be surprised if more artsy types have a much easier time recognizing these fake photos, despite all the component pieces roughly being where they should be.
- kthejoker2 7y agoFunny how 95% of the time the most obvious "wrong" thing is unrealistic teeth, teeth have just as much variety as hair and skin but the fake ones all have this mannequin style bridgework (spoken as the son of a dentist)
- ravar 7y agoFor me I figured out how to get 100% just looking at the backgrounds, the fake images have monochrome backgrounds or subtle artifacts.
- mactrey 7y agoIf you scrolled through thousands of actual stock photos I'm sure you'd find plenty that seem "off" in some way. Those just cost a lot more.
- AbrahamParangi 7y agoConvolutional neural nets work operate on low level structure and do high-level structure largely by stacking together low-level structural processing. The "right way" to do high-level structure for images is a fruitful active area of investigation which I expect will probably be tackled in the next 18 months.
- lonelappde 7y agoI don't think it's possible to do without learning a hidden 3D model from 2D data. It's nonsense to learn 2D projections directly from 2D projections, because small errors in the 2D projection can become catastrophic errors in the 3D model that humans use to interpret a 2D photo.
- jvanderbot 7y agoLearn maybe. But its easy to construct 3d information from 2d observations. See structure from motion, shape from shadow, Voxel carving, and even orbit determination.
- fooker 7y ago>18 months. Bold prediction. I wouldn't be surprised if this is an open problem in 18 years.
- AbrahamParangi 7y agoI'll go further: For each problem there are a few sub-problems and a "right way" to do each of them where a "right way" corresponds to an algorithm which efficiently tackles some element of the implicit structure of the problem. I think within 5 years we will have made great progress on algorithmically discovering these "right ways" across many problem domains. I think both differentiable neural architecture search[0] and the MAML with implicit gradients[1] stuff is very interesting from this perspective. [0]https://arxiv.org/abs/1905.13577 https://arxiv.org/abs/1905.13577 [1]https://arxiv.org/abs/1909.04630 https://arxiv.org/abs/1909.04630
- binarymax 7y agoAlso as a starving artist who isn't starving because I have a non-artist job, I agree wholeheartedly. I've also done some research into this and capsule networks are the closest SOTA we have to solving this problem. There isn't any 2D projections of 3D space work with these networks AFAIK, but the purpose and intuition points in this direction. Also, quality data is always the hard part here, so we might be able to bootstrap some of this structure learning with 3D models created for games, projected as 2D, and use that as transfer learning somehow.
- jvanderbot 7y agoIn a sense, 3D models are the ultimate compression of 2D information, and therefore the most amenable of 2D transfer learning?