14 ms·

A sneak peek at MetaHuman Creator

171 points by jsheard 6y ago

dagmx 6y ago
Some more useful links Announce trailer: https://www.youtube.com/watch?v=S3F1vZYpH8c https://www.youtube.com/watch?v=S3F1vZYpH8c Sample characters: https://www.unrealengine.com/marketplace/en-US/learn/metahumans https://www.unrealengine.com/marketplace/en-US/learn/metahum...
rgovostes 6y ago
The first time I saw something like this was when Elder Scrolls IV: Oblivion used FaceGen (https://facegen.com/ https://facegen.com/) for character customization. Realism has improved a tiny bit in the intervening 15 years.
mmastrac 6y ago
Still in the uncanny valley, but it feels like we're climbing out of it slowly.
- kleer001 6y ago
  Yea, those teeth are a bit gnarly. But I'm sure they'll nail it down real quick.
  - 1shooner 6y ago
    I would be interested to hear from an animator what makes teeth so challenging.
    - KineticLensman 6y ago
      I'm not a figure modeller but I heavily use the Genesis 8 digital human from Daz 3D [0] My guess is that teeth haven't been that interesting to date and that the modellers don't have access to hundreds of reference pictures / geometries of teeth for their modelling. As opposed to reference images of skin textures, etc. Tongues are probably the same. In fact I don't think I have ever created a render of a figure with their tongue out. Once they take the time, the teeth should be easy to improve. [0] https://www.daz3d.com/technology/ https://www.daz3d.com/technology/
      - Budabellly 6y ago
        I would say that modeling teeth with geometry is very easy. The material, lighting, and rendering is the challenge, especially when you consider all of the occlusion (and hard/soft body dynamics) from your lips that is inevitable with teeth. Some form of subsurface scattering[1] is a must, and not easily done in games/apps because you can't really "bake" it like you do for most other textures to run performantly. [1] https://en.wikipedia.org/wiki/Subsurface_scattering#:~:text=Subsurface%20scattering%20(SSS)%2C%20also,surface%20at%20a%20different%20point https://en.wikipedia.org/wiki/Subsurface_scattering#:~:text=....
        kleer001 6y ago
        (20 years vfx experience, me) Subsurface scattering is the correct answer. It's a very small area and a very expensive calculation. Teeth are translucent and the coloured lighting on them changes rapidly.
    - cma 6y ago
      I think one thing is they are translucent/subsurface scattering and the enamel->dentin layer shows through underneath in kind of volumetric ways. Even lower teeth will show through behind the tips of upper teeth a bit (with a standard overbite).
    - Igelau 6y ago
      It's probably an 80:20 problem of sorts. More like a 99:1. Externally, people mostly appear to be comprised of flesh and hair. Pick any random square inch and the odds are that you will land on flesh or hair. That gets different on eyeballs, but they are still just bags of goo connected to muscle. The devil there is in the details of the blood vessels and the iris and things. And then you get to teeth, which are like wet rocks. You're building this thing that is going to bomb if it can't do soft, meaty, hairy things convincingly and then there's this itty bitty section you're only going to see sometimes that is a dark membraneous cave full of wet rocks.
    - kleer001 6y ago
      vfx veteran here Technically speaking an animator might know. A lighter would have a better idea. And finally a shader writer could tell you exactly. If you were interested in the more technical breakdown of responsibilities in the industry: (of course it'll change from studio to studio, but this is pretty good) https://discover.therookies.co/2019/05/22/job-titles-in-3d-animation-visual-effects-and-what-they-actually-mean/ https://discover.therookies.co/2019/05/22/job-titles-in-3d-a...
- Smaug123 6y ago
  Having seen only the two videos on the article's page, I actually thought they'd passed the valley. To me, they successfully looked artificial (though there were moments where they could have been real) but almost never creepy.
uses 6y ago
Things are going to get really weird in the next decade or two. Right now AI is able to "communicate" with us mostly via text, and indirectly via algorithms. With human-like agents that have human-like voices, appearances, and mannerisms, they'll be able to communicate with millions of average Joes, and all in a way particular to that specific Joe. AI being able to do infinite amounts of "ambient work" and expose us to the results via human-like entities makes for a very different world of possibilities. And right now the big AIs that most people directly interface with are clumsy and abstract, but they're also tied in the backend to large pools of closed data owned by mostly well-meaning entities. But how will things change when and if the AI Joe interacts with isn't directed and contained by wealthy US-based corporations?
- jimsmart 6y ago
  You might be interested in the work of Soul Machines [0] (online demo on website), a company formed by Mark Sagar [1][2]. There are a few interesting talks by him if you search on Youtube. Their AI-driven avatars are already in use in a handful of places, and fairly well received. "The future is already here – it's just not evenly distributed." — Appearance-wise, I think MetaHuman's stuff looks better than Soul Machines' — but SM's avatars are backed with a conversational AI, and are not available as a UE-plugin. So it's not an apples-for-apples comparison of course. [0] https://www.soulmachines.com https://www.soulmachines.com [1] https://www.auckland.ac.nz/en/engineering/community-engagement/alumni/alumni-stories/mark-sagar.html https://www.auckland.ac.nz/en/engineering/community-engageme... [2] https://www.ibm.com/watson/advantage-reports/future-of-artificial-intelligence/mark-sagar.html https://www.ibm.com/watson/advantage-reports/future-of-artif...
  - suyash 6y ago
    nice, this is going to become a commodity technology in few years. I can see Unity following and then so many other 3d design tool makers with similar capabilities.
- olooney 6y ago
  Chloe, the digital avatar on the menu of the video game Detroit: Becoming Human is sort of a sneak peak into what that might be like. (Potential spoilers! Chloe has her own story line, and if you want to experience that, don't watch the whole video.) https://www.youtube.com/watch?v=eNTUhKs_xwQ https://www.youtube.com/watch?v=eNTUhKs_xwQ
klmadfejno 6y ago
I think this looks amazing. A+ work. Free alternatives I've explored are at least an order of magnitude worse than this, and often difficult to integrate.
DethNinja 6y ago
Very impressive shaders, though I wonder what the performance will be. I guess these might be fine for single player games where there are 10 to 15 characters on screen at a time but I doubt they are ready for use cases in MMORPGs or similar games.
witherk 6y ago
Impressive. There is a cartoonlike quality. I wonder if that is there to keep the models from looking creepy or if they just wanted their models to look cartoony.
- Igelau 6y ago
  This is an awkward thing to demo because it has look good, but it also has to look "good". I didn't think the second video with the black woman and the asian man was very impressive. They look airbrushed, which makes sense because you don't want to say "hey, look at how believable these ugly craggy people are", but on the other hand I think it plays it too safe by not showing off what they can do. Of course, that's not really the point. The point is that the metahuman creator tool apparently makes it very fast-and-easy-ish to create "metahuman" characters that look a notch or two better than the Final Fantasy movie from 20 years ago. The older faces that they made in the video that demoed the tool itself, rather than the result, were pretty impressive.
Geee 6y ago
It's very impressive. The innovation here is the generator, that is able to generate realistically constrained geometry and materials. It's probably based on massive amount of 3D scans and GANs, or something similar. Notice that they're realtime rendered, so they don't look completely realistic, but that is the easy part that has been solved already. It's possible to render those characters with offline renderers and they will look photorealistic.
- Pulcinella 6y ago
  They aren’t photorealistic but most of them seemed pretty close to my eyes, like movie actors under perfect lighting with a bit too much makeup.
  - ksec 6y ago
    I think they have been close for a few years already. We are edging closer every year but it is still somewhat Digital. Which is sort of the end of the S curve taking a very long time to perfection. May be in 5 years time I can barely tell the difference. Edit: May be it is the Real Time Rendering. But the real innovation is achieving this in hours and not weeks or months.
    - witherk 6y ago
      Sometimes I wonder if we are also getting better at distinguishing artificial humans as time goes on. I remember as a kid thinking that G-man in halflife-2 was almost photorealistic.
- Arelius 6y ago
  I strongly doubt there are any GANs involved, they are rarely used in production entertainment, and a solution doesn't really exist for vector geometry afaik. Likely very few scans as well. Just lots and lots of artist hours.
  - Geee 6y ago
    I don't think you can create constraint-based system with 'artist hours'. Just like you can't create realistic 2D face generator without a GAN. It has to be based on samples of real people. There exists lots of generators, such as Daz3D, but you can't create realistic results in these because of their simplistic model. I haven't tested this yet though, so I might be assuming things.
Igelau 6y ago
Those teeth look disturbingly herbivorous.
coolspot 6y ago
> MetaHuman Creator is a cloud-streamed app No piracy, no software ownership and all IP stays in Epic’s datacenter.
- cma 6y ago
  You can't copy the customizer (assuming it rebakes new occlusion maps and stuff based on the shape customization), but the final result is a bit more open ended than that: "You’ll also get the source data in the form of a Maya file, including meshes, skeleton, facial rig, animation controls, and materials."
- pkaye 6y ago
  Software as a service.
munificent 6y ago
Many years ago, someone told me a little heuristic to determine if any new technology is likely to be successful. It doesn't always pick winners, but it rarely picks losers: [_] Can this be used for porn? I think we have a winner here. (Other examples: 8mm film, VHS, the Internet, newsgroups, JPEG, streaming video, etc.)
- deleted 6y ago
  [deleted]
- suyash 6y ago
  lol, what an interesting test. Yes, the multi billion dollar porn industry is for sure to leverage this.
superkuh 6y ago
Oh neat. Someone made a modern version of the "Poser" software series.
tekromancr 6y ago
Just imagining the next wave of indie games with hyper-realistic humans with subsurface scattering in low-poly, cell shaded environments
- dbish 6y ago
  That actually sounds like it could be interesting in the right game. Kind of a dreamscape feel
suyash 6y ago
What's going to be the cost of using this? Why not make a plugin in Unreal instead of having to learn and juggle a separate tool?
atum47 6y ago
As a 3D artist I remember being very very difficult to model a realistic human being. Then came zBrush and made it a little bit easier. I just saw the IGN video for this and I got to say, it's really impressive.
throwaway743 6y ago
How is this different from MakeHuman?

171 points by jsheard 6y ago

Some more useful links Announce trailer: https://www.youtube.com/watch?v=S3F1vZYpH8c https://www.youtube.com/watch?v=S3F1vZYpH8c Sample characters: https://www.unrealengine.com/marketplace/en-US/learn/metahumans https://www.unrealengine.com/marketplace/en-US/learn/metahum...

rgovostes 6y ago

The first time I saw something like this was when Elder Scrolls IV: Oblivion used FaceGen (https://facegen.com/ https://facegen.com/) for character customization. Realism has improved a tiny bit in the intervening 15 years.

mmastrac 6y ago

Still in the uncanny valley, but it feels like we're climbing out of it slowly.

kleer001 6y ago
Yea, those teeth are a bit gnarly. But I'm sure they'll nail it down real quick.
- 1shooner 6y ago
  I would be interested to hear from an animator what makes teeth so challenging.
  - KineticLensman 6y ago
    I'm not a figure modeller but I heavily use the Genesis 8 digital human from Daz 3D [0] My guess is that teeth haven't been that interesting to date and that the modellers don't have access to hundreds of reference pictures / geometries of teeth for their modelling. As opposed to reference images of skin textures, etc. Tongues are probably the same. In fact I don't think I have ever created a render of a figure with their tongue out. Once they take the time, the teeth should be easy to improve. [0] https://www.daz3d.com/technology/ https://www.daz3d.com/technology/
    - Budabellly 6y ago
      I would say that modeling teeth with geometry is very easy. The material, lighting, and rendering is the challenge, especially when you consider all of the occlusion (and hard/soft body dynamics) from your lips that is inevitable with teeth. Some form of subsurface scattering[1] is a must, and not easily done in games/apps because you can't really "bake" it like you do for most other textures to run performantly. [1] https://en.wikipedia.org/wiki/Subsurface_scattering#:~:text=Subsurface%20scattering%20(SSS)%2C%20also,surface%20at%20a%20different%20point https://en.wikipedia.org/wiki/Subsurface_scattering#:~:text=....
      - kleer001 6y ago
        (20 years vfx experience, me) Subsurface scattering is the correct answer. It's a very small area and a very expensive calculation. Teeth are translucent and the coloured lighting on them changes rapidly.
  - cma 6y ago
    I think one thing is they are translucent/subsurface scattering and the enamel->dentin layer shows through underneath in kind of volumetric ways. Even lower teeth will show through behind the tips of upper teeth a bit (with a standard overbite).
  - Igelau 6y ago
    It's probably an 80:20 problem of sorts. More like a 99:1. Externally, people mostly appear to be comprised of flesh and hair. Pick any random square inch and the odds are that you will land on flesh or hair. That gets different on eyeballs, but they are still just bags of goo connected to muscle. The devil there is in the details of the blood vessels and the iris and things. And then you get to teeth, which are like wet rocks. You're building this thing that is going to bomb if it can't do soft, meaty, hairy things convincingly and then there's this itty bitty section you're only going to see sometimes that is a dark membraneous cave full of wet rocks.
  - kleer001 6y ago
    vfx veteran here Technically speaking an animator might know. A lighter would have a better idea. And finally a shader writer could tell you exactly. If you were interested in the more technical breakdown of responsibilities in the industry: (of course it'll change from studio to studio, but this is pretty good) https://discover.therookies.co/2019/05/22/job-titles-in-3d-animation-visual-effects-and-what-they-actually-mean/ https://discover.therookies.co/2019/05/22/job-titles-in-3d-a...
Smaug123 6y ago
Having seen only the two videos on the article's page, I actually thought they'd passed the valley. To me, they successfully looked artificial (though there were moments where they could have been real) but almost never creepy.

uses 6y ago

Things are going to get really weird in the next decade or two. Right now AI is able to "communicate" with us mostly via text, and indirectly via algorithms. With human-like agents that have human-like voices, appearances, and mannerisms, they'll be able to communicate with millions of average Joes, and all in a way particular to that specific Joe. AI being able to do infinite amounts of "ambient work" and expose us to the results via human-like entities makes for a very different world of possibilities. And right now the big AIs that most people directly interface with are clumsy and abstract, but they're also tied in the backend to large pools of closed data owned by mostly well-meaning entities. But how will things change when and if the AI Joe interacts with isn't directed and contained by wealthy US-based corporations?

jimsmart 6y ago
You might be interested in the work of Soul Machines [0] (online demo on website), a company formed by Mark Sagar [1][2]. There are a few interesting talks by him if you search on Youtube. Their AI-driven avatars are already in use in a handful of places, and fairly well received. "The future is already here – it's just not evenly distributed." — Appearance-wise, I think MetaHuman's stuff looks better than Soul Machines' — but SM's avatars are backed with a conversational AI, and are not available as a UE-plugin. So it's not an apples-for-apples comparison of course. [0] https://www.soulmachines.com https://www.soulmachines.com [1] https://www.auckland.ac.nz/en/engineering/community-engagement/alumni/alumni-stories/mark-sagar.html https://www.auckland.ac.nz/en/engineering/community-engageme... [2] https://www.ibm.com/watson/advantage-reports/future-of-artificial-intelligence/mark-sagar.html https://www.ibm.com/watson/advantage-reports/future-of-artif...
- suyash 6y ago
  nice, this is going to become a commodity technology in few years. I can see Unity following and then so many other 3d design tool makers with similar capabilities.
olooney 6y ago
Chloe, the digital avatar on the menu of the video game Detroit: Becoming Human is sort of a sneak peak into what that might be like. (Potential spoilers! Chloe has her own story line, and if you want to experience that, don't watch the whole video.) https://www.youtube.com/watch?v=eNTUhKs_xwQ https://www.youtube.com/watch?v=eNTUhKs_xwQ

klmadfejno 6y ago

I think this looks amazing. A+ work. Free alternatives I've explored are at least an order of magnitude worse than this, and often difficult to integrate.

DethNinja 6y ago

Very impressive shaders, though I wonder what the performance will be. I guess these might be fine for single player games where there are 10 to 15 characters on screen at a time but I doubt they are ready for use cases in MMORPGs or similar games.

witherk 6y ago

Impressive. There is a cartoonlike quality. I wonder if that is there to keep the models from looking creepy or if they just wanted their models to look cartoony.

Igelau 6y ago
This is an awkward thing to demo because it has look good, but it also has to look "good". I didn't think the second video with the black woman and the asian man was very impressive. They look airbrushed, which makes sense because you don't want to say "hey, look at how believable these ugly craggy people are", but on the other hand I think it plays it too safe by not showing off what they can do. Of course, that's not really the point. The point is that the metahuman creator tool apparently makes it very fast-and-easy-ish to create "metahuman" characters that look a notch or two better than the Final Fantasy movie from 20 years ago. The older faces that they made in the video that demoed the tool itself, rather than the result, were pretty impressive.

Geee 6y ago

It's very impressive. The innovation here is the generator, that is able to generate realistically constrained geometry and materials. It's probably based on massive amount of 3D scans and GANs, or something similar. Notice that they're realtime rendered, so they don't look completely realistic, but that is the easy part that has been solved already. It's possible to render those characters with offline renderers and they will look photorealistic.

Pulcinella 6y ago
They aren’t photorealistic but most of them seemed pretty close to my eyes, like movie actors under perfect lighting with a bit too much makeup.
- ksec 6y ago
  I think they have been close for a few years already. We are edging closer every year but it is still somewhat Digital. Which is sort of the end of the S curve taking a very long time to perfection. May be in 5 years time I can barely tell the difference. Edit: May be it is the Real Time Rendering. But the real innovation is achieving this in hours and not weeks or months.
  - witherk 6y ago
    Sometimes I wonder if we are also getting better at distinguishing artificial humans as time goes on. I remember as a kid thinking that G-man in halflife-2 was almost photorealistic.
Arelius 6y ago
I strongly doubt there are any GANs involved, they are rarely used in production entertainment, and a solution doesn't really exist for vector geometry afaik. Likely very few scans as well. Just lots and lots of artist hours.
- Geee 6y ago
  I don't think you can create constraint-based system with 'artist hours'. Just like you can't create realistic 2D face generator without a GAN. It has to be based on samples of real people. There exists lots of generators, such as Daz3D, but you can't create realistic results in these because of their simplistic model. I haven't tested this yet though, so I might be assuming things.

Igelau 6y ago

Those teeth look disturbingly herbivorous.

coolspot 6y ago

> MetaHuman Creator is a cloud-streamed app No piracy, no software ownership and all IP stays in Epic’s datacenter.

cma 6y ago
You can't copy the customizer (assuming it rebakes new occlusion maps and stuff based on the shape customization), but the final result is a bit more open ended than that: "You’ll also get the source data in the form of a Maya file, including meshes, skeleton, facial rig, animation controls, and materials."
pkaye 6y ago
Software as a service.

munificent 6y ago

Many years ago, someone told me a little heuristic to determine if any new technology is likely to be successful. It doesn't always pick winners, but it rarely picks losers: [_] Can this be used for porn? I think we have a winner here. (Other examples: 8mm film, VHS, the Internet, newsgroups, JPEG, streaming video, etc.)

deleted 6y ago
[deleted]
suyash 6y ago
lol, what an interesting test. Yes, the multi billion dollar porn industry is for sure to leverage this.

superkuh 6y ago

Oh neat. Someone made a modern version of the "Poser" software series.

tekromancr 6y ago

Just imagining the next wave of indie games with hyper-realistic humans with subsurface scattering in low-poly, cell shaded environments

dbish 6y ago
That actually sounds like it could be interesting in the right game. Kind of a dreamscape feel

suyash 6y ago

What's going to be the cost of using this? Why not make a plugin in Unreal instead of having to learn and juggle a separate tool?

atum47 6y ago

As a 3D artist I remember being very very difficult to model a realistic human being. Then came zBrush and made it a little bit easier. I just saw the IGN video for this and I got to say, it's really impressive.

throwaway743 6y ago

How is this different from MakeHuman?