4 ms·

There's definitely something uncanny about the lips. I feel like the edges of the face are also more static than I expect (especially when she makes dramatic fa

There's definitely something uncanny about the lips. I feel like the edges of the face are also more static than I expect (especially when she makes dramatic facial expressions like the "angry" expression, which ought to pull the skin a bit). Very video-gamey, not super convincing. But then again this is running a model in just a few seconds, with input from consumer hardware! Makes me wonder what you could do with more capable hardware (or if the iPhone hardware is really THAT good). It does make me think of one excellent application for this software: scanning your own face to make an in-game character who looks exactly like you. I assume I'm not the only one who's labored over character generators for hours to produce a decent-looking character. I think there's a neat opportunity here to get video game players to scan their own face as input instead.

ambicapter 4y ago
This demo doesn't demonstrate scanning your own face to make an in-game character though, just using your face to create the animation mesh. You then have to use an existing high-quality model to generate the final result. Generating a high-quality model from a camera phone is a whole other can of worms.
- pr0zac 4y ago
  Epic released the tooling for doing an iPhone face scan to create a mesh that can then be used to create a rigged Metahuman model about 6 months ago! https://www.unrealengine.com/en-US/blog/how-to-use-realitycapture-to-scan-yourself-for-mesh-to-metahuman https://www.unrealengine.com/en-US/blog/how-to-use-realityca... My significant other works for Epic on the cinematics team so we've played around with Metahuman a bit. Scans we've done weren't immediately perfect (like the video on that linked page mentions, hair and ears don't scan well) but cleaning it up was basically just deleting those portions of the mesh, letting RealityCapture fill in the rest of the head for the model, and then adding similar looking hair from Metahuman's database. So there's probably a little bit of work needed before we get "scan your head with your phone to create a video game character automatically" but if you're cool requiring hair selection as opposed to auto generating it from the scan we're pretty much there.

ambicapter 4y ago

This demo doesn't demonstrate scanning your own face to make an in-game character though, just using your face to create the animation mesh. You then have to use an existing high-quality model to generate the final result. Generating a high-quality model from a camera phone is a whole other can of worms.

pr0zac 4y ago
Epic released the tooling for doing an iPhone face scan to create a mesh that can then be used to create a rigged Metahuman model about 6 months ago! https://www.unrealengine.com/en-US/blog/how-to-use-realitycapture-to-scan-yourself-for-mesh-to-metahuman https://www.unrealengine.com/en-US/blog/how-to-use-realityca... My significant other works for Epic on the cinematics team so we've played around with Metahuman a bit. Scans we've done weren't immediately perfect (like the video on that linked page mentions, hair and ears don't scan well) but cleaning it up was basically just deleting those portions of the mesh, letting RealityCapture fill in the rest of the head for the model, and then adding similar looking hair from Metahuman's database. So there's probably a little bit of work needed before we get "scan your head with your phone to create a video game character automatically" but if you're cool requiring hair selection as opposed to auto generating it from the scan we're pretty much there.