4 ms·

Pretty cool. Maybe someone more knowledgeable can explain why this is different than existing facial capture techniques? Is it just the same stuff repackaged in

• meh8881 4y ago
  You can do it on an iPhone with no additional equipment and can apply it to a MetaHuman model, which is a relatively high quality human model that epic has tools to generate.
  • dorkwood 4y ago
    It’s worth noting that both the FaceCap and Moves By Maxon apps can also capture facial animation using only an iPhone with no additional equipment. It’s hard to tell from the video, but I wouldn’t be surprised if they used their face capture animation to drive wrinkle maps[1] on their MetaHuman models. This is something that other apps don’t offer out of the box, but which can significantly increase realism. [1] https://m.youtube.com/watch?v=nydjtjIncSk https://m.youtube.com/watch?v=nydjtjIncSk
    • meh8881 4y ago
      What do you do with those app outputs though? I’m guessing it still feels hard because it won’t automatically map to a mesh that is good to go in your game engine? The MetaHumans drop in models are the thing that makes this magic imo. It’s not the motion capture tech so much as the complete pipeline to produce game assets.
      • dorkwood 4y ago
        FaceCap gives you an FBX animated with about 50 different morph targets. If you wanted to use that in a game, you’d probably need to load all the morph targets into a shader yourself. I agree that the automatic integration with MetaHuman is the main benefit.
  • nmfisher 4y ago
    Actually you’ve already been able to do that for a few years now with Epic’s LiveLinkFace app (which was basically just a wrapper around Apple’s ARKit). This is just their own (much higher quality version) of it.
    • meh8881 4y ago
      I’ve played around with it (the initial version). I didn’t think LiveLink was good enough personally.
  • WheatMillington 4y ago
    I keep seeing people say this without explaining what the huge thing strapped to the actor's face is, that clearly isn't a phone. Not explained in the article either.
    • meh8881 4y ago
      I’m not sure what you mean. There’s one shot in the video where they show a full motion rig and clearly explain that this system works with both a phone and/or a professional stereo system. I’m guessing that is for when you want to move your body around and capture the facial expressions associated with doing so. You need some sort of rig to keep the camera in your face if you’re moving your face.
• pzone 4y ago
  A few differences compared to other commercial iPhone-based capture techniques: - Whole-clip solve instead of noisy frame-by-frame streaming - Automatically generating a face model to calibrate the rig - Solving from raw sensor data instead of Apple ARKit pose data VFX studios typically develop proprietary solutions a few years in advance, so it can be hard to say what is truly "new and different."
• barrysteve 4y ago
  This Bebylon presentation at SIGGRAPH 2018 is an example of the (now much older) way of doing the same iPhone face capture. https://www.youtube.com/watch?v=lXZhgkNFGfM https://www.youtube.com/watch?v=lXZhgkNFGfM The detail on the new Metahuman face capture is visibly better and a smoother capture pipeline has been a valuable goal for a some time. Noisy mocap and re-rigging animations to share them between models, takes time. [Quote from article] The algorithm uses a "semantic space solution" that Mastilovic said guarantees the resulting animation "will always work the same in any face logic... it just doesn't break when you move it onto something else."
• GeekyBear 4y ago
  >why this is different than existing facial capture techniques In the past, it was common to use video capture with reference markings on the actor so you could figure out how they were moving in three diminsions. From this demo, it looks like they are directly leveraging the 3D data from the iPhone's Lidar sensor in addition to the video camera data.
  • junon 4y ago
    Right but that's been available for a while through various apps.