4 ms·
NPGA: Neural Parametric Gaussian Avatars – high-fidelity digital faces
- kaivi 2y agoTheir github page has some videos: https://simongiebenhain.github.io/NPGA/ https://simongiebenhain.github.io/NPGA/
- vessenes 2y agoUm, wow. These are really, really good. They are not perfect, but the improvements on fidelity, open mouth, eyes over a GAN-based approach are .. real high. This is the first paper I’ve seen with videos that compare a person with a re-render side by side, and it’s a nice way to see what the model’s good at, and what it’s not. Some perf numbers (which they say are unoptimized): 30-60 hrs on a 3080 for the avatar model, and rendering in the 20-40fps range on the same hardware. Basically good enough for a commercial implementation. They don’t mention latency of the CNN side that I can find, which is obviously a big question for chat scenarios, although not a big deal for pre-rendered scenes.
- brandonpelfrey 2y agoAgreed, very impressive results. It's both ~worrying and amazing that I'm sure an AI agent could just directly trace a path in the expression latent space to produce a photorealistic and real-time rendered head.
- vessenes 2y agoit feels to me like the gaussian-based algorithm improvements will likely force new render pipelines away from triangles sooner rather than later. It's hard to imagine giving up fidelity like this. And re-rendering to textured triangles is not fast right now. Should be a fun couple of years!
- spookie 2y agoThe real issue is the dependency in a real life enactment. Not even considering the performance.
- azinman2 2y agoGiven the now clear abuse potential (and other issues surrounding AI generally), why do papers like this discuss the motivation as if it’s a for sure requirement to do such work? “The creation of high-fidelity, digital versions of human heads is an important stepping stone in the process of further integrating virtual components into our everyday lives.” It’s not clear to me that’s a desirable outcome.
- throwaway1105q 2y agoI don't understand your question. There are some cases of abuse and so the researchers shouldn't discuss applications of the research?
- techbro92 2y agoIt is kind of a requirement. Scientific papers will typically describe the motivation for a work to start the introduction. The real motivation of course is “I need to publish something”.
- throwaway1105q 2y agoThe real motivation is "I want to research this field", publishing is part of the way. Nobody is doing research "to publish something", there are much simpler fields to enter if that was the motivation.
- superb_dev 2y agoIf you decide to research in $field, then you’ll typically need to keep publishing papers to keep your job
- throwaway1105q 2y agoIndeed. That's what I'm saying - you decide you want to research and so you publish. Nobody decides to publish and so does research.
- supportengineer 2y agoIn the novel Snow Crash, one of the characters, Juanita, does just this - she is known for making realistic facial expressions possible in the metaverse.