3 ms·
Um, wow. These are really, really good. They are not perfect, but the improvements on fidelity, open mouth, eyes over a GAN-based approach are .. real high. Th
by vessenes 2y ago
Um, wow. These are really, really good. They are not perfect, but the improvements on fidelity, open mouth, eyes over a GAN-based approach are .. real high.
This is the first paper I’ve seen with videos that compare a person with a re-render side by side, and it’s a nice way to see what the model’s good at, and what it’s not.
Some perf numbers (which they say are unoptimized): 30-60 hrs on a 3080 for the avatar model, and rendering in the 20-40fps range on the same hardware. Basically good enough for a commercial implementation. They don’t mention latency of the CNN side that I can find, which is obviously a big question for chat scenarios, although not a big deal for pre-rendered scenes.
- brandonpelfrey 2y agoAgreed, very impressive results. It's both ~worrying and amazing that I'm sure an AI agent could just directly trace a path in the expression latent space to produce a photorealistic and real-time rendered head.
- vessenes 2y agoit feels to me like the gaussian-based algorithm improvements will likely force new render pipelines away from triangles sooner rather than later. It's hard to imagine giving up fidelity like this. And re-rendering to textured triangles is not fast right now. Should be a fun couple of years!
- spookie 2y agoThe real issue is the dependency in a real life enactment. Not even considering the performance.