3 ms·
It's still on the wav2lip[1] level from 5 years ago (look at the teeth) just slightly higher resolution. The only real player with a moat is probably flawlessai
by BoredPositron 2y ago
It's still on the wav2lip[1] level from 5 years ago (look at the teeth) just slightly higher resolution. The only real player with a moat is probably flawlessai. [2]
[1] https://github.com/Rudrabha/Wav2Lip https://github.com/Rudrabha/Wav2Lip
[2] https://www.flawlessai.com/ https://www.flawlessai.com/
- HuiLi1998 2y agoActually, our work not only targets lip movements, it can also produce more realistic head movements and facial muscle movements. ps: It also achieved a good syncnnet score on 200 randomly collected wild image-audio pair. And it is open sourced.:)
- BoredPositron 2y agoI get exactly what you are doing but I don't understand what would be novel about it? It really looks just like a simple SD/wav2vec/insightface/animatediff pipeline, everyone can plug together in comfy or with diffusers. The muscle claim is also a bit dubious... PS: With insightface models in your reqs the oss aspect is also pretty much void for any use other than research and your readme should reflect that.
- lossolo 2y agoWhat's SOTA open source?
- Loughla 2y agoThe flawlessai is interesting, but even in their formal example for marketing there are weird tells. The actresses eyes are too jerky and weird during the changed sections. It's strange, but in a way that I'm not sure I could tag without the context of the rest of the video. Is this what we're in for? Human movement that is just normal enough to pass, but not really natural enough to be comfortable with?
- kevindamm 2y agoWe're sure to be in an uncanny valley for a little while first, the real question is whether it is followed by a winter or some of that sweet verisimilitude.