6 ms·
Generating Expressive Portrait Videos with Audio2Video
- smusamashah 3y agoThis is amazing, even the breathing can be seen. Next big step (or may be its done already) is to generate expressive audio and we are all set for a generated model you can have a video call with.
- pmx 3y agoSuno's Bark has expressive audio https://suno-ai.notion.site/Bark-v0-Examples-e572bcfcdf65429c916d4c6dd8ae175b https://suno-ai.notion.site/Bark-v0-Examples-e572bcfcdf65429...
- turnsout 3y agoThis is incredibly good—definitely good enough for videoconferencing. To me this is a more promising path than the classical 3D approach Apple is taking with Personas on the Vision Pro. Just need to make it realtime!
- artninja1988 3y agoLooks like it's from some of the same authors as this paper: https://github.com/HumanAIGC/AnimateAnyone https://github.com/HumanAIGC/AnimateAnyone . Which sadly never released the code or model.
- noduerme 3y agoComing soon to a political scandal near you...
- ashildr 3y agoThis is becoming frightening now. Maybe we need to start to cryptographically sign original sources and require others to do so to prove if something is a real recording? I can spot some details that don’t seem to be moving ‘right’ according to physics, but this is extremely convincing.
- jerpint 3y agoPerhaps signing any kind of upload or content with a GPG key that proves your identity?
- pedalpete 3y agoPeople will claim this isn't possible, or will never work, but they often miss the fact that we don't need to sign every video. Only source videos of certain people need to be signed. For example, Tom Hanks was used to shill some dental product (I think). He's a public figure. He, and we, would want to be able to validate that anything using his likeness is actually him. For me, nobody cares if a video of me is me or not, so it doesn't need to have a signature.
- grugagag 3y agoNobody cares if a video of you is you or not for now but this is subject to change in the near future. Perhaps advertising to your friends and acquaintances with a video of you selling or endorsing some scam product would make you care more?
- ShamelessC 3y agoGets mentioned a lot. Typical response is “take a picture/video/recording of the video with your own crypto signature”. Thwarted.
- ashildr 3y agoYes, but we don’t have an accessible vetting infrastructure for this. I don’t know anything about “ShamelessC” so I have no idea if a video of Hillary Clinton drinking the blood of children should be trusted or not.
- ShamelessC 3y agoI believe you're just describing the current system.
- 3y ago
- ashildr 3y agoMaybe someone more on the autistic spectrum here can tell if they feel that the facial expressions are “right”? The examples are too convincing to my brain to properly focus on analyzing that part.
- smusamashah 3y agoNot autistic or anything but I got motion sick from Sora generated videos. So much that I don't want to look at most pf them again, at least on Sora page in original quality. I could see things getting blurry and some blending. The very last video on the page has teeth issue. Expressions were as good as an actor for me and lip movement looked just perfect.
- CSSer 3y agoThis is impressive overall, but I wonder if someone can provide some insight into, or confirmation of, an observation of mine. I’m having a hard time describing it, but there’s a stiff, shiny, and plastic feel to a lot of these. You can really see it in the video of the woman wearing sunglasses. No one can, or at least likely wouldn’t, hold their eyebrow muscles that stiff for that long. Combined with the head and mouth movements, the expressiveness just feels off, almost animatronic. Does anyone else get that?
- unraveller 3y agoIf you can describe it that just means the next version can avoid it. But what you describe sounds like modern soap operas which are plastic surgery expositions. The last mona lisa doing shakespeare and the last ai girl doing multiple voices are extremely convincing, I think you over estimate just how intensely we look at things normally after we accept them. This is great for many things right now and will be greater once they can perfect all the subtle movements that make video worth watching over audio (easier information absorption and recall).
- grugagag 3y agoI agree, it looks uncanny. The immitation will get better and better. Sometimes in wondering why we’re fascinated with generating artificial humans when there are literally billions of humans that could be filmed with cameras.
- nicwolff 3y agoHumans want to be paid.
- lacrimacida 3y agoWhat are you gonna do with an economy where people don’t get paid then? You cant have your cake and eat it too