3 ms·

What I want is to be able to feed in a bunch of videos and generate an animatable (from talking) 3D face from that data. I suppose you in theory only need 3 ima

by ge96 2y ago

What I want is to be able to feed in a bunch of videos and generate an animatable (from talking) 3D face from that data. I suppose you in theory only need 3 images (front and sides). But mapping pixels to motion is interesting (facial expressions). There wouldn't be depth data so it would be inferred from shadows

ilaksh 2y ago
Replicate has Hunyuan video training now. https://replicate.com/blog/fine-tune-video https://replicate.com/blog/fine-tune-video Also, Kling 1.6 Elements works pretty okay if you use the same person/face for each element. Kling also has lip sync. Or this lip sync with replicate: https://replicate.com/bytedance/latentsync https://replicate.com/bytedance/latentsync Or there is HeyGen or D-ID or Synthesia, or tavus.io for full interactive digital twins.
- ge96 2y ago
  Thanks
timdiggerm 2y ago
Why do you want to do that?
- ge96 2y ago
  My case is not directly nefarious, for example an old popular YouTuber that streamed in the early 2000s taking their content and making a model of them for personal use like a 3D chat bot but with that person's quirks Edit: when I say "nefarious" I mean you can use that tech to impersonate someone (eg. political reason) but for my case it's more the creeper type cloning someone for personal use eg. Replika Tangent, the holo vtubers industry is interesting since they build up these characters with some unique persona/theme and then people follow that specific model, they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing
  - GaggiX 2y ago
    >they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing The most popular vtuber on Twitch is an AI tho
    - ge96 2y ago
      You talking NeuroSama? I haven't kept up with it in a bit I'm not sure if that's truly AI since the Turtle drives her Edit: if the source was open I'd believe it
      - GaggiX 2y ago
        >I'm not sure if that's truly AI It has always been a LLM. There is no human typing at insane speed to the TTS.
        ge96 2y ago
        I'm referring to live interception of messages which I guess has to be done to be compliant with Twitch's terms -- there is a human there edit: but yeah the fact that so many people interact with her shows generated content can keep people occupied
      - deleted 2y ago
        [deleted]

by ge96 2y ago

ilaksh 2y ago

Replicate has Hunyuan video training now. https://replicate.com/blog/fine-tune-video https://replicate.com/blog/fine-tune-video Also, Kling 1.6 Elements works pretty okay if you use the same person/face for each element. Kling also has lip sync. Or this lip sync with replicate: https://replicate.com/bytedance/latentsync https://replicate.com/bytedance/latentsync Or there is HeyGen or D-ID or Synthesia, or tavus.io for full interactive digital twins.

ge96 2y ago
Thanks

timdiggerm 2y ago

Why do you want to do that?

ge96 2y ago
My case is not directly nefarious, for example an old popular YouTuber that streamed in the early 2000s taking their content and making a model of them for personal use like a 3D chat bot but with that person's quirks Edit: when I say "nefarious" I mean you can use that tech to impersonate someone (eg. political reason) but for my case it's more the creeper type cloning someone for personal use eg. Replika Tangent, the holo vtubers industry is interesting since they build up these characters with some unique persona/theme and then people follow that specific model, they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing
- GaggiX 2y ago
  >they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing The most popular vtuber on Twitch is an AI tho
  - ge96 2y ago
    You talking NeuroSama? I haven't kept up with it in a bit I'm not sure if that's truly AI since the Turtle drives her Edit: if the source was open I'd believe it
    - GaggiX 2y ago
      >I'm not sure if that's truly AI It has always been a LLM. There is no human typing at insane speed to the TTS.
      - ge96 2y ago
        I'm referring to live interception of messages which I guess has to be done to be compliant with Twitch's terms -- there is a human there edit: but yeah the fact that so many people interact with her shows generated content can keep people occupied
    - deleted 2y ago
      [deleted]