3 ms·
These look.. great, by and large. Hands are super natural, coherency is really high. Showing off piano chord blocking is a huge flex. I’d like to play with thi
by vessenes 2y ago
These look.. great, by and large. Hands are super natural, coherency is really high. Showing off piano chord blocking is a huge flex.
I’d like to play with this! No code, but bytedance often releases models, so I’m hopeful. It’s significantly better than vasa, and looks likely to be an iteration of that architecture.
- liuliu 2y agoByteDance didn't release their text-to-video model, which is the base of this work, so I would think unlikely.
- echelon 2y agoTencent is releasing a ton of stuff though! https://aivideo.hunyuan.tencent.com/ https://aivideo.hunyuan.tencent.com/ Github is overflowing with Tencent, Alibaba, and Ant Group models. Typically licensed as Apache 2, and replete with pretrained weights and fine tuning scripts.
- liuliu 2y agoThe training process in OmniHuman-1 seems to be straightforward to replicate once Tencent releases their image-to-video model too.
- echelon 2y agoT2V is already I2V if you're enterprising enough to open up the model and play with the latents. The I2V modality is almost just a trick.
- liuliu 2y agoYes, the Llava model can encode image, and you can encode image into 3D vae space. Without fine-tune the model though, you are not going to have fidelity to original (if only use Llava's SigLIP to encode), or end up with image with limited motion (3D vae encoded latents as the first frame then doing vid2vid).
- tkgally 2y agoThe hands do look good, but the spacing of the black and white keys on the piano keyboard is off. Similar problem with the strings and fretboard on the guitar. Such glitches won’t be noticed by as many people as weird hands are, though.
- gavinguang 2y agohttps://omnihuman1.org/ https://omnihuman1.org/ try it