4 ms·

I let myself get mildly excited with the last Omni release, but it turns out it (and this one) can't do the one practical thing I want - Sync generated video to

I let myself get mildly excited with the last Omni release, but it turns out it (and this one) can't do the one practical thing I want - Sync generated video to provided pre-existing audio. Meanwhile, I'm happily using Minimax H3 locally on my 12Gb 4070RTX to finally finish the lip syncing to recorded dialog on my abandoned 20 year old Flash animation hobby projects.

msabalau 1mo ago
Yeah, that'd be a great feature.
jarjoura 1mo ago
Pretty sure I heard one of the PMs in a podcast a few weeks ago say they are intentionally not building support for it out of concerns of enabling deep-fakes. I agree though. My issue is the cost for using AI video models is way too high for anyone not building anything serious with them, at the same time they are too restricted for actually using professionally. Prompting them with text to get something generated is cute, but then you just end up creating slop that everyone hates, ultimately devaluing the power of these things.
- doctorpangloss 1mo ago
  Deepfakes are kind of the point though.
KolmogorovComp 1mo ago
> Minimax H3 locally on my 12Gb 4070RTX Minimax H3 is about 240Gb alone, how do you do? How much quantised is it, and how good are the results?
- vunderba 1mo ago
  Not op, but where are you getting that number from? Even the full 16-bit precision model is only about 66GB. Most people are running the stock release of Minimax H3 using the INT8 quant and it's about ~20GB. https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffus... https://docs.comfy.org/tutorials/video/minimax/minimax-h3 https://docs.comfy.org/tutorials/video/minimax/minimax-h3
- Guillaume86 1mo ago
  I'm running it on my 8 years old 2080ti 11 gb VRAM + 32 gb ram. After tasking fable to optimize the setup for a few nights it's pretty good for some quick funny clips (2min30 for 5s , 5mins for 10s, with reference pics to insert anyone in the clip, not great quality but acceptable for some fun on a phone). I haven't had as much fun with Gen AI since the Stable Diffusion days.
- Nihilartikel 1mo ago
  The int8 release is very capable and runs acceptably on consumer gpus

msabalau 1mo ago

Yeah, that'd be a great feature.

jarjoura 1mo ago

Pretty sure I heard one of the PMs in a podcast a few weeks ago say they are intentionally not building support for it out of concerns of enabling deep-fakes. I agree though. My issue is the cost for using AI video models is way too high for anyone not building anything serious with them, at the same time they are too restricted for actually using professionally. Prompting them with text to get something generated is cute, but then you just end up creating slop that everyone hates, ultimately devaluing the power of these things.

doctorpangloss 1mo ago
Deepfakes are kind of the point though.

KolmogorovComp 1mo ago

> Minimax H3 locally on my 12Gb 4070RTX Minimax H3 is about 240Gb alone, how do you do? How much quantised is it, and how good are the results?

vunderba 1mo ago
Not op, but where are you getting that number from? Even the full 16-bit precision model is only about 66GB. Most people are running the stock release of Minimax H3 using the INT8 quant and it's about ~20GB. https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffusion_models https://huggingface.co/Comfy-Org/MiniMax-H3/tree/main/diffus... https://docs.comfy.org/tutorials/video/minimax/minimax-h3 https://docs.comfy.org/tutorials/video/minimax/minimax-h3
Guillaume86 1mo ago
I'm running it on my 8 years old 2080ti 11 gb VRAM + 32 gb ram. After tasking fable to optimize the setup for a few nights it's pretty good for some quick funny clips (2min30 for 5s , 5mins for 10s, with reference pics to insert anyone in the clip, not great quality but acceptable for some fun on a phone). I haven't had as much fun with Gen AI since the Stable Diffusion days.
Nihilartikel 1mo ago
The int8 release is very capable and runs acceptably on consumer gpus