3 ms·
I'm in the camp of "satisfying long-watch video is a long way off", but I don't think matching audio to video will be a big hurdle. Conceptually -- if you're g
by swatcoder 3y ago
I'm in the camp of "satisfying long-watch video is a long way off", but I don't think matching audio to video will be a big hurdle.
Conceptually -- if you're generating the video from a script, you can feed the script and video into a foley and voice generator that recognizes "door slammed" and inserts the kind of sound associated with a door slam and that recognizes face changes and generates speech audio keyed to match. Those tasks will be solved independently on the way to long-form video itself getting sorted because you can exercise them with non-AI video along the way.
I'd be shocked if somebody hasn't already made some good demos of the foley and lip sync stuff.