3 ms·
> At that point, since a "short video" is really a scene, we are basically at film stage. But silent films, no? Or where are things at as far as generated audi
by joshuahedlund 3y ago
> At that point, since a "short video" is really a scene, we are basically at film stage.
But silent films, no? Or where are things at as far as generated audio that syncs to the video? (sound effects, dialogue, music - each of which will have its own challenges)
- swatcoder 3y agoI'm in the camp of "satisfying long-watch video is a long way off", but I don't think matching audio to video will be a big hurdle. Conceptually -- if you're generating the video from a script, you can feed the script and video into a foley and voice generator that recognizes "door slammed" and inserts the kind of sound associated with a door slam and that recognizes face changes and generates speech audio keyed to match. Those tasks will be solved independently on the way to long-form video itself getting sorted because you can exercise them with non-AI video along the way. I'd be shocked if somebody hasn't already made some good demos of the foley and lip sync stuff.