3 ms·
Could you use something like this to summarize the content of a video? Or would you first need to extract the images from the video (like 1 fps, with ffmpeg) an
by ggregoire 4y ago
Could you use something like this to summarize the content of a video? Or would you first need to extract the images from the video (like 1 fps, with ffmpeg) and tell the AI to analyze all the images and summarize them like if it was a video?
- all2 4y agoI'd do this using both streams of information: audio and video. I'd segment the audio semantically based on the topic of discussion, and I'd segment the video based on editing, subjects in scene, etc. We could start simply and just have a "timestamp": [ subjects, in, frame] key-value. It'd take some fiddling to sort how to mesh these two streams of data back together. The first thing I'd try is segment by time chunks (the resolution of which would depend on min/max segment lengths in video and audio streams) and then clump the time chunks together based on audio+video content.