2 ms·
Now that you mention it, basic implementation is rather trivial: isolate keyframes (which is in the video data¹), aggregate the transcripts chunks (e.g. .vtt sh
by mdp2021 5y ago
Now that you mention it, basic implementation is rather trivial: isolate keyframes (which is in the video data¹), aggregate the transcripts chunks (e.g. .vtt should be available) within those keyframes according to the timestamps.
Issues: first of all, transcript quality: those of TED (Chris Anderson's "Technology Entertainment Design" conferences) are curated, those of YT are oftentimes not. Then, discriminating significant and less significant keyframes, but if one only needs "decent" instead of "perfect", good compromises for heuristic algorithms (more "parametrized procedures" or "recipes") can surely be found.
You are tempting me... In an evening one should get something already usable. Better than a presentation, I see the transcript text with aside thumbnails linking to the full-sized images. As someone who sometimes studies material from video, when I am listening or watching I am not in the strict need for text analysis, and when I am working on the text itself I do not need the video. On the other hand, if I think of the MIT OpenCourseware material and similar (Yale etc.), the "blackboard" shots can be precious to have alongside the text.
--
¹ ffmpeg -i video.mkv -vf "select=eq(pict_type\,I)" -vsync vfr -frame_pts true keyframes-%02d.jpg
- savingsPossible 5y agoplease, do create this and notify us!