3 ms·
I use ffmpeg to run a fun twitch stream VOD to highlight reel pipeline. Example with explicit language: https://www.youtube.com/watch?v=ETR3IXyGgEo https://www.
by loser777 6y ago
I use ffmpeg to run a fun twitch stream VOD to highlight reel pipeline. Example with explicit language: https://www.youtube.com/watch?v=ETR3IXyGgEo https://www.youtube.com/watch?v=ETR3IXyGgEo.
ffmpeg handles everything from frame-extraction (to feed a deep learning model), audio spectrogram calculation (also features for the model), video trimming (to cut the interesting clips), video concatenation (to join the clips), and the text overlay
- memexy 6y agoWhat is the deep learning model and what does it do?
- loser777 6y agoIt's just a standard vision convnet like ResNet-18, or ResNet-50. It gets fed facecam with an audio spectrogram concatenated to it (pretty hacky, but seems to help). All it does is binary prediction of {interesting, not interesting}, and I use some heuristics to pick regions of video based on how many frames were "interesting" to the model. Feel free to take a look at the (research quality at best) code: https://github.com/eqy/autotosis https://github.com/eqy/autotosis
- memexy 6y agoThanks.