3 ms·
Neat project, you may want to consider adding audio processing too (eg sound detected) as part of the video. You could go deeper and compare samples of audio t
by plasma 5y ago
Neat project, you may want to consider adding audio processing too (eg sound detected) as part of the video.
You could go deeper and compare samples of audio that could be uploaded separately (eg, siren sounds), check out MFCC processing https://en.wikipedia.org/wiki/Mel-frequency_cepstrum#Applications https://en.wikipedia.org/wiki/Mel-frequency_cepstrum#Applica... to do Shazam-style audio comparison.
I wonder too if you process it on a per-frame basis or you can take series of frames too (eg analyze the last 5 seconds of frames) to detect things like a "hand wave".
- mvoodarla 5y agoThanks for the comment! Audio is definitely an interesting angle. We haven't spoken to a lot of folks yet that want that feature, but I'd guess some productivity tools that let you rewatch Zoom calls, baby monitoring, media playback, or a few other things like that might mind audio useful. We'll explore it for sure. We currently process on the per-frame basis but have the optionality to increase the window size of 1, to any arbitrary size like 300 frames. This is what allows us to currently detect "actions" for some of our customers. Of course, it's not the same as a sliding window but it's something we're considering.
- parkerhiggins 5y agoBingo. In enterprise environments audio is vastly more valuable than video for searching internal meeting and discussion recordings. Often a recording title is not enough.