3 ms·
Senko – Very Fast Speaker Diarization
- hamza_q_ 1y ago1 hour of audio processed in 5 seconds (RTX 4090, Ryzen 9 7950X). ~17x faster than Pyannote 3.1. On M3 MacBook Air, 1 hour in 23.5 seconds (~14x faster). This is a custom speaker diarization pipeline I've developed; it's a modified version of the pipeline found in the excellent 3D-Speaker project by Alibaba Research. My optimizations/modifications were the following: - changed VAD model - multi-threaded Fbank feature extraction - batched inference of CAM++ embeddings model - clustering is accelerated by RAPIDS, when NVIDIA GPU available Optimizations aside, massive credit needs to be given to the CAM++ speaker embeddings model, whose efficiency is where the majority of the speed comes from. This pipeline powers the Zanshin media player, which is an attempt at a usable integration of diarization in a media player. Check it out here: https://zanshin.sh https://zanshin.sh And discuss here: https://news.ycombinator.com/item?id=45104866 https://news.ycombinator.com/item?id=45104866 Let me know what you think! Were you also frustrated by how slow speaker diarization is? Does Senko's speed unlock new use cases for you? Cheers, everyone.