3 ms·
>Matching faces to voices relies on simple co-occurence heuristic, and will not work in certain scenarios (e.g. if the whole conversation between two people is
by totetsu 4y ago
>Matching faces to voices relies on simple co-occurence heuristic, and will not work in certain scenarios (e.g. if the whole conversation between two people is recorded from a single angle)
This seems like the really hard part. maybe if there was a way to find the time lips moves for a face. or guess gender and age of both face and voice.. or If the audio is a stereo mix, using relative position
- photoGrant 4y agoThere's this that can differentiate speakers https://github.com/CorentinJ/Real-Time-Voice-Cloning https://github.com/CorentinJ/Real-Time-Voice-Cloning
- oslem 4y agoI think you are referring to GE2E [1], which is the speaker identification model used in that project. [1] https://arxiv.org/abs/1710.10467 https://arxiv.org/abs/1710.10467
- melony 4y agoThat's from an eternity ago. Speaker diarization has come a long way.
- ChrisKnott 4y agoWhat's the best pretrained model available? The best I've tried is pyannote.
- totetsu 4y agoBut matching a speaker to a face is a different problem differentiating between speech in a recording