4 ms·
Do you have any references for this or a link to a commercial service? I'm currently in the process or trying to extract some background voices in a video (an
by rapjr9 4y ago
Do you have any references for this or a link to a commercial service? I'm currently in the process or trying to extract some background voices in a video (an interview where the faint background conversation is in English and the loud overdub is in Bulgarian). I tried Melodyne but it seems to only separate on pitch, not volume, and the pitchs are too similar (mono, three voices, all female) and words are made of lots of short phonemes which each create a "note" and makes editing impossible. I looked into Izotope RX as well and it does not seem capable of doing this either. There are services that can automatically add subtitles using speech-to-text translators but they are expensive, and I'd prefer to have the background voices rather than the Bulgarian interpretation of them translated back into English. It seems possible to do, in Peter Jackson's The Beatles:Get Back they were able to separate voices from loud foreground musical sounds and other ambient talking, but that technology was custom and doesn't seem to be publicly available yet.
- kajecounterhack 4y agohttps://arxiv.org/abs/2110.10739 https://arxiv.org/abs/2110.10739 I haven't seen it provided as a commercial service or free model yet, but there is open source code for Mixit that lets you train using the open source / canned FUSS dataset.
- rapjr9 4y agoThanks! That led me to this which looks like a good place to start: https://github.com/google-research/sound-separation/blob/master/models/neurips2020_mixit/README.md https://github.com/google-research/sound-separation/blob/mas...