4 ms·
This sounds like a great ml app idea: Remove music from audio while keeping the other voices. I wonder how hard it would be to keep them separate.
by calgoo 5y ago
This sounds like a great ml app idea: Remove music from audio while keeping the other voices. I wonder how hard it would be to keep them separate.
- MereInterest 5y agoSpleeter [1] does a really good job of it. It can separate out the vocals, accompaniment, drums, bass, and piano. It does a really phenomenal job at it, though there can be a bit of reverb in the vocals if there's a lot of reverb in the base. Splitting out individual vocalists is a lot harder, and I haven't found any pre-trained models that manage it. There's a lot of speaker separation [2] models, but they are trained on separate sentences overlapping. [1] https://github.com/deezer/spleeter/releases https://github.com/deezer/spleeter/releases [2] https://paperswithcode.com/task/speaker-separation https://paperswithcode.com/task/speaker-separation
- nmstoker 5y agoI immediately thought of spleeter too as I've had great success in experiments to strip out unwanted music overlays (eg podcasts that insist on adding their end of show theme tune whilst guests are still speaking!) I haven't tried speaker separation, but it seems there are some models from SpeechBrain on HuggingFace that seem to handle this: https://huggingface.co/models?filter=Speech%20Separation https://huggingface.co/models?filter=Speech%20Separation
- heavyset_go 5y agoIf you know what song is playing, then it's somewhat trivial to remove it from the audio, no ML needed.