20 ms·
Haven’t tried this demo, but in my experience these open-source models that split music into four components work absolutely fantastically. Not quite perfectly
by comex 3y ago
Haven’t tried this demo, but in my experience these open-source models that split music into four components work absolutely fantastically. Not quite perfectly — if you remove vocals, the remaining track may have faint echos of vocals - but astoundingly well compared to the state of the art just 5 years ago or so.
However… what if you want more than four components? What if you want to split a complex arrangement into a separate component for each individual instrument? Does anyone know of any interesting research in this area?
- jononor 3y agoThe primary challenge is the availability of sufficiently large datasets with many separate stems/tracks. Those that find a way to build that, will have the ability to churn out such a model using existing architectures. The companies doing AI mixing have a huge advantage in this area.
- gnabgib 3y agoGave it a shot - as the webpage warns, it does take a bit off time. The results are indeed impressive, although the separation isn't perfect.. each division (drum/bass/vocals) has slight echos of each other (bass line occasionally includes a slight muffled vocal, vocals include some snare, hi-hat, and strings). It's a great starting point though.
- sevagh 3y agoDemucs [1], one of the leading/SOTA systems, has an experimental 6-source model, `htdemucs_6s`, which adds piano and guitar: >We are also releasing an experimental 6 sources model, that adds a guitar and piano source. Quick testing seems to show okay quality for guitar, but a lot of bleeding and artifacts for the piano source. I also believe Audioshake [2] (a company in the space) is doing guitar separation as well. 1: https://github.com/facebookresearch/demucs https://github.com/facebookresearch/demucs 2: https://www.audioshake.ai/ https://www.audioshake.ai/