3 ms·
just curious, are there any open models doing the opposite of audio synthesis? As in able to generate the stems for a song?
by 2c2c2c 3y ago
just curious, are there any open models doing the opposite of audio synthesis? As in able to generate the stems for a song?
- edude03 3y agohttps://github.com/facebookresearch/demucs https://github.com/facebookresearch/demucs
- sdenton4 3y agoHa, I was working on neutral voice compression, which consists of an encoder which creates what the kids are calling tokens these days, and a decoder which synthesizes speech from the tokens. AudioLM puts a language model on top of the compression tokens, and thus can generate speech or other audio. There's piles of recent papers pushing that approach into music generation. Mulan is a name that comes to mind. Or maybe your interested in audio separation to get at the isolated instruments? There's lots of great work on that, as well. Like MixIT, which is an unsupervised audio separation system.