3 ms·
Perhaps LoRA (Low-Rank Adaptation) training techniques could be used for these types of models, like they're currently being used with LLMs and latent text-to-i
by spudlyo 3y ago
Perhaps LoRA (Low-Rank Adaptation) training techniques could be used for these types of models, like they're currently being used with LLMs and latent text-to-image diffusion models.
- operator-name 3y agoSadly looks unlikely if the base model wasn't trained on vocals. > Mitigations: Vocals have been removed from the data source using corresponding tags, and then using a state-of-the-art music source separation method, namely using the open source Hybrid Transformer for Music Source Separation (HT-Demucs). > Limitations: The model is not able to generate realistic vocals. (https://github.com/facebookresearch/audiocraft/blob/main/model_cards/MUSICGEN_MODEL_CARD.md https://github.com/facebookresearch/audiocraft/blob/main/mod...) I suspect this was a combination of playing it safe and that the model isn't well architected to reproduce meaningful vocals.