4 ms·

Wonder how far off the whole "generate music based on your existing music library" thing is going to be? That'll make musicians happy with big tech as well, ju

Wonder how far off the whole "generate music based on your existing music library" thing is going to be? That'll make musicians happy with big tech as well, just like artists are. *sigh*

zitterbewegung 3y ago
Why not do generate music you like which wouldn’t need you to upload your library and would have RLHF baked in.
- ElFitz 3y ago
  Something like the algorithm TikTok uses. First probing by offering a variety of content that should match based on what little information you have on the user (ip location, locale, etc). Then use the user’s action to iteratively refine your classification, until you end up with something tailor-made.
  - zitterbewegung 3y ago
    Uh more like Reddit with up and down and also how long you listen.
LewisVerstappen 3y ago
The Record labels are far, far more litigious than the art community.
- PaulDavisThe1st 3y ago
  They can't litigate a person doing this at home, and never redistributing. I suppose they might try, anyway.
  - kmeisthax 3y ago
    The RIAA pioneered copyright enforcement at the individual level back in the 2000s, they absolutely would try to sue downstream AudioCraft users.
    - throwaway290 3y ago
      They should start with AudioCraft itself, conceptually it's derivative work and it doesn't matter if it's "open source" or not. Try throwing in someone's sample in a song and publish it saying "no copyright infringement intended and I totally don't make any money from it"... If it becomes popular, see how long it stays up until DMCA takedown. And we know this dataset is already popular.
      - PaulDavisThe1st 3y ago
        > and publish it This is precisely the opposite of the context I was remarking on.
        throwaway290 3y ago
        AudioCraft itself is published. That's the context I am remarking on.
    - PaulDavisThe1st 3y ago
      [flagged]
      - manquer 3y ago
        Even before streaming, You never "owned" any music legally [1], you merely owned a physical copy of a performance[3] of a song, that in no way gives you the right to make derivative works [2] automatically. Also it doesn't really matter on what the law says, RIAA in the last iteration, relied on the fact it you would rather pay a fine than be able to afford expensive lawyers to fight the specifics out in court on average. It was always about disproportionate ability to bring resources against individual "offenders" to create fear among everyone to deter "undesirable" forms of copying, not necessarily what the legal protections were. --- [1] Unless you specifically commissioned it under a contract which gave you the right [2] See recent cases including those related to Kris Kashtanova and Andy Warhol. [3] Not the song, just the performance, aka Taylor Swift version, for good explanation of how the rights are divvied up in the music industry a Planet Money series covers it well https://www.npr.org/sections/money/2022/10/29/1131927591/inflation-song-music-label-earnest-jackson https://www.npr.org/sections/money/2022/10/29/1131927591/inf...
  - jsheard 3y ago
    Is training a "pirate model" something you'd reasonably be able to do at home though, given the compute requirements? The analogous "image generation at home" is only possible due to a for-profit entity with significant resources choosing to (a) play fast-and-loose with the provenance of their training set and (b) giving away the resulting model for free, if the open source community had to train their models from scratch then as best as I can tell they would still be stuck in the dark ages generating vague goopy abominations.
    - throwuwu 3y ago
      You could take a model trained on CC content and then fine tune it on copyrighted material cheaply and quickly
    - PaulDavisThe1st 3y ago
      Currently, yes, available compute power @ home does indeed seem like a limitation. Whether that remains true going forward seems a little unclear to me.
spudlyo 3y ago
Perhaps LoRA (Low-Rank Adaptation) training techniques could be used for these types of models, like they're currently being used with LLMs and latent text-to-image diffusion models.
- operator-name 3y ago
  Sadly looks unlikely if the base model wasn't trained on vocals. > Mitigations: Vocals have been removed from the data source using corresponding tags, and then using a state-of-the-art music source separation method, namely using the open source Hybrid Transformer for Music Source Separation (HT-Demucs). > Limitations: The model is not able to generate realistic vocals. (https://github.com/facebookresearch/audiocraft/blob/main/model_cards/MUSICGEN_MODEL_CARD.md https://github.com/facebookresearch/audiocraft/blob/main/mod...) I suspect this was a combination of playing it safe and that the model isn't well architected to reproduce meaningful vocals.
deleted 3y ago
[deleted]
chrisjj 3y ago
Anyone having listened to this MusicGen's output samples would surely answer "a million miles". Seriously, you couldn't sell this output for a free mobile clicker game.
Workaccount2 3y ago
Ugh, I dread having to listen to everyone's hyper personal music because they swear up and down to the point of tears that "IT'S THE BEST SONG EVER CREATED! EVER!!!", while the constantly prod for you to affirm how amazing the song is. Bruh, music is subjective as hell, and I can already tell I hate this song.

Wonder how far off the whole "generate music based on your existing music library" thing is going to be? That'll make musicians happy with big tech as well, just like artists are. *sigh*

zitterbewegung 3y ago

Why not do generate music you like which wouldn’t need you to upload your library and would have RLHF baked in.

ElFitz 3y ago
Something like the algorithm TikTok uses. First probing by offering a variety of content that should match based on what little information you have on the user (ip location, locale, etc). Then use the user’s action to iteratively refine your classification, until you end up with something tailor-made.
- zitterbewegung 3y ago
  Uh more like Reddit with up and down and also how long you listen.

LewisVerstappen 3y ago

The Record labels are far, far more litigious than the art community.

PaulDavisThe1st 3y ago
They can't litigate a person doing this at home, and never redistributing. I suppose they might try, anyway.
- kmeisthax 3y ago
  The RIAA pioneered copyright enforcement at the individual level back in the 2000s, they absolutely would try to sue downstream AudioCraft users.
  - throwaway290 3y ago
    They should start with AudioCraft itself, conceptually it's derivative work and it doesn't matter if it's "open source" or not. Try throwing in someone's sample in a song and publish it saying "no copyright infringement intended and I totally don't make any money from it"... If it becomes popular, see how long it stays up until DMCA takedown. And we know this dataset is already popular.
    - PaulDavisThe1st 3y ago
      > and publish it This is precisely the opposite of the context I was remarking on.
      - throwaway290 3y ago
        AudioCraft itself is published. That's the context I am remarking on.
  - PaulDavisThe1st 3y ago
    [flagged]
    - manquer 3y ago
      Even before streaming, You never "owned" any music legally [1], you merely owned a physical copy of a performance[3] of a song, that in no way gives you the right to make derivative works [2] automatically. Also it doesn't really matter on what the law says, RIAA in the last iteration, relied on the fact it you would rather pay a fine than be able to afford expensive lawyers to fight the specifics out in court on average. It was always about disproportionate ability to bring resources against individual "offenders" to create fear among everyone to deter "undesirable" forms of copying, not necessarily what the legal protections were. --- [1] Unless you specifically commissioned it under a contract which gave you the right [2] See recent cases including those related to Kris Kashtanova and Andy Warhol. [3] Not the song, just the performance, aka Taylor Swift version, for good explanation of how the rights are divvied up in the music industry a Planet Money series covers it well https://www.npr.org/sections/money/2022/10/29/1131927591/inflation-song-music-label-earnest-jackson https://www.npr.org/sections/money/2022/10/29/1131927591/inf...
- jsheard 3y ago
  Is training a "pirate model" something you'd reasonably be able to do at home though, given the compute requirements? The analogous "image generation at home" is only possible due to a for-profit entity with significant resources choosing to (a) play fast-and-loose with the provenance of their training set and (b) giving away the resulting model for free, if the open source community had to train their models from scratch then as best as I can tell they would still be stuck in the dark ages generating vague goopy abominations.
  - throwuwu 3y ago
    You could take a model trained on CC content and then fine tune it on copyrighted material cheaply and quickly
  - PaulDavisThe1st 3y ago
    Currently, yes, available compute power @ home does indeed seem like a limitation. Whether that remains true going forward seems a little unclear to me.

spudlyo 3y ago

Perhaps LoRA (Low-Rank Adaptation) training techniques could be used for these types of models, like they're currently being used with LLMs and latent text-to-image diffusion models.

operator-name 3y ago
Sadly looks unlikely if the base model wasn't trained on vocals. > Mitigations: Vocals have been removed from the data source using corresponding tags, and then using a state-of-the-art music source separation method, namely using the open source Hybrid Transformer for Music Source Separation (HT-Demucs). > Limitations: The model is not able to generate realistic vocals. (https://github.com/facebookresearch/audiocraft/blob/main/model_cards/MUSICGEN_MODEL_CARD.md https://github.com/facebookresearch/audiocraft/blob/main/mod...) I suspect this was a combination of playing it safe and that the model isn't well architected to reproduce meaningful vocals.

deleted 3y ago

[deleted]

chrisjj 3y ago

Anyone having listened to this MusicGen's output samples would surely answer "a million miles". Seriously, you couldn't sell this output for a free mobile clicker game.

Workaccount2 3y ago

Ugh, I dread having to listen to everyone's hyper personal music because they swear up and down to the point of tears that "IT'S THE BEST SONG EVER CREATED! EVER!!!", while the constantly prod for you to affirm how amazing the song is. Bruh, music is subjective as hell, and I can already tell I hate this song.