3 ms·

I'm hooked on Stable Diffusion. It's the most impressive sudden leaps in technology in my 30 or so years of being old enough to understand it. Much of the power

I'm hooked on Stable Diffusion. It's the most impressive sudden leaps in technology in my 30 or so years of being old enough to understand it. Much of the power it has, is based on the material it is trained on, and I'm very appreciative of the work and effort that was made in order to make it what it is. And, it's only getting started. That said... I get it. Huggingface are working on a diffusion based model for music. And guess what? It's extremely important for them to only use opt-in or commissioned training data. Why? Because the music industry, unlike artists on artstation and the likes, have a lot of lawyers and can protect their copyrighted works. Why should it really be any different for visual arts? I honestly don't see why. That isn't to say I'm not going to keep on using stable diffusion, nor do I think there is anything that can be done to stop it. But, I do think artists should be compensated, and such models should not be based on the work of anyone, unless they want it to be.

jeroenhd 4y ago
Monetary risk, that's why. Cross the wrong people and you'll find yourself extradited to another country and tried for copyright laws you didn't even know existed. I'm honestly surprised OpenAI models know about Disney properties.
- okamiueru 4y ago
  No. I get why. I'm just saying that it's morally equivalent to use copyrighted music to train AI generated music, as it is to use copyrighted art to generate art. > Dance Diffusion is also built on datasets composed entirely of copyright-free and voluntarily provided music and audio samples. Because diffusion models are prone to memorization and overfitting, releasing a model trained on copyrighted data could potentially result in legal issues. In honoring the intellectual property of artists while also complying to the best of their ability with the often strict copyright standards of the music industry, keeping any kind of copyrighted material out of training data was a must. Reads very different to any statement made by Hugginface or the LION database regarding stable diffusion, which do not mention the concept of an artist or artist work a single time. And Hugginface are the only ones open about it. Midjourney and Dall-E and Imagen most assuredly doing the same for their black boxes.
heisenbit 4y ago
Music business has for reasons connected to broadcast, phonograph and CD been extremely profitable. Less in the streaming world but still ok. This powers a lot of lawyers for the next decades. Visual artists earned a lot less.
GaggiX 4y ago
Where do you read that Huggingface is training a music model? I think you're confusing it with Harmonai.
- okamiueru 4y ago
  You're right. I was confusing them. The disconnect between the discussion and approach to training music models and image models still applies, although the hypocrisy I implied does not. Harmonai are approaching it sensibly. Huggingface and most other image diffusion models developers are not.
  - rockemsockem 4y ago
    You might be interested to know then that Harmonai and Stability AI (creators of stable diffusion) are tightly connected. I think Harmonai is actually part of stability, but it's a bit murky. https://twitter.com/StabilityAI/status/1605012677188718592?t=wI5EBm4usAUTi9QuGrgyAg&s=19 https://twitter.com/StabilityAI/status/1605012677188718592?t...
BeFlatXIII 4y ago
The music industry should be like visual arts, not the other way around. Do we want more inane lawsuits based on vague similarities?
- okamiueru 4y ago
  No, no, I agree about the insane lawsuits for similar sounding music. The popular music is mostly regurgitated stuff anyways. But the equivalent here would be "Penny Lane as if composed by Michael Jackson and sung by Taylor Swift". My example is bad, because I stopped following popular music a decade or two ago. But replace the deceased artists with living ones. My point probably should be made clearer. You know how you can say "in the art style of X"? Well, it doesn't matter if Huggingface made that possible. You can, relatively easily, train that concept with a collection of paintings by X. Then you can go ahead an make art in their style. Now, from a technology point of view, that is nothing short of amazing. I still cannot get my head around how absolutely ridiculously powerful it is. And, even the people who play with this, don't seem to fully grasp it either. The world will change in the next 4 years. What I'm wondering, is, what should we, as a society, find acceptable? Why should someone be able to train "in the style of X" where X is a set of EVERYTHING, and make money of it, without the say of X, or even them getting anything for it? Have you checked the evaluation of Huggingface? It's in the order of 1-10 billion USD. There is definitely the argument of anti-copyright. I get that part too. But there is definitely someone who will end up with the bigger stick, and it isn't the people holding the paint brushes who made it possible. That seems just a little bit unfair, and perhaps unwise. Also, I'll end with a point that no one so far has brought up, even though I've followed the discussion both for and against AI. Which is "whitewashing" art. Now, the example I'm going to show isn't very good, but I also spent 5 minutes on this. Where would you draw the line on when Alexander Wild no longer has copyright over his photography? https://imgur.com/a/cm6IrzG https://imgur.com/a/cm6IrzG
  - BeFlatXIII 4y ago
    Thank you for articulating the split between the users of AI models and the owners of the models. I like using the models in the manner of Marcel Duchamp’s readymades and from that perspective, all the arguments about artists’ moral rights are nothing more than regulatory mumbo-jumbo and sour grapes. However, I do strongly agree that there is something morally icky about companies such as “Open”AI and HuggingFace making bank off the training data they’ve scraped. That’s why I prefer Stable Diffusion and its clones, as it lets us run it locally without someone making money in a pay-per-prompt (or, worse, monthly subscription) model. Democratize AI to anyone with a strong enough computer or don’t do it at all.

jeroenhd 4y ago

Monetary risk, that's why. Cross the wrong people and you'll find yourself extradited to another country and tried for copyright laws you didn't even know existed. I'm honestly surprised OpenAI models know about Disney properties.

okamiueru 4y ago
No. I get why. I'm just saying that it's morally equivalent to use copyrighted music to train AI generated music, as it is to use copyrighted art to generate art. > Dance Diffusion is also built on datasets composed entirely of copyright-free and voluntarily provided music and audio samples. Because diffusion models are prone to memorization and overfitting, releasing a model trained on copyrighted data could potentially result in legal issues. In honoring the intellectual property of artists while also complying to the best of their ability with the often strict copyright standards of the music industry, keeping any kind of copyrighted material out of training data was a must. Reads very different to any statement made by Hugginface or the LION database regarding stable diffusion, which do not mention the concept of an artist or artist work a single time. And Hugginface are the only ones open about it. Midjourney and Dall-E and Imagen most assuredly doing the same for their black boxes.

heisenbit 4y ago

Music business has for reasons connected to broadcast, phonograph and CD been extremely profitable. Less in the streaming world but still ok. This powers a lot of lawyers for the next decades. Visual artists earned a lot less.

GaggiX 4y ago

Where do you read that Huggingface is training a music model? I think you're confusing it with Harmonai.

okamiueru 4y ago
You're right. I was confusing them. The disconnect between the discussion and approach to training music models and image models still applies, although the hypocrisy I implied does not. Harmonai are approaching it sensibly. Huggingface and most other image diffusion models developers are not.
- rockemsockem 4y ago
  You might be interested to know then that Harmonai and Stability AI (creators of stable diffusion) are tightly connected. I think Harmonai is actually part of stability, but it's a bit murky. https://twitter.com/StabilityAI/status/1605012677188718592?t=wI5EBm4usAUTi9QuGrgyAg&s=19 https://twitter.com/StabilityAI/status/1605012677188718592?t...

BeFlatXIII 4y ago

The music industry should be like visual arts, not the other way around. Do we want more inane lawsuits based on vague similarities?

okamiueru 4y ago
No, no, I agree about the insane lawsuits for similar sounding music. The popular music is mostly regurgitated stuff anyways. But the equivalent here would be "Penny Lane as if composed by Michael Jackson and sung by Taylor Swift". My example is bad, because I stopped following popular music a decade or two ago. But replace the deceased artists with living ones. My point probably should be made clearer. You know how you can say "in the art style of X"? Well, it doesn't matter if Huggingface made that possible. You can, relatively easily, train that concept with a collection of paintings by X. Then you can go ahead an make art in their style. Now, from a technology point of view, that is nothing short of amazing. I still cannot get my head around how absolutely ridiculously powerful it is. And, even the people who play with this, don't seem to fully grasp it either. The world will change in the next 4 years. What I'm wondering, is, what should we, as a society, find acceptable? Why should someone be able to train "in the style of X" where X is a set of EVERYTHING, and make money of it, without the say of X, or even them getting anything for it? Have you checked the evaluation of Huggingface? It's in the order of 1-10 billion USD. There is definitely the argument of anti-copyright. I get that part too. But there is definitely someone who will end up with the bigger stick, and it isn't the people holding the paint brushes who made it possible. That seems just a little bit unfair, and perhaps unwise. Also, I'll end with a point that no one so far has brought up, even though I've followed the discussion both for and against AI. Which is "whitewashing" art. Now, the example I'm going to show isn't very good, but I also spent 5 minutes on this. Where would you draw the line on when Alexander Wild no longer has copyright over his photography? https://imgur.com/a/cm6IrzG https://imgur.com/a/cm6IrzG
- BeFlatXIII 4y ago
  Thank you for articulating the split between the users of AI models and the owners of the models. I like using the models in the manner of Marcel Duchamp’s readymades and from that perspective, all the arguments about artists’ moral rights are nothing more than regulatory mumbo-jumbo and sour grapes. However, I do strongly agree that there is something morally icky about companies such as “Open”AI and HuggingFace making bank off the training data they’ve scraped. That’s why I prefer Stable Diffusion and its clones, as it lets us run it locally without someone making money in a pay-per-prompt (or, worse, monthly subscription) model. Democratize AI to anyone with a strong enough computer or don’t do it at all.