4 ms·

I feel like there are three issues at play here: -Using her name to describe/advertise the fine-tuned model. -Using her illustrations to fine-tune the model.

by Imnimo 4y ago

I feel like there are three issues at play here: -Using her name to describe/advertise the fine-tuned model. -Using her illustrations to fine-tune the model. -Using a larger body of potentially unlicensed images to train the base model. For the first, if we had decided that the other steps were fair use or whatever, would it be better or worse if the fine-tuned model had been made available with no mention of the identity of the author of the training images? I'm not sure. For the second, there is surely a limit after which this sort of thing becomes unambiguously unacceptable. Suppose you fine-tune so aggressively on a small dataset that eventually the model simply reproduces exactly the training images. Now you're obviously violating copyright. But where exactly is line before that? If I have a base model that was trained on fully licensed images, and I make one single gradient descent step using a copyrighted image, making imperceptible changes to the model's output, surely the resulting images not suddenly in violation. It seems to me that the standard should be that if a human were to draw the output by hand after looking at the training images, would we consider it a violation of copyright? As a thought experiment, imagine someone who lacks the ability to draw but can instead hand-write the weights of a neural network to produce the desired output - it shouldn't matter which process they use. For the third, what if I spent a long time prompt engineering on a model trained entirely on properly licensed data and was able to generate a prompt format that produced the outputs we see from this fine-tuned model? In other words, for any generative model, there is a space of reachable outputs, and it's not so clear that these images did not already lie in that space before fine-tuning.

theptip 4y ago
I think this is the right way of chopping the problem space. In this case, the artist expressed a preference to not have her name used, which seems reasonable, and the author renamed the repo to accommodate that. One could easily imagine the opposite scenario, where the artist objects to not being credited. We are in a weird stage where the indexes into this style space are quite weird and ad-hoc; look at all the prompt engineering black magic. It’s worth noting though that this problem only showed up because there weren’t many examples of this style in the data set; when every frame of every Disney animation is in there, it won’t need you to refer to one artist’s name. I do wonder if a PR and UI improvement would be to layer a language model on top to build the prompt, or perhaps even bake this into the model itself, so you can use more generic (and possibly iterative) style descriptions instead of having to refer to an artist by name. Basically use AI to solve the prompt engineering problem too.
doodlebugging 4y ago
> In other words, for any generative model, there is a space of reachable outputs, and it's not so clear that these images did not already lie in that space before fine-tuning. I'm trying to understand what you are saying here. It sounds like you are saying that something that doesn't yet exist must already exist simply because a model has been created which can yield this non-existent thing as an output? Your example uses properly licensed data as an input but seems to imply that creating a model that is capable of producing a particular output means that the output in effect already exists as a prior work before it has ever been created by virtue of the model's ability to create it on command. I'm probably over-thinking all this.
- Imnimo 4y ago
  Well, at some point it's like the monkeys writing shakespeare. Instead of a complicated neural network, we could have written a program that just outputs random pixel values. We'd definitely be able to find all of the outputs of Stable Diffusion, as well as all of the works of the original artists coming out of that program. It'd just take us a lot of waiting and watching. I don't know that that's enough to constitute "prior art" in any meaningful sense, though. The way I look at it is that the function of Stable Diffusion or any other diffusion model is to pare down the output space of the "random pixel machine". It learns which regions of image space are likely and which are unlikely, and so when you sample an image, you tend to get ones that people like rather than random noise. You could imagine an idealized diffusion process whose output space is truly continuous and which assigns non-zero probability to the entire image space (all possible pixel arrangements), with higher probability assigned to "good" regions, and lower probability assigned to "bad" regions. If I sample from such a model repeatedly, I will eventually (in the mathematical sense of probability 1 in the limit) get out an image that looks exactly like a Hollie Mengert illustration, even if the model has never seen one during training. I'll even eventually get out an image that looks exactly like an illustration that an unborn artist will create in the year 2047. Now, in practice, it's a little less clear. Are there regions of the image space that Stable Diffusion assigns exactly zero probability, such that no amount of sampling will ever generate that image? Are there real, "good" images that Stable Diffusion assigns very low probability such that generating them is no better than sampling from the "random pixel machine"?
lbotos 4y ago
> But where exactly is line before that? Unfortunately the line of "fair use" is very blurry and only gets clarified in specific instances, with lawyers and humans debating if there was damage done. It's extremely frustrating and these advances are going to pressure fair use for sure. People will generate infringing work with AI and people will generate newly copyrightable work with AI. The danger is, as an artist, it's hard to trust if the model has given you something that is copyrighted as you have not necessarily seen it before. (You the human can cite your references, and it's possible you were subliminally influenced, but with SD, generic prompts may give you "copyrighted details" that you don't know.)
- Imnimo 4y ago
  Yeah, definitely. It's more starkly clear with Copilot, where you can point to verbatim reproduction of code. And it probably helps Stable Diffusion that images are fuzzier, so you're less likely to generate something that's close enough to be a copyright violation, but there's really no way to be certain that your output is not a memorized training example, even if it's statistically very unlikely.
s1artibartfast 4y ago
Relevant to #2 https://news.ycombinator.com/item?id=33424817 https://news.ycombinator.com/item?id=33424817
krinchan 4y ago
I mean, they're literally out here training models on Disney's copyrighted works. People seem to miss the point that the copyright ambiguity can work both ways and the courts are almost always on the side of corporate copyright holders. I posit with a few good NSFW scandals so the legislation can be dressed up as protecting the children, Disney will get the legislative intervention it wants re: copyright and stable diffusion. I realize that seems pretty US-centric but it's surprising the ways Disney can reach internationally to protect it's IP. It'll be interesting to see if Stable Diffusion ends up relegated to the parts of the Internet outside of the big three "jurisdictions" so to speak: US, EU, and China.

by Imnimo 4y ago

theptip 4y ago

I think this is the right way of chopping the problem space. In this case, the artist expressed a preference to not have her name used, which seems reasonable, and the author renamed the repo to accommodate that. One could easily imagine the opposite scenario, where the artist objects to not being credited. We are in a weird stage where the indexes into this style space are quite weird and ad-hoc; look at all the prompt engineering black magic. It’s worth noting though that this problem only showed up because there weren’t many examples of this style in the data set; when every frame of every Disney animation is in there, it won’t need you to refer to one artist’s name. I do wonder if a PR and UI improvement would be to layer a language model on top to build the prompt, or perhaps even bake this into the model itself, so you can use more generic (and possibly iterative) style descriptions instead of having to refer to an artist by name. Basically use AI to solve the prompt engineering problem too.

doodlebugging 4y ago

> In other words, for any generative model, there is a space of reachable outputs, and it's not so clear that these images did not already lie in that space before fine-tuning. I'm trying to understand what you are saying here. It sounds like you are saying that something that doesn't yet exist must already exist simply because a model has been created which can yield this non-existent thing as an output? Your example uses properly licensed data as an input but seems to imply that creating a model that is capable of producing a particular output means that the output in effect already exists as a prior work before it has ever been created by virtue of the model's ability to create it on command. I'm probably over-thinking all this.

Imnimo 4y ago
Well, at some point it's like the monkeys writing shakespeare. Instead of a complicated neural network, we could have written a program that just outputs random pixel values. We'd definitely be able to find all of the outputs of Stable Diffusion, as well as all of the works of the original artists coming out of that program. It'd just take us a lot of waiting and watching. I don't know that that's enough to constitute "prior art" in any meaningful sense, though. The way I look at it is that the function of Stable Diffusion or any other diffusion model is to pare down the output space of the "random pixel machine". It learns which regions of image space are likely and which are unlikely, and so when you sample an image, you tend to get ones that people like rather than random noise. You could imagine an idealized diffusion process whose output space is truly continuous and which assigns non-zero probability to the entire image space (all possible pixel arrangements), with higher probability assigned to "good" regions, and lower probability assigned to "bad" regions. If I sample from such a model repeatedly, I will eventually (in the mathematical sense of probability 1 in the limit) get out an image that looks exactly like a Hollie Mengert illustration, even if the model has never seen one during training. I'll even eventually get out an image that looks exactly like an illustration that an unborn artist will create in the year 2047. Now, in practice, it's a little less clear. Are there regions of the image space that Stable Diffusion assigns exactly zero probability, such that no amount of sampling will ever generate that image? Are there real, "good" images that Stable Diffusion assigns very low probability such that generating them is no better than sampling from the "random pixel machine"?

lbotos 4y ago

> But where exactly is line before that? Unfortunately the line of "fair use" is very blurry and only gets clarified in specific instances, with lawyers and humans debating if there was damage done. It's extremely frustrating and these advances are going to pressure fair use for sure. People will generate infringing work with AI and people will generate newly copyrightable work with AI. The danger is, as an artist, it's hard to trust if the model has given you something that is copyrighted as you have not necessarily seen it before. (You the human can cite your references, and it's possible you were subliminally influenced, but with SD, generic prompts may give you "copyrighted details" that you don't know.)

Imnimo 4y ago
Yeah, definitely. It's more starkly clear with Copilot, where you can point to verbatim reproduction of code. And it probably helps Stable Diffusion that images are fuzzier, so you're less likely to generate something that's close enough to be a copyright violation, but there's really no way to be certain that your output is not a memorized training example, even if it's statistically very unlikely.

s1artibartfast 4y ago

Relevant to #2 https://news.ycombinator.com/item?id=33424817 https://news.ycombinator.com/item?id=33424817

krinchan 4y ago

I mean, they're literally out here training models on Disney's copyrighted works. People seem to miss the point that the copyright ambiguity can work both ways and the courts are almost always on the side of corporate copyright holders. I posit with a few good NSFW scandals so the legislation can be dressed up as protecting the children, Disney will get the legislative intervention it wants re: copyright and stable diffusion. I realize that seems pretty US-centric but it's surprising the ways Disney can reach internationally to protect it's IP. It'll be interesting to see if Stable Diffusion ends up relegated to the parts of the Internet outside of the big three "jurisdictions" so to speak: US, EU, and China.