6 ms·

Related: "Invasive Diffusion: How one unwilling illustrator found herself turned into an AI model" [0]. It's astonishingly simple to train both LoRAs and Embed

Related: "Invasive Diffusion: How one unwilling illustrator found herself turned into an AI model" [0]. It's astonishingly simple to train both LoRAs and Embeddings nowadays -- if you combine it with tech like ControlNet, you can more or less have direct control over the illustration as well (see this example I converted to Hollie Megert's style with barely any effort [1] using the model from [0]) -- but FWIW I was playing with the WhatsApp AI stickers recently and human-made illustrations still have a "je ne sais quoi" that sets them apart, vaguely reminiscent of "realistic" animation vs. "squash and stretch". I would not be worried as an artist*, but I would be worried if I am making assets for corporations in Corporate Memphis [2]. [0] https://waxy.org/2022/11/invasive-diffusion-how-one-unwilling-illustrator-found-herself-turned-into-an-ai-model/ https://waxy.org/2022/11/invasive-diffusion-how-one-unwillin... [1] https://i.imgur.com/ZwtKug5.jpeg https://i.imgur.com/ZwtKug5.jpeg [2] https://en.wikipedia.org/wiki/Corporate_Memphis https://en.wikipedia.org/wiki/Corporate_Memphis * In fact, I think I'd be excited, it can be a great tool, and people who lack artistic sensibility might find it almost as hard to use as to make art themselves. However, I do think there needs to be some kind of protection from having your 'style' commoditized without consent, using unauthorized images of your own art.

deprecative 2y ago
That last part is the issue. Once the material is released publicly there is zero way to prevent it from being used for training. I like the idea of these things but they're predicated on theft and exploitation.
- jsheard 2y ago
  There are active countermeasures in the form of Glaze/Nightshade, but I don't know how effective those turned out to be in practice.
  - ccgreg 2y ago
    I wonder if Glaze/Nightshade makes it difficult for software to describe the image for a blind person?
    - GaggiX 2y ago
      Not really, because Nightshade should have made image labeling more difficult, but if you try it, you'll see that it doesn't do anything; multimodal models are too powerful nowadays to be fooled by small adversarial noise generated using CLIP LPIPS (small enough not to be too noticeable to us). And Glaze does not try to interfere with labeling.
  - astrange 2y ago
    They don't work, and nothing in the category can ever truly work. Models and adversarial data are equally powerful - you can find an adversarial example for any current model, but you can also train a model that can handle any existing adversarial data. Any image is a good example of /something/ - at worst it's only an example of itself and irrelevant for any other class. Don't know if there's a name for this principle but it's kind of like a Church-Turing thesis for data.
spywaregorilla 2y ago
> However, I do think there needs to be some kind of protection from having your 'style' commoditized without consent, using unauthorized images of your own art. 1) Hire artist to copy the style for 10 images 2) Train model on those problem solved. If you choose "using your images specifically" to steal your style as the hill you want to die on, then you're going to find it's a very useless line of defense.
- voiceblue 2y ago
  That's common enough that there's a name for it [0]. "Stealing the style" is not so much a problem as commoditization, which may find some measure of defense in terms of trademark (not copyright). In this case, for example, the model is named hollieMengert_v1, and we are referring to the style as "Hollie Mengert's art style". [0] https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design
  - spywaregorilla 2y ago
    There is absolutely not defense unless "hollie mengert's style" is trademarked; and even then the protection extends only to that name, not the artistic style itself. Clean room design has nothing to do with this. It is perfectly acceptable to overtly copy a style.
- pbhjpbhj 2y ago
  Just because you hired artists to copy someone else's work doesn't mean your model isn't the result of copyright infringement, indeed it makes it somewhat more obvious imo. Maybe that would be Fair Use, but things like purpose, weak transformation (in the hired artists copies), competition against the original, and business motive would all suggest it relies on non-Fair Use copying. I don't know why you think something that involves direct copying shouldn't be a 'hill to die on' for a copyright infringement claim?
  - spywaregorilla 2y ago
    Not copy the work. Copy the style. Because style is not copyrightable. What I'm saying is that if you believe that your personal style is your valuable asset and choose to protect it by saying people can't train on your images; you ought to be aware that it's trivially easy to build a model that replicates your style and does not require your images.
LettuceSand12 2y ago
Where can I learn these techniques?
- nickthegreek 2y ago
  Stablediffusion subreddit
vunderba 2y ago
Corporate Memphis seems to have that very "flat almost clip-art" style to it. Generative art is ideal for this type of stock art use case where 90% is simply good enough for the vast majority of situations. I feel like the gig economy (things like Fiverr) are unfortunately going to get absolutely crushed by generative art systems since most of their customers aren't that particular and are likely fine with anything that DALL-E / SD can pump out.
- jsheard 2y ago
  > I feel like the gig economy (things like Fiverr) Predictably this is already happening, it doesn't take much browsing in categories such as tattoo design to spot the hallmarks of AI generated artwork. To add insult to injury they often pretend they aren't using AI, so they can up-charge for more intricate artwork that would take more effort if they weren't really just adding more synonyms for "detailed" to their Midjourney prompt. Fiverr does have a dedicated category for AI art, but they don't seem to be making any effort whatsoever to keep AI art out of every other category, so it's pointless.
  - deleted 2y ago
    [deleted]
  - asabla 2y ago
    > it doesn't take much browsing in categories such as tattoo design to spot the hallmarks of AI generated artwork Which is kind of interesting. My first tattoo has its roots from it. Where I used ChatGPT together with DALL-e to express the concept I wanted on my body, and not a direct translation of it. Worked out pretty well, got a lot easier to discuss how and what things would work and how to apply them.
  - grugagag 2y ago
    Soon enough generative art will be everywhere and the style uniformized so when some brands will want to differentiate themselves they will hire people again. They could charge more for the service, whoever survives the generative AI squeeze…
krisoft 2y ago
> I do think there needs to be some kind of protection from having your 'style' commoditized without consent, using unauthorized images of your own art I worry any attempt at that will strangle all creative endevours. Copying others style is not a new thing. It is the stuff culture is made of.
- deleted 2y ago
  [deleted]
Der_Einzige 2y ago
Can we just talk about the fact that the most popular fine-tuned model for SDXL is called PonyDiffusion? The circumstances of its creation, and the whole entire world of like, booru websites that you have to explain to boomers in suits with VP titles who are trying to invest in this space is just surreal. It's like a reverse of when they were raising their kids and giving us "the talk" as children. Now we have to give them "the talk" about what the word "doujinshi" means... for science!
- grob-gambit 2y ago
  [flagged]

deprecative 2y ago

That last part is the issue. Once the material is released publicly there is zero way to prevent it from being used for training. I like the idea of these things but they're predicated on theft and exploitation.

jsheard 2y ago
There are active countermeasures in the form of Glaze/Nightshade, but I don't know how effective those turned out to be in practice.
- ccgreg 2y ago
  I wonder if Glaze/Nightshade makes it difficult for software to describe the image for a blind person?
  - GaggiX 2y ago
    Not really, because Nightshade should have made image labeling more difficult, but if you try it, you'll see that it doesn't do anything; multimodal models are too powerful nowadays to be fooled by small adversarial noise generated using CLIP LPIPS (small enough not to be too noticeable to us). And Glaze does not try to interfere with labeling.
- astrange 2y ago
  They don't work, and nothing in the category can ever truly work. Models and adversarial data are equally powerful - you can find an adversarial example for any current model, but you can also train a model that can handle any existing adversarial data. Any image is a good example of /something/ - at worst it's only an example of itself and irrelevant for any other class. Don't know if there's a name for this principle but it's kind of like a Church-Turing thesis for data.

spywaregorilla 2y ago

> However, I do think there needs to be some kind of protection from having your 'style' commoditized without consent, using unauthorized images of your own art. 1) Hire artist to copy the style for 10 images 2) Train model on those problem solved. If you choose "using your images specifically" to steal your style as the hill you want to die on, then you're going to find it's a very useless line of defense.

voiceblue 2y ago
That's common enough that there's a name for it [0]. "Stealing the style" is not so much a problem as commoditization, which may find some measure of defense in terms of trademark (not copyright). In this case, for example, the model is named hollieMengert_v1, and we are referring to the style as "Hollie Mengert's art style". [0] https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design
- spywaregorilla 2y ago
  There is absolutely not defense unless "hollie mengert's style" is trademarked; and even then the protection extends only to that name, not the artistic style itself. Clean room design has nothing to do with this. It is perfectly acceptable to overtly copy a style.
pbhjpbhj 2y ago
Just because you hired artists to copy someone else's work doesn't mean your model isn't the result of copyright infringement, indeed it makes it somewhat more obvious imo. Maybe that would be Fair Use, but things like purpose, weak transformation (in the hired artists copies), competition against the original, and business motive would all suggest it relies on non-Fair Use copying. I don't know why you think something that involves direct copying shouldn't be a 'hill to die on' for a copyright infringement claim?
- spywaregorilla 2y ago
  Not copy the work. Copy the style. Because style is not copyrightable. What I'm saying is that if you believe that your personal style is your valuable asset and choose to protect it by saying people can't train on your images; you ought to be aware that it's trivially easy to build a model that replicates your style and does not require your images.

LettuceSand12 2y ago

Where can I learn these techniques?

nickthegreek 2y ago
Stablediffusion subreddit

vunderba 2y ago

Corporate Memphis seems to have that very "flat almost clip-art" style to it. Generative art is ideal for this type of stock art use case where 90% is simply good enough for the vast majority of situations. I feel like the gig economy (things like Fiverr) are unfortunately going to get absolutely crushed by generative art systems since most of their customers aren't that particular and are likely fine with anything that DALL-E / SD can pump out.

jsheard 2y ago
> I feel like the gig economy (things like Fiverr) Predictably this is already happening, it doesn't take much browsing in categories such as tattoo design to spot the hallmarks of AI generated artwork. To add insult to injury they often pretend they aren't using AI, so they can up-charge for more intricate artwork that would take more effort if they weren't really just adding more synonyms for "detailed" to their Midjourney prompt. Fiverr does have a dedicated category for AI art, but they don't seem to be making any effort whatsoever to keep AI art out of every other category, so it's pointless.
- deleted 2y ago
  [deleted]
- asabla 2y ago
  > it doesn't take much browsing in categories such as tattoo design to spot the hallmarks of AI generated artwork Which is kind of interesting. My first tattoo has its roots from it. Where I used ChatGPT together with DALL-e to express the concept I wanted on my body, and not a direct translation of it. Worked out pretty well, got a lot easier to discuss how and what things would work and how to apply them.
- grugagag 2y ago
  Soon enough generative art will be everywhere and the style uniformized so when some brands will want to differentiate themselves they will hire people again. They could charge more for the service, whoever survives the generative AI squeeze…

krisoft 2y ago

> I do think there needs to be some kind of protection from having your 'style' commoditized without consent, using unauthorized images of your own art I worry any attempt at that will strangle all creative endevours. Copying others style is not a new thing. It is the stuff culture is made of.

deleted 2y ago
[deleted]

Der_Einzige 2y ago

Can we just talk about the fact that the most popular fine-tuned model for SDXL is called PonyDiffusion? The circumstances of its creation, and the whole entire world of like, booru websites that you have to explain to boomers in suits with VP titles who are trying to invest in this space is just surreal. It's like a reverse of when they were raising their kids and giving us "the talk" as children. Now we have to give them "the talk" about what the word "doujinshi" means... for science!

grob-gambit 2y ago
[flagged]