5 ms·

In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it shoul

by in3d 4y ago

In addition to removing NSFW images from the training set, this 2.0 release apparently also removed commercial artist styles and celebrities [1]. While it should be possible to fine tune this model to create them anyway using DreamBooth or a similar approach, they clearly went for the safe route after taking some heat. 1. https://twitter.com/emostaque/status/1595731407095140352?s=46&t=5Zxp66AzZrIrvccjqNMJPA https://twitter.com/emostaque/status/1595731407095140352?s=4...

poisonarena 4y ago
this is really disappointing
arez 4y ago
does this mean that stuff like artstation and deviantart doesn't work anymore as prompts? That would be a huge change
- in3d 4y ago
  Based on sample images I’ve seen, “Greg Rutkowski” doesn’t work anymore for example.
- siraben 4y ago
  Ran the model locally. Neither "trending on artstation" nor "Greg Rutowski" make any differences to the image anymore.[0] I suspect that people will find keywords that would improve the aesthetics further again, or that fine-tuning will also take place. [0] https://imgsli.com/MTM1ODQ5 https://imgsli.com/MTM1ODQ5
  - astrange 4y ago
    “Prompting” and “keywords” are not an essential part of this technology. If you like tokens, make your own tokens with textual-inversion or image inputs.
CuriouslyC 4y ago
Removing NSFW content is fine, people who care about that can work around it easily. Removing celebrities and commercial artists was a mistake though and I expect this will need to be really impressive in other ways or people aren't going to bother using it.
- akie 4y ago
  It's remarkable, this sense of entitlement people have. You literally have a computer program here that can make photorealistic imagery of almost ANYTHING you ask it to, which was impossible even half a year ago, and here you are complaining that people won't use it unless it incorporates all of the protected imagery of famous artists and celebrities. Amazing.
  - dismantlethesun 4y ago
    Is it entitled to think that a 2.0 will not have regressions on useful functionality?
    - blitzar 4y ago
      yes
  - rattray 4y ago
    Loss aversion is strong in humans.
  - sebzim4500 4y ago
    If version 2 is worse than version then obviously a lot of people will use version 1. That doesn't make them entitled.
  - hfbff 4y ago
    I don't think you're using the principle of charity here ("make the best interpretation of a post"). The person isn't complain, he/she is just saying that people will probably return to v1 unless v2 has something impressive to compensate.
choxi 4y ago
Mixing artist names was by far the most effective way to create aesthetically pleasing images, this is a huge change. DreamBooth can only fine-tune on a couple dozen images, and you can't train multiple new concepts in one model, but maybe someone will do a regular fine-tune or train a new model.
- ShamelessC 4y ago
  I'd be curious how well the model still performs given such prompts. Disparate concepts, interpolation, n' all that. Surely it performs worse - but I bet it gets closer than you might think.
  - choxi 4y ago
    Here’s a comparison study on Reddit: https://www.reddit.com/r/StableDiffusion/comments/z3ferx/xy_plot_comparisons_of_sd_v15_ema_vs_sd_20_x768/ https://www.reddit.com/r/StableDiffusion/comments/z3ferx/xy_... It does look like artist names have a significantly reduced effect.
    - ShamelessC 4y ago
      Oh very cool! Indeed filtering the data results in outputs closer to DALLE-2.
- deleted 4y ago
  [deleted]
- Applejinx 4y ago
  That really depends on whether you mean 'like artist X' as 'aesthetically pleasing'. I was fooling around with furry diffusion and got to try a few different models. Yiffy understood artist names, and furry did not: it had further training but stripped of artist tags. All these models are pretty good as that community is strong on art, styles, art skill, and tagging, causing the models to be a serious test case for what's possible. The model with artist names was indeed capable of invoking their styles (for instance, an artist with exceptional anatomy rendering had it translate into the AI version). The more-trained model without the artist names was much more intelligent. It was simply more capable of quality output, so long as your intention wasn't 'remind me of this artist'. I think that's likely to be true in the general case, too. This tech is destined for artist/writer/creator enhancement, so it needs to get smarter at divining INTENT, not just blindly generating 'knock-offs' with little guidance. What you want is better tagging in the dataset, and more personalized. If I have a particular notion of an 'angry sky', this tech should be able to deliver that unfailingly, in any context I like. Greg Rutkowski not required or invoked :)
machina_ex_deus 4y ago
I predicted back when they started backpedaling that there's a chance that sd1.4 or 1.5 will be the best available model to the general public, for a very long duration, because the backlash will force them to self-castrate themselves. You can see nobody likes this new model in any of the stable diffusion communities. It's a big flop and for a good reason. The reason it was so successful in the first place was because you could combine artist names to get the model to the outcome you want. I'll again remind anyone who thinks they might want to use this to download a working version of SD now. They might break their own libraries in the future, and getting SD1.4 could be a real hassle in a year or so. Getting the right .ckpt file, which can have pickled python malware, is not so trivial, and this will get worse in time. It's going to diverge into castrated official model that intentionally breaks the older models and older models from unofficial shady sources that might contain malware.
- jmiskovic 4y ago
  That's like saying that obtaining the On the Origin of Species or Linux kernel will be harder in future. If anything the SD weights will be increasingly ubiquitous as they start embedding it into consumer electronics.
  - mhuffman 4y ago
    >If anything the SD weights will be increasingly ubiquitous as they start embedding it into consumer electronics. I suspect for similar liability issues as SD 2.0, that they will not strt embedding sub-2.0 weights into consumer electronics.
- dmix 4y ago
  These models will always work best with open datasets and open platforms for this reason. Social media/"AI ethics" pressure groups will eventually come from these organizations (see Meta's recent debacle with Galactica). Being an unknown org without these pressures was a big reason Stable Diffusion got so popular in the first place.
- troad 4y ago
  As someone completely unfamiliar with SD but interested in playing around with it in the future, what exactly should I download, to have a fully local instance of 1.4 or 1.5?
astrange 4y ago
This is extremely misleading and you seem to have confused all the other replies. StableDiffusion 1.0 used CLIP released by OpenAI. 2.0 uses a CLIP retrained from scratch by Stability. We don’t know OpenAI’s dataset so don’t know what was in it or how to recreate it. Nothing was “removed”.

by in3d 4y ago

poisonarena 4y ago

this is really disappointing

arez 4y ago

does this mean that stuff like artstation and deviantart doesn't work anymore as prompts? That would be a huge change

in3d 4y ago
Based on sample images I’ve seen, “Greg Rutkowski” doesn’t work anymore for example.
siraben 4y ago
Ran the model locally. Neither "trending on artstation" nor "Greg Rutowski" make any differences to the image anymore.[0] I suspect that people will find keywords that would improve the aesthetics further again, or that fine-tuning will also take place. [0] https://imgsli.com/MTM1ODQ5 https://imgsli.com/MTM1ODQ5
- astrange 4y ago
  “Prompting” and “keywords” are not an essential part of this technology. If you like tokens, make your own tokens with textual-inversion or image inputs.

CuriouslyC 4y ago

Removing NSFW content is fine, people who care about that can work around it easily. Removing celebrities and commercial artists was a mistake though and I expect this will need to be really impressive in other ways or people aren't going to bother using it.

akie 4y ago
It's remarkable, this sense of entitlement people have. You literally have a computer program here that can make photorealistic imagery of almost ANYTHING you ask it to, which was impossible even half a year ago, and here you are complaining that people won't use it unless it incorporates all of the protected imagery of famous artists and celebrities. Amazing.
- dismantlethesun 4y ago
  Is it entitled to think that a 2.0 will not have regressions on useful functionality?
  - blitzar 4y ago
    yes
- rattray 4y ago
  Loss aversion is strong in humans.
- sebzim4500 4y ago
  If version 2 is worse than version then obviously a lot of people will use version 1. That doesn't make them entitled.
- hfbff 4y ago
  I don't think you're using the principle of charity here ("make the best interpretation of a post"). The person isn't complain, he/she is just saying that people will probably return to v1 unless v2 has something impressive to compensate.

choxi 4y ago

Mixing artist names was by far the most effective way to create aesthetically pleasing images, this is a huge change. DreamBooth can only fine-tune on a couple dozen images, and you can't train multiple new concepts in one model, but maybe someone will do a regular fine-tune or train a new model.

ShamelessC 4y ago
I'd be curious how well the model still performs given such prompts. Disparate concepts, interpolation, n' all that. Surely it performs worse - but I bet it gets closer than you might think.
- choxi 4y ago
  Here’s a comparison study on Reddit: https://www.reddit.com/r/StableDiffusion/comments/z3ferx/xy_plot_comparisons_of_sd_v15_ema_vs_sd_20_x768/ https://www.reddit.com/r/StableDiffusion/comments/z3ferx/xy_... It does look like artist names have a significantly reduced effect.
  - ShamelessC 4y ago
    Oh very cool! Indeed filtering the data results in outputs closer to DALLE-2.
deleted 4y ago
[deleted]
Applejinx 4y ago
That really depends on whether you mean 'like artist X' as 'aesthetically pleasing'. I was fooling around with furry diffusion and got to try a few different models. Yiffy understood artist names, and furry did not: it had further training but stripped of artist tags. All these models are pretty good as that community is strong on art, styles, art skill, and tagging, causing the models to be a serious test case for what's possible. The model with artist names was indeed capable of invoking their styles (for instance, an artist with exceptional anatomy rendering had it translate into the AI version). The more-trained model without the artist names was much more intelligent. It was simply more capable of quality output, so long as your intention wasn't 'remind me of this artist'. I think that's likely to be true in the general case, too. This tech is destined for artist/writer/creator enhancement, so it needs to get smarter at divining INTENT, not just blindly generating 'knock-offs' with little guidance. What you want is better tagging in the dataset, and more personalized. If I have a particular notion of an 'angry sky', this tech should be able to deliver that unfailingly, in any context I like. Greg Rutkowski not required or invoked :)

machina_ex_deus 4y ago

I predicted back when they started backpedaling that there's a chance that sd1.4 or 1.5 will be the best available model to the general public, for a very long duration, because the backlash will force them to self-castrate themselves. You can see nobody likes this new model in any of the stable diffusion communities. It's a big flop and for a good reason. The reason it was so successful in the first place was because you could combine artist names to get the model to the outcome you want. I'll again remind anyone who thinks they might want to use this to download a working version of SD now. They might break their own libraries in the future, and getting SD1.4 could be a real hassle in a year or so. Getting the right .ckpt file, which can have pickled python malware, is not so trivial, and this will get worse in time. It's going to diverge into castrated official model that intentionally breaks the older models and older models from unofficial shady sources that might contain malware.

jmiskovic 4y ago
That's like saying that obtaining the On the Origin of Species or Linux kernel will be harder in future. If anything the SD weights will be increasingly ubiquitous as they start embedding it into consumer electronics.
- mhuffman 4y ago
  >If anything the SD weights will be increasingly ubiquitous as they start embedding it into consumer electronics. I suspect for similar liability issues as SD 2.0, that they will not strt embedding sub-2.0 weights into consumer electronics.
dmix 4y ago
These models will always work best with open datasets and open platforms for this reason. Social media/"AI ethics" pressure groups will eventually come from these organizations (see Meta's recent debacle with Galactica). Being an unknown org without these pressures was a big reason Stable Diffusion got so popular in the first place.
troad 4y ago
As someone completely unfamiliar with SD but interested in playing around with it in the future, what exactly should I download, to have a fully local instance of 1.4 or 1.5?

astrange 4y ago

This is extremely misleading and you seem to have confused all the other replies. StableDiffusion 1.0 used CLIP released by OpenAI. 2.0 uses a CLIP retrained from scratch by Stability. We don’t know OpenAI’s dataset so don’t know what was in it or how to recreate it. Nothing was “removed”.