16 ms·

Stable Diffusion Is the Most Important AI Art Model Ever

305 points by brundolf 4y ago

wdb 4y ago
Anyone aware of an AI / art model that allows to convert pixel art into vector drawings? A nice open-source model for that would be really handy :) Still fascinating how these DALE-E and Stable Diffusion models work though
- GaggiX 4y ago
  You can use the img2img script in the stable diffusion repo to convert your pixel art into vector drawings
deleted 4y ago
[deleted]
soueuls 4y ago
I am gonna ask the weird question, but why are we trying to prevent generating CP content? 1. Sharing is publicly is still considered highly illegal. 2. The biggest problem with everything CP related is that to produce this kind of content, real children were used, tortured and often killed. Should we not allowed the model to generate CP content in order to reduce the number of real children being hurt/killed? Plus, this of course would not prevent any authorities from tracking who is sharing this kind of content.
pvillano 4y ago
ha! Generative art is an image search engine with fancy interpolation. Would it be tractible to find a list of nearest training examples? Then you could cite the stolen art. Imagine that as a Twitter bot.
dopeboy 4y ago
Not since Shazam have I had a Neo "whoa" moment like this. Some of these generations are incredible.
amelius 4y ago
Where can I learn more about how this algorithm works?
- r2_pilot 4y ago
  This may be of interest to you: https://github.com/CompVis/stable-diffusion https://github.com/CompVis/stable-diffusion
daenz 4y ago
>once they figure out how to control potentially harmful generations Is it just me, or does anyone else think that this is an impossible and futile task? I don't have a solid grasp on what kind of censorship is possible with this technology, but the goal seems to be on par with making sure nobody says anything mean online. People are extremely creative and are going to find the prompts that generate the "harmful" images.
- buildbot 4y ago
  Yeah, the best they can do is the filter's on top of the output. These models are complex enough that with some reverse engineering you can find "secret" languages to instruct them that would be able to get around input filtering.
  - adhesive_wombat 4y ago
    AI Engine Optimisation could be a good consultancy gig. Figure out how to get your clients the results they want by gaming the rules and filters. Reminds me of the mysterious control of Conjoiner Drives in Alastair Reynold's books.
- adhesive_wombat 4y ago
  Right? Any keyboard can generate "harmful" content, do we need to figure out how to prevent "harmful generations" at the USB HID level?
  - robocat 4y ago
    Run the filter on the image output, not the written input?
    - jwitthuhn 4y ago
      Stable diffusion does run a filter on the output in its default configuration. Any image it deems 'unsafe' gets replaced with a picture of Rick Astley. The thing about that is that it is open source, so you can trivially disable that filter if you like. https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/scripts/txt2img.py#L309 https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...
buildbot 4y ago
I have been playing around with it using ROCM+6900XT, makes a good alternative to DALLE. They have different strengths, DALLE seems better at lighting instructions and cityscapes, but Stable Diffusion is better at sketches. Also, you can fine tune it on whatever you want which is awesome. One interesting effect I have noticed on myself though is that after staring at DALLE or Stable Diffusion generated images for a long time then viewing "real" media, I get the same sense of wrongness that the output is not quite right for awhile, like my brain has been tweaking its processing to prefer the AI art as the ground truth!
- 71a54xd 4y ago
  Yep, I've found the same to be true. Hopefully at some point the model is optimized to introspect complex lighting / textures a bit better.
- cube2222 4y ago
  That's funny, for me dalle2 is in practice miles ahead on pencil sketches, but stable diffusion is cool because the parameters can be customized, which helps with many phrases. Also, you can just leave it running and producing images for an hour. Also, there's no content filtering, but I don't recommend playing around with that if you're sensitive. The lifeless husks and various mixes of body parts I got when playing around with it with fairly benign phrases could very well be used for a horror movie. It might be that I haven't yet found the right phrase for stable diffusion for pencil sketches though, as for dalle2 it's just "<describe what you want>, artstation, pencil sketch, 4k" to generate consistently great pictures.
  - at_a_remove 4y ago
    4chan is having a field day with AI generated porn of celebrities (often with ridiculous prompts) and selecting the most unsettling. One for Billie Ellish looks like some kind of orphaned shoggoth/succubus hybrid just made its first attempt at luring in someone for a meal: "You like human females, yes?" Cataract eyes, aggressive lobotomy mouth, it forgot to pay attention to shoulders and didn't know spines existed or what they were for. Or a second attempt, this time at Bjork, suggesting some kind of lost hominid which consumed only melons in a predator-rich environment.
deleted 4y ago
[deleted]
nullc 4y ago
It'll be interesting to see what happens when a copyright troll ( https://doctorow.medium.com/a-bug-in-early-creative-commons-licenses-has-enabled-a-new-breed-of-superpredator-5f6360713299 https://doctorow.medium.com/a-bug-in-early-creative-commons-... ) realizes that they can acquire the rights to models distributed under these vague-as-fog moral panic licenses, or distribute their own and have people actually use them, and start extracting rents. These licenses will do little to nothing to stop abuse: The abusers will already conceal their identities because their actions are immoral or even illegal (fraud, harassment, etc). But they create a whole host of new liabilities for the users because the definitions are exceedingly subjective. It's tremendously important to make these tools actually open. But open with a lurking liability bomb stops short of the goal. While stability.ai may never turn into a troll or sell their rights to one, that isn't necessarily true for the next model that comes around.
- fragmede 4y ago
  That raises the question. What is the economic licensing for StableDiffusion et al? Can I download it and set it up, and then charge people money to run it?
sf_sugar_daddy 4y ago
empiricus 4y ago
I tried searching "nude" prompt. Instant regret.
- metadat 4y ago
  Definitely don't search "nipples".. ugh... https://lexica.art/ https://lexica.art/ (Direct link doesn't work, count yourself lucky)
  - artificial 4y ago
    Chubby vikings ahoy!
veridies 4y ago
One oddity for me (and I haven't played with a lot of AI art, so maybe this is normal): every time I try to describe a person, it generates like four to seven different faces.
Buttons840 4y ago
I saw the word "safety" a few places in the article. What does "safety" mean in this context?
- i_like_apis 4y ago
  In this context, it’s buzz-killing, pearl-clutching, often woke, nonsense: … attempting to limit the generation space to omit porn, copyright infringement, violence, racially “unbalanced” content … etc.
- ttul 4y ago
  You can ask the AI to generate a picture of horrid things, and it will oblige.
  - emikulic 4y ago
    You can generate things the author doesn't like. And since you're doing it on your video card at home, nobody can stop you.
    - Buttons840 4y ago
      If a computer model could produce the world's best porn, would that be a good or bad thing? Many harmful effects of porn would be amplified, but it would reduce the exploitation of real people in the industry. A moral question society will soon face I think.
      - acdha 4y ago
        I think the bad things would be targeted use of realistic images: for example, imagine the horrible things some people experience in school multiplied by “leaked” photos, or someone’s abusive ex distributing “proof” of their infidelity / unsuitability to have custody, etc. There’s a theory that over time people would stop believing everything they see but there’s still plenty of time for millions of tragedies before that happens, if it ever does. Forensics is going to be a growth industry.
    - 4y ago
afpx 4y ago
After generating 5000 images with these tools, I believe the killer app will be the one that gives the artist the most control. I want a view and a scene and be able to manipulate both in real time. Like, View: 50mm film, wide-angle Scene: rectangular room with window -> show preview Scene: add table -> show preview Scene: move table left -> show preview Scene: add mug on table -> show preview View: center on mug Right now, there’s little control and it’s a lot of random guessing, “Hmm what happens if I add these two terms?”
- spywaregorilla 4y ago
  I think the ideal UX will be the ability to markup images with little comments and have it adapt accordingly. The prompt interface is bad. One of the biggest reasons being that you have virtually no control on the spatial aspect of your additions. Being able to say "add an elephant here and remove this lamp" will be big. Being able to do so with a doodle of an elephant to suggest posing will be even better.
- gamegoblin 4y ago
  Have you seen the img2img results? You draw kind of a crappy Microsoft Paint style image, give it some text for how you want it to actually look, and it does the transformation. For example: https://www.reddit.com/r/StableDiffusion/comments/wwgge8/another_img2img_attempt/ https://www.reddit.com/r/StableDiffusion/comments/wwgge8/ano... Consider also this example of someone splicing Stable Diffusion into a proper image editor and using a combination of img2img, text to image, inpainting, and normal photoshop tools: https://www.reddit.com/r/StableDiffusion/comments/wyduk1/show_rstablediffusion_integrating_sd_in_photoshop/ https://www.reddit.com/r/StableDiffusion/comments/wyduk1/sho...
  - fragmede 4y ago
    Try out Img2Img easily at https://huggingface.co/spaces/huggingface/diffuse-the-rest https://huggingface.co/spaces/huggingface/diffuse-the-rest
- orbital-decay 4y ago
  The natural language alone is one of the worst ways to control image generation. The model knows how to generate anything, but it's own "language" is nothing like yours. It's like writing in Finnish, twisting it in such a way that it would yield coherent Chinese poems after Google Translate. You will end up inserting various garbage into your input and not getting the result you like anyway. img2img gives much better result because you can explain your intent with higher order tools than just textual input alone. What would be best is to properly integrate models like that into some painting software like Krita. Imagine a brush that only affects freckles, blue teapots, fingers, or sharp corners. (or any other thing in a prompt) Or a brush that learns your personal style and transfers it onto a rough sketch you make, speeding up the process. Many possibilities. I think they are already making an img2img plugin for Photoshop. Watch the demo, it's kind of impressive. [0] It's just a rudimentary prototype of what's possible with a properly trained model, but it already looks like a drop-in replacement for photobashing (as an example). https://old.reddit.com/r/StableDiffusion/comments/wyduk1/show_rstablediffusion_integrating_sd_in_photoshop/ https://old.reddit.com/r/StableDiffusion/comments/wyduk1/sho...
fernly 4y ago
It isn't going to impress the person in the street until it actually follows your instructions. I tried several times to express "a tall three-legged stool" but even with the "CFG" (how much the image will be like your prompt) at max, it gave me stools with four or ultimately, two legs. Also tried "a four-legged spider" (don't ask) and got first an eight-legged spider, and next, a spider with eight legs, but four of them were blurred. Sure, dumb, pedestrian requests, no imagination, but a five-year-old would quickly get impatient with its inability to follow simple directions.
- bogwog 4y ago
  At the pace that these things are progressing, I bet we’ll have that kind of control in like a year or less
- astrange 4y ago
  You could sketch a three legged stool in MSPaint and use img2img. I think looking at text prompts as an essential part of the technology is very limiting. You could be using a mix of text, storyboards, images of objects to place inside it, sketches of the desired layout, etc… in fact why not replace your game's renderer with it?
  - jmiskovic 4y ago
    > in fact why not replace your game's renderer with it? What astrange idea, surely it would never work? /winkwink
can16358p 4y ago
Tried it. While it's a huge win to be open source, I find the results always inferior to Midjourney (and DALL-E). I tried to generate some artistic results with variety of prompts and Midjourney always won hands down. But of course, since it's open source, many community tweaks and colab notebooks/forks will probably put it in par with DALL-E by time. But I have trouble imagining Stable Diffusion competing against Midjourney anytime soon: the different is day and night.
- tough 4y ago
  Mid journey uses SD under the good (as of recently, no?)
  - desindol 4y ago
    No
    - corysama 4y ago
      The row of three pics near the top of the article are from a new model they have in the works that uses SD under the hood. It was available briefly as a test run. MJ’s additional magic over SD gets even better results.
  - can16358p 4y ago
    If the beta they introduced briefly for a few days was SD, I had to add that it always, 100% of the time, procuded much inferior results compared to the MJ "v3". It was so bad that if they'd replaced v3 with it (good that they didn't) I'd probably have stopped using MJ and had cancelled my subscription.
    - GaggiX 4y ago
      The stable diffusion model will replace the v3 model soon as it's actually superior in coherence and details (I don't know how you can actually say otherwise) but the model v3 will still be available as the v2.
whywhywhywhy 4y ago
I’ve generated over 1000 images in the last 48 hours. It’s better and faster than using Dall-E, I can literally just leave a prompt churning away in the background for the same costs of playing a high end videogame and check on the results when I want. Honestly if I was a commercial concept artist or illustrator that didn’t have a signature style I’d be really worried. We’re truly gonna see the power of this tech as a tool now it’s not gatekept.
- bitshiftfaced 4y ago
  > Honestly if I was a commercial concept artist or illustrator that didn’t have a signature style I’d be really worried. The prices people pay for any kind of picture art are about to take a nosedive. Stock art websites are going to be hit hard, any kind of graphics artist, any kind of commissioned artist. I wonder if (human) models will be taking a pay cut as a result.
- pg5 4y ago
  For album art, I've found MidJourney is the best of the three. However, I am mainly generating creepy stuff due to my musical genre.
  - whywhywhywhy 4y ago
    I heard Midjourney is adding extra prompts to anything you submit that give it the signature style is has. Pretty sure you could get the same style stuff out of SD if you knew what to add.
tough 4y ago
This (Stable Diffusion) is relaly a game changer. I'm just a run of the mill software engineer (mostly webdev). I never cared about ML, or Data Science, Had been playing with dall-e the past few weeks after getting beta access, but it's too limited/meh and ran out of credits soon. Then DreamStudio (SD SaaS) launched to the public, and I was blown away. Then I tried to run txt2img on my mac, which I did, but it's too cumbersome/slow Then found out about replicate, which also exposes an API to interact with and run the models. I've been since then having fun with it, built some scripts with playwright, and doing generative art with stable diffusion, I'm no artist, but it's so much fun, and results so visually pleasing, I cannot not pursue the urge to explore this. I will be starting an anon account in twitter and try to sell some of my art as NFT's, we'll see where it gets me. Just ordered a card to get around NSFW filters (They' are nonsense and flag some random stuff) If you want to try it the easiest way is dreamstudio/replicate.com https://replicate.com/stability-ai/stable-diffusion https://replicate.com/stability-ai/stable-diffusion
- hwers 4y ago
  This message brought to you by replicate.com
desindol 4y ago
The only point I am bit skeptical is that large language models are just not able to run on consumer hardware so the point with OpenAI is a bit moot.
- jmiskovic 4y ago
  The model runs on just about anything, it's just a question of how fast. My Intel i7 CPU can use OpenVINO to generate a standard image in about 25 minutes. The M1 can do it in ~45 seconds. For comparison, modern GPUs take about 10 seconds.
j7ake 4y ago
Is it possible to generate AI art but also provide a list of citations (maybe weighted) as metadata to help figure which original images most contributed to the generated image? I feel this would help a lot with giving artists more credit with the AI art outputs.
- visarga 4y ago
  Yes, very possible and used in some papers to demonstrate originality/copying behaviour.
dqpb 4y ago
Stability.ai is the industry leader in open AI.
ilaksh 4y ago
I need something like stable diffusion but with transparent backgrounds. Does anyone know how I might do that?
- GaggiX 4y ago
  There are numerous neural networks for image matting.
- xrd 4y ago
  ImageMagick has millions of tools you can use to convert the images. You could script whatever you need using that tool after the AI is done.
  - ilaksh 4y ago
    I use ImageMagick a lot for basic things but there is no way it can remove an arbitrary background. That's an AI task.
goldenkey 4y ago
For those who want to see what kind of erotica it's capable of, here's my tasteful creations: https://www.instagram.com/p/Ch0PHzvvXgr/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/Ch0PHzvvXgr/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwuZlmuJSV/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwuZlmuJSV/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwZmUhukK_/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwZmUhukK_/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChusQWELB-x/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChusQWELB-x/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChureAOL2H0/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChureAOL2H0/?igshid=YmMyMTA2M2Y= The first links are most recent. You can see the progress I've been making as I learn to do better prompt engineering and iterate on existing images by using img2img. The future is here...
xwdv 4y ago
I think these AI art tools are great for finally enabling the masses to unleash their creativity without having to have true art skills. It’s like the equivalent of “no-code” style platforms, except because this is art, we can be a lot more forgiving if the results aren’t perfect. No need for “artists” to have a monopoly on artwork, we’re all artists.
bottlepalm 4y ago
Very exciting to have a brand new technology that you can see quickly advancing and branching out into all kinds of things every week. I wonder how long this current run is going to last and where it ends up.
- DoctorOetker 4y ago
  I'm very impressed, the first sentence I tried was something like "a half submerged archimedes screw ship plowing through sea ice". It had some pretty good ideas of what it might look like. My only gripe is the usual one in the AI field: very sloppy nomenclature. Reading about diffusion models I first expected a novel parametrized family of functions, otherwise known as an "architecture". Instead it seems more like a training method, so a nomenclature of "diffusion training" would seem more apt.
zmmmmm 4y ago
I am curious about the nature of the output being rasterised bitmaps. I would have expected that it was easier for a model to generate output based on primitives that it learned as geometric shapes with spatial relationships (what does an "arm" look like, etc. as a shape). I would like to know if the model does have a layer that represents these and then it effectively "renders" them as rasterised images, or is it really computing at the level of pixels. So far I have not seen anything other than rasterised pixels. I guess it matters because most of these images are unusable for further purposes because they can't really be edited and touched up easily to fix up all the flaws or do the final adaptation. Are there any options that generate the images in anything like vector art that would then facilitate the downstream finishing process rather than fully rasterised bitmaps?
uwuemu 4y ago
All these harmful this and unsafe that... I don't get it. What is the goal here? Are they trying to make it resistant to any "vulgar" inputs (I put "vulgar" in quotes because I can 100% see how they may consider strongly political statements as "vulgar" too)? Or prevent it from producing pornography? Or CP? The last one should be fairly straightforward (kid + nudity = get it out), but the first two are very broad limitations. I understand if you want to make it not super racist, but with pornography, I'm not sure it has a point... even more so given the fact that erotic art is very common in the real world and I'm pretty sure this model (and others like it) can't distinguish between erotic art and pornography (which you can't blame it for given the legal standard seeming to be "I recognize it when I see it"), meaning it can only ever be used to produce SFW imagery scrubbed of any "risque" factors or themes, because the authors didn't want to deal with any kind of erotic art due to this impossibility of separating erotic art from pornography. If the point of all of these models is to get to something resembling an artist, then why intentionally kneecap it from the start and prevent it from producing art?
- audunw 4y ago
  > All these harmful this and unsafe that... I don't get it. What is the goal here? I'm not sure, but they mention biases. So I imagine one thing they want to avoid is that you ask for a drawing of a "criminal" and 90% of the images are of colored people. It should be possible to avoid to minimize these if you review the dataset, at least for certain key words.
xg15 4y ago
> But global paradigm shifts aren’t pleasurable for everyone. As I explained in my latest article on AI art, “How Today's AI Art Debate Will Shape the Creative Landscape of the 21st Century,” we’re getting into a situation—now accelerated with the open-source nature of the model—that’s extremely complex. Artists and other creative professionals are raising concerns and not without reason. Many will lose their jobs, unable to compete with the new apps. Companies like OpenAI, Midjourney, and Stability.ai, although superpowered by the work of many creative workers, haven’t retributed them in any way. And AI users are standing on their shoulders, but without asking for permission first. As I argued there, AI art models like Stable Diffusion pertain to a new category of tools and should be understood with new frameworks of thought adapted to the new realities we’re living in. We can’t simply make analogies or parallelisms with other epochs and expect to be able to explain or predict what it’s going to happen accurately. Some things will be similar and others won’t. We have to treat this impending future as uncharted territory. I wonder if we'll also talk about "conversations", "complex situations" and "the need to treat this as uncharted territory" when some Copilot/GPT3 successor a few years down the line spits out entire production-ready software stacks off the prompt "like Facebook only better" - using our own code as training data.
- trention 4y ago
  >off the prompt "like Facebook only better" This prompt is unspecific to the point of unusableness. Even if this works some day, the spec used will be a lot more detailed, in higher-level pseudocode style.
  - xg15 4y ago
    True, but as you can see with image generators, they can without any problems work off extremely underspecified prompts, they will just use their own priors to fill the gaps. There will absolutely be prompt engineering and I agree that actual, serious prompts will be much more specific than that. I don't think the prompts will necessarily be pseudocode-style. Depending on what trainsets are available, I could imagine we'll have some high-level description of desired features in addition to lots of specifiers which narrow down the specific languages, design patterns, tools etc which should be used in the resulting codebase. You can already use similar prompts with Copilot today by disguising them as comments.
captainmuon 4y ago
So, can someone explain the license to me? I read it and it seems very reasonable. It excludes most of the bad use cases, and doesn't restrict interesting but controversial use cases too much. However, > To generate or disseminate verifiably false information and/or content with the purpose of harming others; How do they define that? So I can generate disseminate false information without the purpose of harming others, just for fun? And what if I believe I am not harming others, but helping others? Can I generate fakes to further my political cause, if I'm convinced it is a "good" cause? And what about Popper's paradox? If I prevent people from harming others, then I am still causing the harmer's harm. I feel they are opening a can of worms here. Also, bad actors will just ignore the license. There is a piece of code that censors obscene generations, you could just comment that out. I feel the license and that filter are not going to stop anybody, but are mostly there for good publicity and so they can wash their hands in innocense...
Aardwolf 4y ago
> Stable Diffusion Is the Most Important AI Art Model Ever at the bottom another linked article: > BLOOM Is the Most Important AI Model of the Decade All very interesting but it's a bit hard to take serious if they exaggerate the titles so much that they contradict each other
- mpaepper 4y ago
  If you want to read an enthusiastic, but less exaggerating piece which explains how the tech works: https://www.paepper.com/blog/posts/how-and-why-stable-diffusion-works-for-text-to-image-generation/ https://www.paepper.com/blog/posts/how-and-why-stable-diffus...
- naillo 4y ago
  Also funny that their first sentences in the second article is > "You may be wondering if such a bold headline is true. The answer is yes." Feel pretty save to tag 'The Algorithmic Bridge' as not something to pay attention to in the future.

305 points by brundolf 4y ago

wdb 4y ago

Anyone aware of an AI / art model that allows to convert pixel art into vector drawings? A nice open-source model for that would be really handy :) Still fascinating how these DALE-E and Stable Diffusion models work though

GaggiX 4y ago
You can use the img2img script in the stable diffusion repo to convert your pixel art into vector drawings

deleted 4y ago

[deleted]

soueuls 4y ago

I am gonna ask the weird question, but why are we trying to prevent generating CP content? 1. Sharing is publicly is still considered highly illegal. 2. The biggest problem with everything CP related is that to produce this kind of content, real children were used, tortured and often killed. Should we not allowed the model to generate CP content in order to reduce the number of real children being hurt/killed? Plus, this of course would not prevent any authorities from tracking who is sharing this kind of content.

pvillano 4y ago

ha! Generative art is an image search engine with fancy interpolation. Would it be tractible to find a list of nearest training examples? Then you could cite the stolen art. Imagine that as a Twitter bot.

dopeboy 4y ago

Not since Shazam have I had a Neo "whoa" moment like this. Some of these generations are incredible.

amelius 4y ago

Where can I learn more about how this algorithm works?

r2_pilot 4y ago
This may be of interest to you: https://github.com/CompVis/stable-diffusion https://github.com/CompVis/stable-diffusion

daenz 4y ago

>once they figure out how to control potentially harmful generations Is it just me, or does anyone else think that this is an impossible and futile task? I don't have a solid grasp on what kind of censorship is possible with this technology, but the goal seems to be on par with making sure nobody says anything mean online. People are extremely creative and are going to find the prompts that generate the "harmful" images.

buildbot 4y ago
Yeah, the best they can do is the filter's on top of the output. These models are complex enough that with some reverse engineering you can find "secret" languages to instruct them that would be able to get around input filtering.
- adhesive_wombat 4y ago
  AI Engine Optimisation could be a good consultancy gig. Figure out how to get your clients the results they want by gaming the rules and filters. Reminds me of the mysterious control of Conjoiner Drives in Alastair Reynold's books.
adhesive_wombat 4y ago
Right? Any keyboard can generate "harmful" content, do we need to figure out how to prevent "harmful generations" at the USB HID level?
- robocat 4y ago
  Run the filter on the image output, not the written input?
  - jwitthuhn 4y ago
    Stable diffusion does run a filter on the output in its default configuration. Any image it deems 'unsafe' gets replaced with a picture of Rick Astley. The thing about that is that it is open source, so you can trivially disable that filter if you like. https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a0f6ee1af8bb9a5d0016ccb27e36dc/scripts/txt2img.py#L309 https://github.com/CompVis/stable-diffusion/blob/69ae4b35e0a...

buildbot 4y ago

I have been playing around with it using ROCM+6900XT, makes a good alternative to DALLE. They have different strengths, DALLE seems better at lighting instructions and cityscapes, but Stable Diffusion is better at sketches. Also, you can fine tune it on whatever you want which is awesome. One interesting effect I have noticed on myself though is that after staring at DALLE or Stable Diffusion generated images for a long time then viewing "real" media, I get the same sense of wrongness that the output is not quite right for awhile, like my brain has been tweaking its processing to prefer the AI art as the ground truth!

71a54xd 4y ago
Yep, I've found the same to be true. Hopefully at some point the model is optimized to introspect complex lighting / textures a bit better.
cube2222 4y ago
That's funny, for me dalle2 is in practice miles ahead on pencil sketches, but stable diffusion is cool because the parameters can be customized, which helps with many phrases. Also, you can just leave it running and producing images for an hour. Also, there's no content filtering, but I don't recommend playing around with that if you're sensitive. The lifeless husks and various mixes of body parts I got when playing around with it with fairly benign phrases could very well be used for a horror movie. It might be that I haven't yet found the right phrase for stable diffusion for pencil sketches though, as for dalle2 it's just "<describe what you want>, artstation, pencil sketch, 4k" to generate consistently great pictures.
- at_a_remove 4y ago
  4chan is having a field day with AI generated porn of celebrities (often with ridiculous prompts) and selecting the most unsettling. One for Billie Ellish looks like some kind of orphaned shoggoth/succubus hybrid just made its first attempt at luring in someone for a meal: "You like human females, yes?" Cataract eyes, aggressive lobotomy mouth, it forgot to pay attention to shoulders and didn't know spines existed or what they were for. Or a second attempt, this time at Bjork, suggesting some kind of lost hominid which consumed only melons in a predator-rich environment.

deleted 4y ago

[deleted]

nullc 4y ago

It'll be interesting to see what happens when a copyright troll ( https://doctorow.medium.com/a-bug-in-early-creative-commons-licenses-has-enabled-a-new-breed-of-superpredator-5f6360713299 https://doctorow.medium.com/a-bug-in-early-creative-commons-... ) realizes that they can acquire the rights to models distributed under these vague-as-fog moral panic licenses, or distribute their own and have people actually use them, and start extracting rents. These licenses will do little to nothing to stop abuse: The abusers will already conceal their identities because their actions are immoral or even illegal (fraud, harassment, etc). But they create a whole host of new liabilities for the users because the definitions are exceedingly subjective. It's tremendously important to make these tools actually open. But open with a lurking liability bomb stops short of the goal. While stability.ai may never turn into a troll or sell their rights to one, that isn't necessarily true for the next model that comes around.

fragmede 4y ago
That raises the question. What is the economic licensing for StableDiffusion et al? Can I download it and set it up, and then charge people money to run it?

sf_sugar_daddy 4y ago

empiricus 4y ago

I tried searching "nude" prompt. Instant regret.

metadat 4y ago
Definitely don't search "nipples".. ugh... https://lexica.art/ https://lexica.art/ (Direct link doesn't work, count yourself lucky)
- artificial 4y ago
  Chubby vikings ahoy!

veridies 4y ago

One oddity for me (and I haven't played with a lot of AI art, so maybe this is normal): every time I try to describe a person, it generates like four to seven different faces.

Buttons840 4y ago

I saw the word "safety" a few places in the article. What does "safety" mean in this context?

i_like_apis 4y ago
In this context, it’s buzz-killing, pearl-clutching, often woke, nonsense: … attempting to limit the generation space to omit porn, copyright infringement, violence, racially “unbalanced” content … etc.
ttul 4y ago
You can ask the AI to generate a picture of horrid things, and it will oblige.
- emikulic 4y ago
  You can generate things the author doesn't like. And since you're doing it on your video card at home, nobody can stop you.
  - Buttons840 4y ago
    If a computer model could produce the world's best porn, would that be a good or bad thing? Many harmful effects of porn would be amplified, but it would reduce the exploitation of real people in the industry. A moral question society will soon face I think.
    - acdha 4y ago
      I think the bad things would be targeted use of realistic images: for example, imagine the horrible things some people experience in school multiplied by “leaked” photos, or someone’s abusive ex distributing “proof” of their infidelity / unsuitability to have custody, etc. There’s a theory that over time people would stop believing everything they see but there’s still plenty of time for millions of tragedies before that happens, if it ever does. Forensics is going to be a growth industry.
  - 4y ago

afpx 4y ago

After generating 5000 images with these tools, I believe the killer app will be the one that gives the artist the most control. I want a view and a scene and be able to manipulate both in real time. Like, View: 50mm film, wide-angle Scene: rectangular room with window -> show preview Scene: add table -> show preview Scene: move table left -> show preview Scene: add mug on table -> show preview View: center on mug Right now, there’s little control and it’s a lot of random guessing, “Hmm what happens if I add these two terms?”

spywaregorilla 4y ago
I think the ideal UX will be the ability to markup images with little comments and have it adapt accordingly. The prompt interface is bad. One of the biggest reasons being that you have virtually no control on the spatial aspect of your additions. Being able to say "add an elephant here and remove this lamp" will be big. Being able to do so with a doodle of an elephant to suggest posing will be even better.
gamegoblin 4y ago
Have you seen the img2img results? You draw kind of a crappy Microsoft Paint style image, give it some text for how you want it to actually look, and it does the transformation. For example: https://www.reddit.com/r/StableDiffusion/comments/wwgge8/another_img2img_attempt/ https://www.reddit.com/r/StableDiffusion/comments/wwgge8/ano... Consider also this example of someone splicing Stable Diffusion into a proper image editor and using a combination of img2img, text to image, inpainting, and normal photoshop tools: https://www.reddit.com/r/StableDiffusion/comments/wyduk1/show_rstablediffusion_integrating_sd_in_photoshop/ https://www.reddit.com/r/StableDiffusion/comments/wyduk1/sho...
- fragmede 4y ago
  Try out Img2Img easily at https://huggingface.co/spaces/huggingface/diffuse-the-rest https://huggingface.co/spaces/huggingface/diffuse-the-rest
orbital-decay 4y ago
The natural language alone is one of the worst ways to control image generation. The model knows how to generate anything, but it's own "language" is nothing like yours. It's like writing in Finnish, twisting it in such a way that it would yield coherent Chinese poems after Google Translate. You will end up inserting various garbage into your input and not getting the result you like anyway. img2img gives much better result because you can explain your intent with higher order tools than just textual input alone. What would be best is to properly integrate models like that into some painting software like Krita. Imagine a brush that only affects freckles, blue teapots, fingers, or sharp corners. (or any other thing in a prompt) Or a brush that learns your personal style and transfers it onto a rough sketch you make, speeding up the process. Many possibilities. I think they are already making an img2img plugin for Photoshop. Watch the demo, it's kind of impressive. [0] It's just a rudimentary prototype of what's possible with a properly trained model, but it already looks like a drop-in replacement for photobashing (as an example). https://old.reddit.com/r/StableDiffusion/comments/wyduk1/show_rstablediffusion_integrating_sd_in_photoshop/ https://old.reddit.com/r/StableDiffusion/comments/wyduk1/sho...

fernly 4y ago

It isn't going to impress the person in the street until it actually follows your instructions. I tried several times to express "a tall three-legged stool" but even with the "CFG" (how much the image will be like your prompt) at max, it gave me stools with four or ultimately, two legs. Also tried "a four-legged spider" (don't ask) and got first an eight-legged spider, and next, a spider with eight legs, but four of them were blurred. Sure, dumb, pedestrian requests, no imagination, but a five-year-old would quickly get impatient with its inability to follow simple directions.

bogwog 4y ago
At the pace that these things are progressing, I bet we’ll have that kind of control in like a year or less
astrange 4y ago
You could sketch a three legged stool in MSPaint and use img2img. I think looking at text prompts as an essential part of the technology is very limiting. You could be using a mix of text, storyboards, images of objects to place inside it, sketches of the desired layout, etc… in fact why not replace your game's renderer with it?
- jmiskovic 4y ago
  > in fact why not replace your game's renderer with it? What astrange idea, surely it would never work? /winkwink

can16358p 4y ago

Tried it. While it's a huge win to be open source, I find the results always inferior to Midjourney (and DALL-E). I tried to generate some artistic results with variety of prompts and Midjourney always won hands down. But of course, since it's open source, many community tweaks and colab notebooks/forks will probably put it in par with DALL-E by time. But I have trouble imagining Stable Diffusion competing against Midjourney anytime soon: the different is day and night.

tough 4y ago
Mid journey uses SD under the good (as of recently, no?)
- desindol 4y ago
  No
  - corysama 4y ago
    The row of three pics near the top of the article are from a new model they have in the works that uses SD under the hood. It was available briefly as a test run. MJ’s additional magic over SD gets even better results.
- can16358p 4y ago
  If the beta they introduced briefly for a few days was SD, I had to add that it always, 100% of the time, procuded much inferior results compared to the MJ "v3". It was so bad that if they'd replaced v3 with it (good that they didn't) I'd probably have stopped using MJ and had cancelled my subscription.
  - GaggiX 4y ago
    The stable diffusion model will replace the v3 model soon as it's actually superior in coherence and details (I don't know how you can actually say otherwise) but the model v3 will still be available as the v2.

whywhywhywhy 4y ago

I’ve generated over 1000 images in the last 48 hours. It’s better and faster than using Dall-E, I can literally just leave a prompt churning away in the background for the same costs of playing a high end videogame and check on the results when I want. Honestly if I was a commercial concept artist or illustrator that didn’t have a signature style I’d be really worried. We’re truly gonna see the power of this tech as a tool now it’s not gatekept.

bitshiftfaced 4y ago
> Honestly if I was a commercial concept artist or illustrator that didn’t have a signature style I’d be really worried. The prices people pay for any kind of picture art are about to take a nosedive. Stock art websites are going to be hit hard, any kind of graphics artist, any kind of commissioned artist. I wonder if (human) models will be taking a pay cut as a result.
pg5 4y ago
For album art, I've found MidJourney is the best of the three. However, I am mainly generating creepy stuff due to my musical genre.
- whywhywhywhy 4y ago
  I heard Midjourney is adding extra prompts to anything you submit that give it the signature style is has. Pretty sure you could get the same style stuff out of SD if you knew what to add.

tough 4y ago

This (Stable Diffusion) is relaly a game changer. I'm just a run of the mill software engineer (mostly webdev). I never cared about ML, or Data Science, Had been playing with dall-e the past few weeks after getting beta access, but it's too limited/meh and ran out of credits soon. Then DreamStudio (SD SaaS) launched to the public, and I was blown away. Then I tried to run txt2img on my mac, which I did, but it's too cumbersome/slow Then found out about replicate, which also exposes an API to interact with and run the models. I've been since then having fun with it, built some scripts with playwright, and doing generative art with stable diffusion, I'm no artist, but it's so much fun, and results so visually pleasing, I cannot not pursue the urge to explore this. I will be starting an anon account in twitter and try to sell some of my art as NFT's, we'll see where it gets me. Just ordered a card to get around NSFW filters (They' are nonsense and flag some random stuff) If you want to try it the easiest way is dreamstudio/replicate.com https://replicate.com/stability-ai/stable-diffusion https://replicate.com/stability-ai/stable-diffusion

hwers 4y ago
This message brought to you by replicate.com

desindol 4y ago

The only point I am bit skeptical is that large language models are just not able to run on consumer hardware so the point with OpenAI is a bit moot.

jmiskovic 4y ago
The model runs on just about anything, it's just a question of how fast. My Intel i7 CPU can use OpenVINO to generate a standard image in about 25 minutes. The M1 can do it in ~45 seconds. For comparison, modern GPUs take about 10 seconds.

j7ake 4y ago

Is it possible to generate AI art but also provide a list of citations (maybe weighted) as metadata to help figure which original images most contributed to the generated image? I feel this would help a lot with giving artists more credit with the AI art outputs.

visarga 4y ago
Yes, very possible and used in some papers to demonstrate originality/copying behaviour.

dqpb 4y ago

Stability.ai is the industry leader in open AI.

ilaksh 4y ago

I need something like stable diffusion but with transparent backgrounds. Does anyone know how I might do that?

GaggiX 4y ago
There are numerous neural networks for image matting.
xrd 4y ago
ImageMagick has millions of tools you can use to convert the images. You could script whatever you need using that tool after the AI is done.
- ilaksh 4y ago
  I use ImageMagick a lot for basic things but there is no way it can remove an arbitrary background. That's an AI task.

goldenkey 4y ago

For those who want to see what kind of erotica it's capable of, here's my tasteful creations: https://www.instagram.com/p/Ch0PHzvvXgr/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/Ch0PHzvvXgr/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwuZlmuJSV/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwuZlmuJSV/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwZmUhukK_/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChwZmUhukK_/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChusQWELB-x/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChusQWELB-x/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChureAOL2H0/?igshid=YmMyMTA2M2Y= https://www.instagram.com/p/ChureAOL2H0/?igshid=YmMyMTA2M2Y= The first links are most recent. You can see the progress I've been making as I learn to do better prompt engineering and iterate on existing images by using img2img. The future is here...

xwdv 4y ago

I think these AI art tools are great for finally enabling the masses to unleash their creativity without having to have true art skills. It’s like the equivalent of “no-code” style platforms, except because this is art, we can be a lot more forgiving if the results aren’t perfect. No need for “artists” to have a monopoly on artwork, we’re all artists.

bottlepalm 4y ago

Very exciting to have a brand new technology that you can see quickly advancing and branching out into all kinds of things every week. I wonder how long this current run is going to last and where it ends up.

DoctorOetker 4y ago
I'm very impressed, the first sentence I tried was something like "a half submerged archimedes screw ship plowing through sea ice". It had some pretty good ideas of what it might look like. My only gripe is the usual one in the AI field: very sloppy nomenclature. Reading about diffusion models I first expected a novel parametrized family of functions, otherwise known as an "architecture". Instead it seems more like a training method, so a nomenclature of "diffusion training" would seem more apt.

zmmmmm 4y ago

I am curious about the nature of the output being rasterised bitmaps. I would have expected that it was easier for a model to generate output based on primitives that it learned as geometric shapes with spatial relationships (what does an "arm" look like, etc. as a shape). I would like to know if the model does have a layer that represents these and then it effectively "renders" them as rasterised images, or is it really computing at the level of pixels. So far I have not seen anything other than rasterised pixels. I guess it matters because most of these images are unusable for further purposes because they can't really be edited and touched up easily to fix up all the flaws or do the final adaptation. Are there any options that generate the images in anything like vector art that would then facilitate the downstream finishing process rather than fully rasterised bitmaps?

uwuemu 4y ago

All these harmful this and unsafe that... I don't get it. What is the goal here? Are they trying to make it resistant to any "vulgar" inputs (I put "vulgar" in quotes because I can 100% see how they may consider strongly political statements as "vulgar" too)? Or prevent it from producing pornography? Or CP? The last one should be fairly straightforward (kid + nudity = get it out), but the first two are very broad limitations. I understand if you want to make it not super racist, but with pornography, I'm not sure it has a point... even more so given the fact that erotic art is very common in the real world and I'm pretty sure this model (and others like it) can't distinguish between erotic art and pornography (which you can't blame it for given the legal standard seeming to be "I recognize it when I see it"), meaning it can only ever be used to produce SFW imagery scrubbed of any "risque" factors or themes, because the authors didn't want to deal with any kind of erotic art due to this impossibility of separating erotic art from pornography. If the point of all of these models is to get to something resembling an artist, then why intentionally kneecap it from the start and prevent it from producing art?

audunw 4y ago
> All these harmful this and unsafe that... I don't get it. What is the goal here? I'm not sure, but they mention biases. So I imagine one thing they want to avoid is that you ask for a drawing of a "criminal" and 90% of the images are of colored people. It should be possible to avoid to minimize these if you review the dataset, at least for certain key words.

xg15 4y ago

> But global paradigm shifts aren’t pleasurable for everyone. As I explained in my latest article on AI art, “How Today's AI Art Debate Will Shape the Creative Landscape of the 21st Century,” we’re getting into a situation—now accelerated with the open-source nature of the model—that’s extremely complex. Artists and other creative professionals are raising concerns and not without reason. Many will lose their jobs, unable to compete with the new apps. Companies like OpenAI, Midjourney, and Stability.ai, although superpowered by the work of many creative workers, haven’t retributed them in any way. And AI users are standing on their shoulders, but without asking for permission first. As I argued there, AI art models like Stable Diffusion pertain to a new category of tools and should be understood with new frameworks of thought adapted to the new realities we’re living in. We can’t simply make analogies or parallelisms with other epochs and expect to be able to explain or predict what it’s going to happen accurately. Some things will be similar and others won’t. We have to treat this impending future as uncharted territory. I wonder if we'll also talk about "conversations", "complex situations" and "the need to treat this as uncharted territory" when some Copilot/GPT3 successor a few years down the line spits out entire production-ready software stacks off the prompt "like Facebook only better" - using our own code as training data.

trention 4y ago
>off the prompt "like Facebook only better" This prompt is unspecific to the point of unusableness. Even if this works some day, the spec used will be a lot more detailed, in higher-level pseudocode style.
- xg15 4y ago
  True, but as you can see with image generators, they can without any problems work off extremely underspecified prompts, they will just use their own priors to fill the gaps. There will absolutely be prompt engineering and I agree that actual, serious prompts will be much more specific than that. I don't think the prompts will necessarily be pseudocode-style. Depending on what trainsets are available, I could imagine we'll have some high-level description of desired features in addition to lots of specifiers which narrow down the specific languages, design patterns, tools etc which should be used in the resulting codebase. You can already use similar prompts with Copilot today by disguising them as comments.

captainmuon 4y ago

So, can someone explain the license to me? I read it and it seems very reasonable. It excludes most of the bad use cases, and doesn't restrict interesting but controversial use cases too much. However, > To generate or disseminate verifiably false information and/or content with the purpose of harming others; How do they define that? So I can generate disseminate false information without the purpose of harming others, just for fun? And what if I believe I am not harming others, but helping others? Can I generate fakes to further my political cause, if I'm convinced it is a "good" cause? And what about Popper's paradox? If I prevent people from harming others, then I am still causing the harmer's harm. I feel they are opening a can of worms here. Also, bad actors will just ignore the license. There is a piece of code that censors obscene generations, you could just comment that out. I feel the license and that filter are not going to stop anybody, but are mostly there for good publicity and so they can wash their hands in innocense...

Aardwolf 4y ago

> Stable Diffusion Is the Most Important AI Art Model Ever at the bottom another linked article: > BLOOM Is the Most Important AI Model of the Decade All very interesting but it's a bit hard to take serious if they exaggerate the titles so much that they contradict each other

mpaepper 4y ago
If you want to read an enthusiastic, but less exaggerating piece which explains how the tech works: https://www.paepper.com/blog/posts/how-and-why-stable-diffusion-works-for-text-to-image-generation/ https://www.paepper.com/blog/posts/how-and-why-stable-diffus...
naillo 4y ago
Also funny that their first sentences in the second article is > "You may be wondering if such a bold headline is true. The answer is yes." Feel pretty save to tag 'The Algorithmic Bridge' as not something to pay attention to in the future.