9 ms·
Some notes on the Stable Diffusion safety filter
- minimaxir 4y agoUnfortunately the safety filters have enough false positives (basically any image with a large amount of fleshy color) to the point that it's just easier to disable it and handle it manually.
- netruk44 4y agoThat'll only work for a little while longer (for future named big-public-release models, obviously the cat's out of the bag for the current version of stable diffusion), right up until the point where they incorporate the filter into the training process. At which point, the end model users get to download will be incapable of producing anything that comes close to triggering the filter, and there will be no way to work around it short of training/fine-tuning your own model, which is prohibitively expensive for 'normal' people, even people with top-of-the-line graphics cards like a 4090.
- pifm_guy 4y agoFine-tuning is pretty cheap compared to the original training run - perhaps just 1% of the cost. Totally within reach of a consortium of.... "entertainment specialists".
- netruk44 4y agoI know a person who fine-tuned stable diffusion, and he said it took 2 weeks of 8xA100 80 GB training time, costing him somewhere between $500-$700 (he got a pretty big discount, too, at today's prices for peer GPU rental it would be over $1,000). Sure, it's peanuts compared to what it must have cost to train stable diffusion from scratch. However, I think most normal people would not consider spending $500 to fine-tune one of these. Edit: Though I do agree that once this kind of filtering is in place during training, NSFW models will begin to pop up all over the place.
- minimaxir 4y agoFor spot-finetuning with Dreambooth (not as good as full-finetuning but can get a specific subject/style much faster), it can be done with about $0.08 of GPU compute, although optimizing it is harder. https://huggingface.co/docs/diffusers/training/dreambooth https://huggingface.co/docs/diffusers/training/dreambooth
- netruk44 4y agoAre these services using textual-inversion? If so, I have to wonder how well they would work on a stable diffusion model that was trained with the filter in place from the start, so that it couldn't generate anything close to the filter. As it is right now, stable diffusion can generate adult imagery by itself, however it seems like it's been fine-tuned after the fact to try to 'cover up' that fact as much as they could before releasing the model publicly.
- gpderetta 4y agoAs far as I understand textual inversion != Dreambooth != Actual fine-tuning
- seaal 4y agoI believe the safety filter is trivial to disable since it was added in one of the last commits prior to Stable Diffusion’s public release and not baked into the model, therefore most forks just remove the safety checker code [1] As far as textual inversion, JoePenna’s Dreambooth [2] implementation uses Textual Inversion. [1] https://github.com/CompVis/stable-diffusion/commit/a6e2f3b120eba81e9c290144f46ceda2145f1248 https://github.com/CompVis/stable-diffusion/commit/a6e2f3b12... [2] https://github.com/JoePenna/Dreambooth-Stable-Diffusion https://github.com/JoePenna/Dreambooth-Stable-Diffusion
- cookingrobot 4y agoYou can fine tune stable diffusion for $10 using this service: https://www.strmr.com/ https://www.strmr.com/ It works super well for putting yourself in the images, the likeness is fantastic. It’s obviously a small training process, they only take 20 images, but it works.
- Animats 4y agoThat problem is being solved. Pornhub now has an AI R&D unit.[1] Their current project is to upscale and colorize out of copyright vintage porn. As a training set, they use modern porn. They point out that they have access to a big training set. Next step, porn generation. [1] https://www.pornhub.com/art/remastured https://www.pornhub.com/art/remastured
- sillysaurusx 4y agoFor a glimpse at what’s possible: https://www.reddit.com/r/unstablediffusion https://www.reddit.com/r/unstablediffusion https://www.reddit.com/r/aiwaifu https://www.reddit.com/r/aiwaifu I’ve been trying to generate tentacle porn since 2019 or so. It’s the whole reason I got into AI. We’re finally there, and it only took three years. Can’t wait to see what 2026 brings. http://n.actionsack.com/pic/media%2FFh08F_hXkAAhalt.jpg http://n.actionsack.com/pic/media%2FFh08F_hXkAAhalt.jpg
- GuB-42 4y ago> https://www.reddit.com/r/unstablediffusion https://www.reddit.com/r/unstablediffusion This subreddit was banned due to a violation of Reddit's rules against non-consensual intimate media. Interesting. Why "non-consensual"? Does it mean Stable Diffusion generated porn of people who actually exist?
- sbierwagen 4y agoYes, reddit routinely bans deepfake subreddits. In practice, this means any net that can produce output that looks like any living person is banned.
- emmelaich 4y agounstable_diffusion is still around. Note the underscore.
- sillysaurusx 4y agoSorry all, I was typing it on my phone and missed an underscore. Here’s the proper link: https://www.reddit.com/r/unstable_diffusion https://www.reddit.com/r/unstable_diffusion
- TaylorAlexander 4y agoThis prediction doesn’t track with what is already happening. Dreambooth is allowing all kinds of people to fine tune their own models at home with nvidia graphics cards, and people are sharing all kinds of updated models that do really well at specific art styles or with NSFW subjects. Go check the nsfw subreddit unstable_diffusion for examples. It seems lots of people are training nsfw models with their own preferred data sets and last I saw someone merged all those checkpoints together in to one model. So if I made a prediction it would be that the training sets for open models from big companies will get scrubbed of nsfw content and then nerds on Reddit will just release their own versions with it added in, and the big companies will make sure everyone knows they didn’t add that stuff and that’s where it will stand.
- netruk44 4y agoI agree with your prediction. Sorry, I was unclear in my post, and left that part unsaid. I agree that it will likely just be the big newly released 'base' models that will be scrubbed of NSFW images, but there's really no way to prevent these models from making those kinds of images at all. It will only take some dedicated individuals, which I know there is no shortage of.
- langitbiru 4y agoThe AI-generated art with Dreambooth works only for avatar type pics. It cannot create fancy gestures (doing a complicated movement with hands, like patting a cat). For now.
- ben_w 4y agoTraining's only prohibitively expensive for normal people today, and the dollar cost per compute operation is still decreasing fairly rapidly.
- xtagon 4y agoThis (training a model with no NSFW content) would be preferable to me. No false positives to worry about. People who do want to generate NSFW stuff can fine-tune or train their own model, nobody owes that functionality to them in freely available ones.
- langitbiru 4y ago"Gal Gadot wearing green suit" triggered it while "Tom Cruise wearing green suit" didn't.
- Wistar 4y agoAs did "young children watching sunset," but not "young boy and girl watching sunset."
- deleted 4y ago[deleted]
- naet 4y agoMight be the word "gal" (which can mean girl or young woman).
- nonethewiser 4y agoAdult porn is filtered too.
- langitbiru 4y agoDoes the picture or the prompt trigger the filter?
- criddell 4y agoAnd then the CSAM filter on your device reports you to some authority.
- BoorishBears 4y agoIf you have things on your device that match entries in the CSAM database, yes there's a chance you're a victim of a targeted attack taking advantage of highly experimental collisions... but the odds you "accidentally generated" that content are not realistic.
- yeet_yeet_yeet 4y ago>not realistic The odds are zero. 1/2^256 = 0. In cryptography these odds are treated as zero until you generate close to 2^128 images. Unfortunately there's no word in natural English to describe how unlikely. The most precise is "zero".
- criddell 4y agoHow can you be so sure? As I understand it, the hash is of features in the image and not the image itself. Are the CSAM feature detection heuristics public?
- ben_w 4y agoAre you assuming that digital images are evenly distributed over the set of all possible 256 bit vectors? Because I don't think that's a reasonable assumption. Even if image recognition was perfectly solved with no known edge cases (ha!), when an entire topic is a semantic stop-sign for most people, you can't expect the mysterious opaque box that is a guilty-enough-to-investigate detection mechanism to be something that gets rapid updates and corrections when new failure modes are discovered.
- jerf 4y agoYou should spend some time with an internet search engine and the term "perceptual hashing". What you're talking about is another type of hashing, which can be useful for classifying image files, but not images. The former has a very concrete definition that is specified down to the bit; the latter is a fuzzy space because it's trying to yield similar (not necessarily identical) hashes for images that humans consider similar. Much different space, much different problem, much different collision situation. Cryptographic hashing is not the only kind of hashing.
- tifik 4y ago> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content without consent of the people who might see it I understand that it's their TOS and they can put pretty much anything in there, but this item seems... odd. I don't really know why exactly this stands out to me. Maybe it's because it's practically un-enforceable? Are they just covering all their bases legally? Trying to think of a good metaphore; let's try this: If you are an artist and someone commissions you to create an art piece that might be sexual, can you say "ok, but you have to ask for consent before you show it to people", and you enshrine it in the contract. Obviously gross violations like trolling by spamming porn are pretty clear cut, but what about the more nuanced cases when you say, display it on your personal website? Are you supposed to have an NSFW overlay? Isn't opening a website sort of implying that you consent to seeing whatever is on there, unless you have a strong preconception of what content the page is expected to display? I might be hugely overthinking this.
- netruk44 4y agoI think the issue they're mainly worried about might be exemplified with a prompt of 'my little pony'. A children's show with quite a lot of adult imagery associated with it on the internet. A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. Did the researchers manage to filter out these images before training? Or is the model aware of both 'kinds' of 'my little pony' images? If the researchers aren't sure they got rid of all of the adult content, then there's really no way to guarantee the model isn't about to ruin some oblivious person's day. So then, do you require people generating images to be intricately familiar with the training dataset? Or do you attempt to prevent any kind of surprise like this by just blocking 'unexpected' interactions like this?
- jimbob45 4y agoA child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. So everyone has to have gimpy AI just because parents can't be expected to take responsibility for what their child does and does not see? Why the fuck is a child being allowed to play with something that can very easily spit out salacious images accidentally? Wouldn't it be significantly easier to add censorship to the prompt input instead? It seems like these tech companies see yet another opportunity to add censorship to their products and can hardly hide their giddy excitement.
- sacrosancty 4y ago
- vintermann 4y agoNow I really want to see the naughty picture of a dolphin swimming in a sea of vectors that the model refused to show us.
- deleted 4y ago[deleted]
- par 4y agoInteresting write up but kind of moot considering there are many nsfw models that are super easy to plugin and use along side stable diffusion (via img2img) to generate all manners of imagery to your hearts content.
- cuddlyogre 4y agoI can understand giving a user the option to filter out something they might not want to see. But the idea that the technology itself should be limited based on the subjective tastes and whims of the day makes my stomach churn. It's not too disconnected from altering a child's brain so that he is incapable of understanding concepts his parents don't like.
- maxbond 4y agoNope, it's not like that in any way at all. AI aren't children, they're a technological artefact a group of people assembled, with no more moral dimension than a toaster. The people who would be the targets of Stable Diffusion-generated porn, depictions they did not consent to, they're actual people who's privacy would be harmed. Technology is an artefact of the society that produced it, and is shaped by it's values. This is not the symptom of any sort of problem. This type of technological fetishism that holds that technology should be developed for it's own sake and that the well being of society is secondary should be discarded. Technology should be developed to make people's lives better or to expand our understanding, not just because it can be. That's how we end up with the proliferation of harmful technologies of no benefit.
- geysersam 4y ago> The people who would be the targets of Stable Diffusion-generated porn, depictions they did not consent to Is it just me finding this notion of someone generating nsfw content to show it to other people without their consent really far fetched? It's about a hundred times more likely they just want it for themselves. The internet has billions of terrible images. If someone wants to spam awful porn today, they can just do it. No SD necessary.
- maxbond 4y agoThere are already communities of people creating & sharing this imagery. The problem is not that people would spam porn. The problem is that stalkers and creeps can & will use this technology to violate people's privacy. Celebrities and influencers complain bitterly in interviews about foot wiki, and try to control what pictures of them are posted to stop people from collecting pictures of their feet. How are they meant to feel about someone being able to push a button and produce a fictional sex tape of theirs? If every photo they post has the potential to be turned into porn, what impact do you imagine that having on people? Furthermore, how are the people making these AIs meant to feel about that? If they want to limit their technology to not produce these images - we're many to understand that as "child abuse"? If I run a porn site and I take down revenge porn when it's reported to me, am I engaging in abuse?
- dimensionc132 4y ago
- dec0dedab0de 4y agoI didn't realize until just now that stable diffusion was not open source or free software (as defined by the osi or fsf respectively) For anyone else curious here is the license: https://github.com/CompVis/stable-diffusion/blob/main/LICENSE https://github.com/CompVis/stable-diffusion/blob/main/LICENS...
- cat_plus_plus 4y agoSo basically we spend more energy on preventing juvenile misuses of technology than realizing it's full potential? People can also just Photoshop someone's head onto an approximately matching naked body and probably be satisfied, thinking with the other head is not very sophisticated. Important part, it's not that person, it's your own imagination. If you indulge in it in private so be it, if you post it publicly, there can be consequences for disrespect. In a way it's better that someone who may have unwisely allowed real pornographic images of themselves to be taken has plausible deniability. A lot of tech, including digital video was initially about porn because its consumers don't have super discerning tastes and can tolerate early glitches. A more important question is where more mature human beings take it from there. If someone is seriously disabled, having a lifelike VR avatar would be quite liberating. How long are we going to delay such assistive technologies for the sake of our squeamishness?
- visarga 4y ago> A lot of tech, including digital video was initially about porn because its consumers don't have super discerning tastes and can tolerate early glitches. Why do you think they don't have discerning taste? Maybe they had no better alternatives early on.
- throwaway0x7E6 4y agoboy, I sure do miss the brief period of time between religious zealots losing their power and ideological zealots gaining theirs. it was good while it lasted.
- 100071231660378 4y ago$ sl config --user ui.username 'Alyssa P. Hacker <alyssa@example.com>'