7 ms·
> And yes, StableDiffusion from the original repo will rick roll you if you try to generate something that triggers its NSFW filter. It goes without saying tha
by p-e-w 4y ago
> And yes, StableDiffusion from the original repo will rick roll you if you try to generate something that triggers its NSFW filter.
It goes without saying that the authors of a piece of software have the right to make the software do whatever they want, but that shouldn't stop us from recognizing that AI engineers are starting to act like megalomaniac overseers who consider it part of their mission to steer humanity onto the "right" path.
Who exactly do these people think they are?
Imagine this behavior from a web browser. "The URL of the file you were trying to download triggered my NSFW classifier, so I'm going to replace the file with this funny image."
This isn't funny, it's creepy.
- esjeon 4y agoYeah, but, when it comes to generating NSFW contents, it is illegal to distribute pornographic software in some countries, and doing so can lead to domain blockage and such (i.e. South Korea, believe or not). This is something you gotta understand. Also, if the author wanted to block NSFW contents at all, I'm pretty sure one can actually make the filter inseparable from the main network. This isn't the case here AFAIK.
- p-e-w 4y agoThat makes no sense. You can draw whatever you like with Photoshop. You can search for whatever you want on Google. That doesn't make the software "NSFW", and no country will block Photoshop's domain just because someone used Photoshop to create NSFW content. Don't let people who are clearly motivated by a moralistic desire to control others hide behind BS pseudo-legal excuses.
- manmal 4y agoExplain this to insert regime when they block your GitHub repo or make using your software a felony. You are proposing to act on ideology, while such mitigations are rooted in pragmatism. You often can’t have both.
- esjeon 4y ago1. Google do have a stuff called "SafeSearch"[1] to filter explicit contents, and they do have a DMCA-like system for processing violation reports from governments and legal entities. That is, Google is already cooperating. 2. Photoshop's sole function is to let people draw, so drawing a porn w/ Photoshop is 100% users' responsibility. In case of image generators, the AI itself has the capabilities to create NSFW contents. If you're talking about the interactive nature of "prompt", that logic also makes any adult games SFW - they are safe until the user clicks something. [1]: https://support.google.com/websearch/answer/510 https://support.google.com/websearch/answer/510
- the-rc 4y agoYou'd be right about Photoshop, but... https://helpx.adobe.com/photoshop/cds.html https://helpx.adobe.com/photoshop/cds.html
- numpad0 4y agoSK and Germany are notorious for that. They don't care how the pixel arrays came to existence, any shapes and forms and representations, even links to such information, that are interpretable as belonging to certain categories are plain illegal there. I believe SK don't even recognize copyright laws for such "illegal" information.
- 0cVlTeIATBs 4y agoDevil's advocate, this output could have unexpected results, making a filter possibly desirable. Browsers do filter by default with safe browsing URL lists for malware, and they gate access to sites with bad certificates. But agreed users should have control, leaving such filters as safe defaults only.
- p-e-w 4y ago> Browsers do filter by default with safe browsing URL lists for malware Yes, for malware. As in, software created by criminals to damage your computer and/or steal your personal information. That's not even remotely comparable to what StableDiffusion does here. What they do is refuse to generate content you requested, based on opaque criteria that no doubt are ultimately influenced by quasi-religious sentiments retained from the Bronze Age.
- code_duck 4y agoThe concept is quite similar to the NSFW filter found on Google and other image searches for many years. Like the “mature” search settings, this filter is active by default but simple to disable.
- Arainach 4y agoWhat's creepy is the very real world that we're approaching where anyone can create porn of anyone else given a few pictures without their consent. That can cause all sorts of lasting psychological harm and other societal negative consequences. Even if this probably can't be entirely avoided, that doesn't mean that the folks developing these models are obligated to make it easy or are incorrect to take steps to avoid it.
- hda2 4y ago> What's creepy is the very real world that we're approaching where anyone can create porn of anyone else given a few pictures without their consent. Why is that a problem, exactly? > That can cause all sorts of lasting psychological harm and other societal negative consequences. Like what? To me this sounds like a problem with how some people want to have absolute control over what others do and want everyone else to play along. That's what I find really creepy. Someone wants to generate porn of me fucking a donkey? Let them. They already could do that with ms-paint and I don't get magically harmed if they do. This reminds me of the old saying: "Sticks and stones may break my bones, but AI-generated/photoshopped/glued-together pictures of me fucking a donkey will never hurt me" :)
- NLPlatypus 4y agoThat proverb isn’t true and stems from a generation that didn’t understand mental health
- Georgelemental 4y agoThe proverb describes a state of mind one should aspire to. "Words will never hurt me" not because saying mean things can never do harm (it obviously can), but because I make an effort to control my emotions and reactions. Just as physical health is improved by a nutritious diet and regular exercise, mental health is improved by proper habits of mind.
- bawolff 4y ago
- qdot76367 4y agoNah it’s just basic due diligence for releasing an open source app of this nature. Turning it off is a simple one line change, because everything is obviously named. This is not some high wall to scale. As someone that manages nsfw open source projects, this move seems fine to me. And actually kinda hilarious.
- p-e-w 4y agoIf what StableDiffusion did was ask the user something like "The prompt you entered may result in the generation of content some people find objectionable. Are you sure you want to proceed?", then I would buy your argument. As currently implemented (and the implementation took more work than a confirmation prompt would have!), it's an obvious attempt to control, rather than protect, users. They try their best to dress it up like a joke, but it clearly isn't.
- ThrowawayTestr 4y agoIt's literally one line of code so you don't generate unexpected boobs. Save your ire for the nannies that won't let you generate NSFW images at all.
- greesil 4y agoYes, the megalomania of someone who makes their code and models freely available.
- p-e-w 4y agoStableDiffusion isn't freely available, in the "Free Software" sense. They use the highly uncommon "CreativeML Open RAIL-M License" which is a wall of text composed of weasel words describing how the software is so incredibly advanced and dangerous that despite the authors' earnest wish to do so, they cannot in good conscience make it genuinely Free Software. These people wrote a bunch of Python code that pipes images and text through a GPU, and they're acting as if they had created a secret weapon that somehow humanity must be protected from. If that's not megalomania, I don't know what is.
- MattRix 4y agoOh please, all the creators of these image AIs (OpenAI, Google, Midjourney, SD, etc) are being very very cautious with this stuff. It’s not going to end humanity, but it could easily lead to some really gross content that would paint the responsible organization in a bad light.
- kaoD 4y agoCan you elaborate? Would Adobe be painted in a bad light if someone committed a crime using Photoshop? Honestly it just feels weird. I've read the license restrictions and I can't see why they are there out what they're preventing.
- tirpen 4y agoPhotoshop doesn't have an automatic "Generate image of Obama selling weapons to Putin" function.
- kaoD 4y agoBut anyone with enough time to learn (or money) can do it anyways without much effort. (I'm trying to understand the exact difference here.) So it boils down to democratization? That anyone can do it regardless of skills acquired?
- throwaway5959 4y agoI’m sure you’re great at parties. A number of people have explained how easy this is to turn off (I did it myself in minutes without outside help by literally commenting out a line of code and changing a variable name). Getting offended at everything doesn’t solve anything and just makes half the US think we’re all snowflakes.
- jannes 4y agoRemember the Terminator movie? It’s about an AI that went too far.
- yieldcrv 4y agoDon’t worry, intentionally warped faces and NSFW filters have like a month, tops
- astrange 4y agoThere's no intentionally warped faces on any of these models, SD is just the first model technically advanced enough to be able to generate decent faces. People have very high standards for those. (What is silly is that DALLE2 won't let you edit a picture with a face in it, so you can't outpaint one of its own generations. But actually you can, if you crop it carefully.)
- Angostura 4y ago> Who exactly do these people think they are? Software engineers who took the time to think about the ways that their work could be used. How many times have we seen on HN people pleading with developers to think about the ethical dimensions of their work?
- vimy 4y agoRead the tweets of the Stability AI, company behind Stable Diffusion, founder. He agrees with you. The image is purely to protect themselves against criticism, takedowns, possible legal action, stuff like that. https://mobile.twitter.com/emostaque https://mobile.twitter.com/emostaque