7 ms·

My friend wants to know when she can use this to generate porn, are we close?

sailingparrot 4y ago
The data used to train those model is specifically filtered to remove sexual content, so the model can't generate porn because it has no idea what it looks like, beyond a few samples that made it past the filter. So no, your "friend" can't use it for that.
- upupandup 4y ago
  Why is it that sexual content is so frowned upon in this space? If it's a content publishing platform I would understand that advertisers don't want that, but this is literally dictating people what is bad and good. I just don't understand this Puritan outrage with text-to-image porn generation.
  - blowski 4y ago
    I'd guess that, for general purpose companies, it's an area full of legal ambiguity and potential for media outrage, so just not worth the risk. However, given the evidence of human history, it's certain that someone with an appetite for exploiting this niche will develop exactly that kind of tool.
  - spywaregorilla 4y ago
    Because it's a lot more annoying for your innocuous content to be rendered as porn when the ai happens to interpret it that way than it is for you to be unable to render your pervy desires intentionally. A porn model should really be it's own thing.
  - sailingparrot 4y ago
    Because you can't control what the model is going to output in response to a query. The model is trained to respond in a way that is aligned but there is no guarantee. Since we certainly don't want to show generated image of porn or violence to someone that didn't specifically ask for that, the easiest way to ensure that's not going to happen is to just not train on that kind of data in the first place. The worst that can happen with a model trained on "safe" images is that the image is irrelevant or makes no sense, meaning you could deploy systems with no human curator on the other end, and nothing bad is going to happen. You lose that ability as soon as you integrate porn. Also with techniques like in-painting, the potential for misuse of a model trained on porn/violence would be pretty terrifying. So the benefits of training on porn seems very small compared to the inconvenience. I don't think it's anything to do with puritanism, it's just that if I am the one putting dollars and time to train such a model I am certainly not going to be taking on the added complexity and implications of dealing with porn to just to make a few people realize their fetishes at the risk of my entire model being undeployable because it's outputting too much porn or violence.
    - upupandup 4y ago
      > porn/violence would be pretty terrifying. uh have you seen American/European mainstream pornography? it's already pretty violent (ex. face slapping, choking, kicking, extreme bdsm). I just don't see why this stuff is allowed and protected by the law (if its not recorded and published its illegal) and then we are suddenly concerned about what text can do. Just one of the many double standards I see in Western society.
      - alach11 4y ago
        One example risk is someone using computer-generated content to extort money, demand ransom, etc. The cheaper and easier this becomes, the more likely it is to be weaponized at scale.
        upupandup 4y ago
        but wouldn't the ability to auto-generate blackmail material mean the value of blackmail would fall? Just from a supply and demand perspective, it makes sense to me why a deepfaked kompromat would put serious discount on such material especially if everybody knows it was generated by an AI. Someone like Trump would just shrug and say the pee tapes are deepfaked. I don't think its possible for AI to bypass forensics either. So again this narrative that "deepfake blackmail" would be dangerous makes no sense.
        ALittleLight 4y ago
        I think it's less for Trump level people and more for basic scams. Imagine just automating a Facebook bot to take someone's picture, edit it into a compromising scene, and message them saying you'll share it with their friends if they don't send you some Bitcoin. This gives you a scalable blackmail tool. Of course, after a while it'll probably stop working, but there will be a period of time where it can be done profitably and a longer period where it will be obnoxious. And, of course, you could probably always use the tool to scare children, who, even in the future, might not know that everyone would shrug off the generated pictures.
  - gs17 4y ago
    I imagine a large part of it is that it could generate photorealistic child porn (also "deepfake" porn of real people) and there's not really a good way to prevent it entirely while also allowing generalized sexual content AFAIK. There's probably some debate on how big a problem this really is, but no one wants their system to be the one with news stories about how it's popular with pedophiles. It was the issue they had with AI Dungeon.
    - pixl97 4y ago
      Correct me if I'm not wrong, but in many countries even simulated child porn is illegal. A model spitting that out could be legally problematic.
      - tnzk 4y ago
        Do they remove certain political or religious ideas which is considered illegal in somewhere as well?
        emikulic 4y ago
        Craiyon certainly doesn't.
  - djbebs 4y ago
    Because the law makes it very difficult to provide such services in the spirit of preventing the exploitation of minors. Make no mistake this is an indirectly legal hurdle.
  - sbierwagen 4y ago
    Because if the model generates anything problematic the New York Times will ruin your life.
- deleted 4y ago
  [deleted]
Mountain_Skies 4y ago
Is there a way to make money selling the model to people who want to use it to make porn? If so, it will trickle down relatively quickly. If not, it'll still eventually trickle down but will take longer.
- TaylorAlexander 4y ago
  Once training can be done on a beefy home rig folks will be all over it.
- TigeriusKirk 4y ago
  Surely you could sell custom prompt runs for porn for a great deal more than OpenAI is charging for generalist custom prompts. Making money at it should be easy, and places like PornHub wouldn't care about any outrage. The real challenge would be limiting criminal and civil liability, at least to my not-in-the-business thinking.
- upupandup 4y ago
  This is already a thing in the kpop fake porn industry. I don't know how Patreon/Onlfans are allowing this to happen, I mean it's a travesty that highly suggestive lyrics and stripper dance moves in scantily clad kpop idols are being used for sexual gratification
  - astrange 4y ago
    Not their fault their country banned actual porn.
planetsprite 4y ago
You'll need to train your own model, though I'm sure if someone manages to crowdsource it there's a very obvious economic incentive.
_blop 4y ago
This article hints that Stable Diffusion can at least generate normal looking nude women: https://techcrunch.com/2022/08/12/a-startup-wants-to-democratize-the-tech-behind-dall-e-2-consequences-be-damned/ https://techcrunch.com/2022/08/12/a-startup-wants-to-democra... There are attempts to gather porn images and train or fine-tune existing networks on it, here's a recent attempt by an art student mentioned in the article above (NSFW!!): https://www.vice.com/en/article/m7ggqq/this-furry-porn-ai-generator https://www.vice.com/en/article/m7ggqq/this-furry-porn-ai-ge...
- isoprophlex 4y ago
  Jesus H Christ those are some seriously cursed hindquarters
  - SV_BubbleTime 4y ago
    I was a better person this morning for not knowing that furries had the term "hindquarters". I mean, that's fine for other people, you do you, but for me, I was better this morning.
isoprophlex 4y ago
You'll have to train it on your own data. as others have mentioned the training data for dall-e, stable diffusion etc has been cleaned prior to training. However, if it is possible to re-start the training process from the weights of a non-sexually aware model, this finetuning might not take all that long..!
GistNoesis 4y ago
I did a show HN about this https://news.ycombinator.com/item?id=31900095 https://news.ycombinator.com/item?id=31900095 a month ago, to experiment with the technology. The training was done in a week-end only, with 2 old gpus (1080ti). Currently waiting to scale-up for improve quality mainly for economic reasons, not quite sure I could recoup the training costs yet. Even more so if I go with cloud training. NVidia will release the 4090 in september, and ethereum may do "the merge" that will make GPU useless for mining so GPU price could be affordable so I can update my home cluster with affordable 3090s. (But electricity prices are also up). Also there are new algorithms every month like the stable diffusion, that would obsolete your previous training. The video generation cost is probably still too expensive compared to just paying a cam girl in a low wage country. But it will probably go down soon. This is also some sensitive data, as plagued with copyright issues, so it's quite troublesome to legally share training datasets to share costs. It also has its own challenges with respect to custom dataset creation with text description, so it's probably a better idea to adapt the algorithm to the currently available data to keep the costs low. Finally once someone releases a model, in the next month there will be at least 3 clones. There is also the problem to find an adult friendly payment processor. And the multitude of potential legal issues. But it's probably inevitable.
TulliusCicero 4y ago
Props for just coming out and saying it
GaggiX 4y ago
It is possible to generate nudes, but not pornographic ones.

The data used to train those model is specifically filtered to remove sexual content, so the model can't generate porn because it has no idea what it looks like, beyond a few samples that made it past the filter. So no, your "friend" can't use it for that.

upupandup 4y ago
Why is it that sexual content is so frowned upon in this space? If it's a content publishing platform I would understand that advertisers don't want that, but this is literally dictating people what is bad and good. I just don't understand this Puritan outrage with text-to-image porn generation.
- blowski 4y ago
  I'd guess that, for general purpose companies, it's an area full of legal ambiguity and potential for media outrage, so just not worth the risk. However, given the evidence of human history, it's certain that someone with an appetite for exploiting this niche will develop exactly that kind of tool.
- spywaregorilla 4y ago
  Because it's a lot more annoying for your innocuous content to be rendered as porn when the ai happens to interpret it that way than it is for you to be unable to render your pervy desires intentionally. A porn model should really be it's own thing.
- sailingparrot 4y ago
  Because you can't control what the model is going to output in response to a query. The model is trained to respond in a way that is aligned but there is no guarantee. Since we certainly don't want to show generated image of porn or violence to someone that didn't specifically ask for that, the easiest way to ensure that's not going to happen is to just not train on that kind of data in the first place. The worst that can happen with a model trained on "safe" images is that the image is irrelevant or makes no sense, meaning you could deploy systems with no human curator on the other end, and nothing bad is going to happen. You lose that ability as soon as you integrate porn. Also with techniques like in-painting, the potential for misuse of a model trained on porn/violence would be pretty terrifying. So the benefits of training on porn seems very small compared to the inconvenience. I don't think it's anything to do with puritanism, it's just that if I am the one putting dollars and time to train such a model I am certainly not going to be taking on the added complexity and implications of dealing with porn to just to make a few people realize their fetishes at the risk of my entire model being undeployable because it's outputting too much porn or violence.
  - upupandup 4y ago
    > porn/violence would be pretty terrifying. uh have you seen American/European mainstream pornography? it's already pretty violent (ex. face slapping, choking, kicking, extreme bdsm). I just don't see why this stuff is allowed and protected by the law (if its not recorded and published its illegal) and then we are suddenly concerned about what text can do. Just one of the many double standards I see in Western society.
    - alach11 4y ago
      One example risk is someone using computer-generated content to extort money, demand ransom, etc. The cheaper and easier this becomes, the more likely it is to be weaponized at scale.
      - upupandup 4y ago
        but wouldn't the ability to auto-generate blackmail material mean the value of blackmail would fall? Just from a supply and demand perspective, it makes sense to me why a deepfaked kompromat would put serious discount on such material especially if everybody knows it was generated by an AI. Someone like Trump would just shrug and say the pee tapes are deepfaked. I don't think its possible for AI to bypass forensics either. So again this narrative that "deepfake blackmail" would be dangerous makes no sense.
        ALittleLight 4y ago
        I think it's less for Trump level people and more for basic scams. Imagine just automating a Facebook bot to take someone's picture, edit it into a compromising scene, and message them saying you'll share it with their friends if they don't send you some Bitcoin. This gives you a scalable blackmail tool. Of course, after a while it'll probably stop working, but there will be a period of time where it can be done profitably and a longer period where it will be obnoxious. And, of course, you could probably always use the tool to scare children, who, even in the future, might not know that everyone would shrug off the generated pictures.
- gs17 4y ago
  I imagine a large part of it is that it could generate photorealistic child porn (also "deepfake" porn of real people) and there's not really a good way to prevent it entirely while also allowing generalized sexual content AFAIK. There's probably some debate on how big a problem this really is, but no one wants their system to be the one with news stories about how it's popular with pedophiles. It was the issue they had with AI Dungeon.
  - pixl97 4y ago
    Correct me if I'm not wrong, but in many countries even simulated child porn is illegal. A model spitting that out could be legally problematic.
    - tnzk 4y ago
      Do they remove certain political or religious ideas which is considered illegal in somewhere as well?
      - emikulic 4y ago
        Craiyon certainly doesn't.
- djbebs 4y ago
  Because the law makes it very difficult to provide such services in the spirit of preventing the exploitation of minors. Make no mistake this is an indirectly legal hurdle.
- sbierwagen 4y ago
  Because if the model generates anything problematic the New York Times will ruin your life.
deleted 4y ago
[deleted]

Mountain_Skies 4y ago

Is there a way to make money selling the model to people who want to use it to make porn? If so, it will trickle down relatively quickly. If not, it'll still eventually trickle down but will take longer.

TaylorAlexander 4y ago
Once training can be done on a beefy home rig folks will be all over it.
TigeriusKirk 4y ago
Surely you could sell custom prompt runs for porn for a great deal more than OpenAI is charging for generalist custom prompts. Making money at it should be easy, and places like PornHub wouldn't care about any outrage. The real challenge would be limiting criminal and civil liability, at least to my not-in-the-business thinking.
upupandup 4y ago
This is already a thing in the kpop fake porn industry. I don't know how Patreon/Onlfans are allowing this to happen, I mean it's a travesty that highly suggestive lyrics and stripper dance moves in scantily clad kpop idols are being used for sexual gratification
- astrange 4y ago
  Not their fault their country banned actual porn.

planetsprite 4y ago

You'll need to train your own model, though I'm sure if someone manages to crowdsource it there's a very obvious economic incentive.

_blop 4y ago

This article hints that Stable Diffusion can at least generate normal looking nude women: https://techcrunch.com/2022/08/12/a-startup-wants-to-democratize-the-tech-behind-dall-e-2-consequences-be-damned/ https://techcrunch.com/2022/08/12/a-startup-wants-to-democra... There are attempts to gather porn images and train or fine-tune existing networks on it, here's a recent attempt by an art student mentioned in the article above (NSFW!!): https://www.vice.com/en/article/m7ggqq/this-furry-porn-ai-generator https://www.vice.com/en/article/m7ggqq/this-furry-porn-ai-ge...

isoprophlex 4y ago
Jesus H Christ those are some seriously cursed hindquarters
- SV_BubbleTime 4y ago
  I was a better person this morning for not knowing that furries had the term "hindquarters". I mean, that's fine for other people, you do you, but for me, I was better this morning.

isoprophlex 4y ago

You'll have to train it on your own data. as others have mentioned the training data for dall-e, stable diffusion etc has been cleaned prior to training. However, if it is possible to re-start the training process from the weights of a non-sexually aware model, this finetuning might not take all that long..!

GistNoesis 4y ago

I did a show HN about this https://news.ycombinator.com/item?id=31900095 https://news.ycombinator.com/item?id=31900095 a month ago, to experiment with the technology. The training was done in a week-end only, with 2 old gpus (1080ti). Currently waiting to scale-up for improve quality mainly for economic reasons, not quite sure I could recoup the training costs yet. Even more so if I go with cloud training. NVidia will release the 4090 in september, and ethereum may do "the merge" that will make GPU useless for mining so GPU price could be affordable so I can update my home cluster with affordable 3090s. (But electricity prices are also up). Also there are new algorithms every month like the stable diffusion, that would obsolete your previous training. The video generation cost is probably still too expensive compared to just paying a cam girl in a low wage country. But it will probably go down soon. This is also some sensitive data, as plagued with copyright issues, so it's quite troublesome to legally share training datasets to share costs. It also has its own challenges with respect to custom dataset creation with text description, so it's probably a better idea to adapt the algorithm to the currently available data to keep the costs low. Finally once someone releases a model, in the next month there will be at least 3 clones. There is also the problem to find an adult friendly payment processor. And the multitude of potential legal issues. But it's probably inevitable.

TulliusCicero 4y ago

Props for just coming out and saying it

GaggiX 4y ago

It is possible to generate nudes, but not pornographic ones.