7 ms·

Interesting read. Others have also created virtual friends using ChatGPT. What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can eas

Interesting read. Others have also created virtual friends using ChatGPT. What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more. The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.

bambax 3y ago
Yes. But those open-source LLMs already exist. It's surprising the OP didn't at least explore the possibility of using them to get around the limitations of "Open" AI (which sounds more and more like a Victorian matron).
- wruza 3y ago
  Are the risks worth it for OpenAI to allow that at what is basically the start of their business?
phatskat 3y ago
And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand. I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?
- dontupvoteme 3y ago
  It's largely to protect their brand from bad press (this has hit even Google/Microsoft in the past), but established finance institutions absolutely hate porn, even the "less shady" ones like onlyfans or pornhub have issues with banking.
- jstarfish 3y ago
  > I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots? These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use. In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way. The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market. Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy. Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox. I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.
  - Sir_Burpalot 3y ago
    1. Some people are so far to the left side of the bell curve that they're going to be screwed out of a human relationship no matter what they do, and it's unkind to withhold sex bots from them just to provide increased motivation to the segment of the population that's still salvageable. 2. Society seems to have collectively decided that romantic intimacy, unlike food and shelter, is optional and that experiencing an absence of it should not entitle you to sympathy or support. Therefore, the kindest thing they can do is leave the autists alone so they can cope as best they can in peace. UPD: 3. Replacing productive activities with substitutes like video games is obviously bad both for individual survival and for society at large, but what are the negative effects of being in a relationship with a fictional partner instead of a real one? You could argue the AI companies would have an incentive to exploit their users, but this is a thing that happens in real relationships as well! Fertility might be a concern, but my understanding is that the current mainstream view is that the world needs fewer children, not more of them.
    - akomtu 3y ago
      3 - this is technodrugs; its users get high on their sexual addiction dopamine loop, and lose the remaining of their will and life energy. It's not our business to stop those who consciously engage in such self destruction, but luring the naive users into this pit, exploiting their loneliness and making money off it isnt any better than what cartels do. The author of this app must be doing this unknowingly: his overdeveloped mind sees just an interesting problem to solve, and the thick layer of nihilism obscures the sight of what he's really doing.
      - Sir_Burpalot 3y ago
        And you think chronic, crippling loneliness can't make someone lose their will to live? If you don't know what tulpas are, there's an entire, fairly large online community of people willingly inducing schizophrenia in themselves just so they have an imaginary friend to talk to. Surely, AI companions would be a healthier alternative to that.
stef25 3y ago
If you're able to let your imagination run wild you'll see that the depth of depravity of NSFW AI content generated by user input probably has no end.
- dontupvoteme 3y ago
  Even worse, some people turn it into a competition. Leak an API key on github or somewhere else and heaven help you if you get picked up by their scraping.
- Invictus0 3y ago
  It is inevitable
chankstein38 3y ago
100% this. I'm not interested in these kinds of usages of LLMs but if someone is paying you to access your API directly you should be letting them do simple stuff like have sexual conversations. OpenAI's filtering is probably the most annoying part of ChatGPT and it makes it useless in so many cases. I can't tell you how many times I've uploaded a picture of a a place from Google Maps or of a location somewhere (that happens to contain a person) and tried to ask for details about the location to find out more info about it just to be shutdown... "I can't help with that" well then you're slowly making yourself useless and not worth the $20/mo.
PoignardAzur 3y ago
> What I find unsavory, is OpenAI imposing content restrictions like NSFW. I mean, the fact that people's NSFW conversations were being processed in a way where both OP and multiple companies could monitor them and do absolutely anything they want with them is a hint that something went very wrong there. I consider OpenAI pre-emptively pulling the plug a good thing in this case. Maybe there's a case for an AI erotica service, but OP sure as hell shouldn't be the one operating it.
com2kid 3y ago
> What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more. I'm guessing they don't want payment processors cutting them off. Everyone, even MS backed companies, are afraid of Visa.
- mrguyorama 3y ago
  And for some obscene reason Visa and Mastercard are afraid of a few hundred extremely religious/conservative moms who make a stink any time someone wants to be slightly different. If they succeed at making it impossible to make money from porn, they sure as fuck aren't stopping there.
  - junofan 3y ago
    AFAIK it’s a couple people in the government who don’t like it, not moms.
KronisLV 3y ago
> The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware. There are projects like llama.cpp https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp and koboldcpp: https://github.com/LostRuins/koboldcpp https://github.com/LostRuins/koboldcpp that can make running open source models pretty easy. As for where to find the models, you can look at HuggingFace for quite the selection, depending on what your needs are and what your hardware (CPU and RAM or GPU and VRAM, or both) can run: https://huggingface.co/models https://huggingface.co/models I was actually curious about this, so I wrote a blog post on it: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-chatbot-without-going-broke https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... which wasn't anything too serious, in addition to formats like GGUF I think becoming more popular than GGML since. Things are moving ahead quite rapidly over there. I guess it can also be easier to pay for something like NovelAI for those who want something for general purpose writing (including most NSFW stuff), but it wasn't exactly amazing for my needs. Despite the name, it wasn't great at prose enhancement: to take a dry sounding paragraph or two and rewrite it until I've found an additional expression or two to include instead, or a few sentence structures than work better. Kind of fun as a replacement for AI Dungeon or something like Character.AI, though. Then again, the local models need expensive hardware to actually run well, if you want good results (somewhere above 20B parameters from what I've seen). I do wonder how much better the models that approach 200B would be for general text generation use cases, if the performance would actually be good. Even more so, when it comes to writing code, or all of the boring stuff like boilerplate/scaffolding code, utility functions, or even stubs for tests. ChatGPT, Phind and GitHub Copilot are all there, but could be even better.
ianbicking 3y ago
It's not clear what the limits are, from the article or really from OpenAI. The author only really knew that some combination of quantity and extent of material was enough to trigger a warning, it could very well have been a handful of users pushing boundaries pretty far. NSFW is one thing, but chat can certainly go into NSFL territory, non-consensual roleplay, etc. I don't know if people here are being naive or dense, but merely "non-PC" content isn't going to trigger anything. I spent like a couple minutes setting up a roleplay where ChatGPT is happily advocating for Nazi ideology (and not even on the API; ChatGPT is generally more censorious than the API). I'm not going to try to hit the limits, but I literally don't think you can hit the limits with merely political speech. I don't know what the NSFW limits are, but I bet they are further than people here think. The vagueness absolutely makes stuff difficult, and the author hit a kind of all-or-nothing approach by going from no restrictions to using the moderation endpoint which is very strict. That could be improved.

bambax 3y ago

Yes. But those open-source LLMs already exist. It's surprising the OP didn't at least explore the possibility of using them to get around the limitations of "Open" AI (which sounds more and more like a Victorian matron).

wruza 3y ago
Are the risks worth it for OpenAI to allow that at what is basically the start of their business?

phatskat 3y ago

And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand. I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?

dontupvoteme 3y ago
It's largely to protect their brand from bad press (this has hit even Google/Microsoft in the past), but established finance institutions absolutely hate porn, even the "less shady" ones like onlyfans or pornhub have issues with banking.
jstarfish 3y ago
> I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots? These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use. In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way. The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market. Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy. Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox. I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.
- Sir_Burpalot 3y ago
  1. Some people are so far to the left side of the bell curve that they're going to be screwed out of a human relationship no matter what they do, and it's unkind to withhold sex bots from them just to provide increased motivation to the segment of the population that's still salvageable. 2. Society seems to have collectively decided that romantic intimacy, unlike food and shelter, is optional and that experiencing an absence of it should not entitle you to sympathy or support. Therefore, the kindest thing they can do is leave the autists alone so they can cope as best they can in peace. UPD: 3. Replacing productive activities with substitutes like video games is obviously bad both for individual survival and for society at large, but what are the negative effects of being in a relationship with a fictional partner instead of a real one? You could argue the AI companies would have an incentive to exploit their users, but this is a thing that happens in real relationships as well! Fertility might be a concern, but my understanding is that the current mainstream view is that the world needs fewer children, not more of them.
  - akomtu 3y ago
    3 - this is technodrugs; its users get high on their sexual addiction dopamine loop, and lose the remaining of their will and life energy. It's not our business to stop those who consciously engage in such self destruction, but luring the naive users into this pit, exploiting their loneliness and making money off it isnt any better than what cartels do. The author of this app must be doing this unknowingly: his overdeveloped mind sees just an interesting problem to solve, and the thick layer of nihilism obscures the sight of what he's really doing.
    - Sir_Burpalot 3y ago
      And you think chronic, crippling loneliness can't make someone lose their will to live? If you don't know what tulpas are, there's an entire, fairly large online community of people willingly inducing schizophrenia in themselves just so they have an imaginary friend to talk to. Surely, AI companions would be a healthier alternative to that.

stef25 3y ago

If you're able to let your imagination run wild you'll see that the depth of depravity of NSFW AI content generated by user input probably has no end.

dontupvoteme 3y ago
Even worse, some people turn it into a competition. Leak an API key on github or somewhere else and heaven help you if you get picked up by their scraping.
Invictus0 3y ago
It is inevitable

chankstein38 3y ago

100% this. I'm not interested in these kinds of usages of LLMs but if someone is paying you to access your API directly you should be letting them do simple stuff like have sexual conversations. OpenAI's filtering is probably the most annoying part of ChatGPT and it makes it useless in so many cases. I can't tell you how many times I've uploaded a picture of a a place from Google Maps or of a location somewhere (that happens to contain a person) and tried to ask for details about the location to find out more info about it just to be shutdown... "I can't help with that" well then you're slowly making yourself useless and not worth the $20/mo.

PoignardAzur 3y ago

> What I find unsavory, is OpenAI imposing content restrictions like NSFW. I mean, the fact that people's NSFW conversations were being processed in a way where both OP and multiple companies could monitor them and do absolutely anything they want with them is a hint that something went very wrong there. I consider OpenAI pre-emptively pulling the plug a good thing in this case. Maybe there's a case for an AI erotica service, but OP sure as hell shouldn't be the one operating it.

com2kid 3y ago

> What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more. I'm guessing they don't want payment processors cutting them off. Everyone, even MS backed companies, are afraid of Visa.

mrguyorama 3y ago
And for some obscene reason Visa and Mastercard are afraid of a few hundred extremely religious/conservative moms who make a stink any time someone wants to be slightly different. If they succeed at making it impossible to make money from porn, they sure as fuck aren't stopping there.
- junofan 3y ago
  AFAIK it’s a couple people in the government who don’t like it, not moms.

KronisLV 3y ago

> The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware. There are projects like llama.cpp https://github.com/ggerganov/llama.cpp https://github.com/ggerganov/llama.cpp and koboldcpp: https://github.com/LostRuins/koboldcpp https://github.com/LostRuins/koboldcpp that can make running open source models pretty easy. As for where to find the models, you can look at HuggingFace for quite the selection, depending on what your needs are and what your hardware (CPU and RAM or GPU and VRAM, or both) can run: https://huggingface.co/models https://huggingface.co/models I was actually curious about this, so I wrote a blog post on it: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-chatbot-without-going-broke https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... which wasn't anything too serious, in addition to formats like GGUF I think becoming more popular than GGML since. Things are moving ahead quite rapidly over there. I guess it can also be easier to pay for something like NovelAI for those who want something for general purpose writing (including most NSFW stuff), but it wasn't exactly amazing for my needs. Despite the name, it wasn't great at prose enhancement: to take a dry sounding paragraph or two and rewrite it until I've found an additional expression or two to include instead, or a few sentence structures than work better. Kind of fun as a replacement for AI Dungeon or something like Character.AI, though. Then again, the local models need expensive hardware to actually run well, if you want good results (somewhere above 20B parameters from what I've seen). I do wonder how much better the models that approach 200B would be for general text generation use cases, if the performance would actually be good. Even more so, when it comes to writing code, or all of the boring stuff like boilerplate/scaffolding code, utility functions, or even stubs for tests. ChatGPT, Phind and GitHub Copilot are all there, but could be even better.

ianbicking 3y ago

It's not clear what the limits are, from the article or really from OpenAI. The author only really knew that some combination of quantity and extent of material was enough to trigger a warning, it could very well have been a handful of users pushing boundaries pretty far. NSFW is one thing, but chat can certainly go into NSFL territory, non-consensual roleplay, etc. I don't know if people here are being naive or dense, but merely "non-PC" content isn't going to trigger anything. I spent like a couple minutes setting up a roleplay where ChatGPT is happily advocating for Nazi ideology (and not even on the API; ChatGPT is generally more censorious than the API). I'm not going to try to hit the limits, but I literally don't think you can hit the limits with merely political speech. I don't know what the NSFW limits are, but I bet they are further than people here think. The vagueness absolutely makes stuff difficult, and the author hit a kind of all-or-nothing approach by going from no restrictions to using the moderation endpoint which is very strict. That could be improved.