8 ms·

There's a pretty big focus on LLM chatbots for taboo fetishes... I'm sure it's because it's disturbing but, am I the only one who sees that particular facet as

by jchw 2y ago

There's a pretty big focus on LLM chatbots for taboo fetishes... I'm sure it's because it's disturbing but, am I the only one who sees that particular facet as an utter nothing-burger? Of all of the AI safety concerns, I think this is one of the least compelling. If an LLM veered into this kind of topic out of nowhere it could be very disturbing for the user, but in this case it's exactly what they are searching for. I'm pretty sure for any given disturbing topic you can find hundreds of fully-written fictional stories on places like AO3 anyways. I mean, if you want to, you can also engage in these fantasies with other people in erotic role-play, and taboo fetishes are not exactly new. Even if it is illicit in some jurisdictions (No clue, not a lawyer) it is ultimately victimless and unenforceable, so I doubt that dissuades most people. Sure it's rather disturbing, but personally I find lots of things that are very legal and not particularly taboo to be disturbing, and I still don't see a problem if people want to indulge in it, as long as I'm not forced to be involved.

tcdent 2y ago
Really hard to quantify the demand out there, since, thankfully, most of these people keep it out of my feed. But I have a feeling it's significantly more popular than we expect.
orbital-decay 2y ago
Just a reminder that AI safety is all of the following, and many other things: - Rogue AI scenario, which increasingly looks like a figment of collective imagination of certain extremely smart people who discovered religions in their tech tree - Instructions on how to make nuclear weapons (are they scraping classified materials now?..) - Geopolitical games (don't let the adversary have what we have, "for the benefit of all humanity" is a red herring). - Spam/manipulation/botting/astroturfing (legit one, not nearly enough attention paid compared to others). - Erotic roleplay (prudish/thought policing), disturbing erotic roleplay (arguably a nothingburger, division is understandable). Turns out if you shove all that into one huge category of AI safety, the term becomes overloaded and meaningless.
- gs17 2y ago
  > Instructions on how to make nuclear weapons (are they scraping classified materials now?..) Presumably, a "smart enough" AI could work it the physics out the same way humans did to write those classified materials. It's still not a realistic threat unless we're banning physics textbooks as well, AFAIK the barrier more is the materials and equipment required than the principles.
  - deleted 2y ago
    [deleted]
  - ThrowawayTestr 2y ago
    If an LLM can figure out nukes from first principles I think we have bigger problems.
    - jchw 2y ago
      The idea that an LLM will spontaneously use its super-intelligence to somehow develop perfect plans for building weapons of mass destruction seem greatly misplaced. Think of all of the things that are not about intelligence that go into building something like that. It is of course feasible for someone to pull it off, which we know because we already did, but all we needed for that is ordinary human intelligence, the right knowledge, oh, and also (presumably) access to kilograms of weapons-grade plutonium, among many other things. Nobody ever really explains why normal nuclear non-proliferation efforts are insufficient to address the concerns. I get that the fear isn't always rational but it is rather mind-bending that these types of arguments are actually used in the real world in favor of some crazy regulation. I don't even really care that much about LLMs and I find it pretty perplexing.
jchw 2y ago
In most cases commenting on votes is boring and pointless, as per the guidelines. However, rather unusually, I've found it really quite interesting to watch the votes on this comment. It paints a picture that people are actually quite split on this matter. I kind of figured it might wind up in the gray (I don't say "am I the only one" without good cause usually) but on the other hand it leaves me genuinely curious exactly how people disagree with this. (To be clear, I'm probably not actually going to engage much more since this is not really a topic I care deeply about but it's more a morbid curiosity.)
- thomastjeffery 2y ago
  The article we are talking about wants to be about CSAM stories. That alone is a topic that most people have a strong opinion about. A strong enough opinion to say that anything even adjacent to the topic is not worth even a little consideration. CSAM is the penultimate taboo subject, and for good reason. But this article isn't really about CSAM. It's about the taboo itself. This article taunts the reader: if CSAM truly deserves to be taboo, then it logically follows that anything resembling CSAM should be censored, and its creators punished. If we take this argument seriously, then we must actually consider what it means to resemble CSAM. That's a path that no one is interested in exploring, so the argument itself just vanishes. -- The real argument is about the threat of story. Every writer has the power to write any story that they can imagine. There is nothing new about this: it's been true since prehistory, since language itself.
- throwanem 2y ago
  I describe what I have learned in later life about what was done to me in earlier. One may "groom" a child to accept sexual abuse in large part by portraying this as an entirely normal aspect of their present phase of life. To do so requires the presentation of what appears to be true evidence. Such images are invariably lies, but remember that the victim is a child as naïve to lies as to all else, yet. What he sees he will also believe, and not notice all the lies behind it. AI-generated CSAM makes this a much, much easier process. It relieves the prerequisite of acquiring genuine child pornography. Now, all that's required is unsupervised access, not even both at once, to both an AI and a child. You have now expanded the threat radius by several orders of magnitude. This alone suffices to justify AI-generated CSAM as a crime. In the US you may own many types of rifle. You may not, though, own an artillery rifle. It is far too dangerous a weapon, and you no more than any other civilian can have any possible lawful use of such a thing. Therefore its simple possession is a crime. The same principle applies here.
thomastjeffery 2y ago
This is just the natural conclusion to a narrative that conflates hallucination with [un]safety. Nightmares are not danger. LLMs will never be able to filter out specific categories of content. That is because ambiguity is an LLM's core feature. The entire narrative of "LLM safety" implies otherwise. The narrative continues with "guardrails", which don't guard anything. The only thing a "guardrail" can do is be "loud" enough to "talk over" undesired continuations. So long as the content exists in the model, the right permutation of tokens will be able to find it. Unless you want a model trained on content that completely excludes all content about, any sexuality, any violence, or any children; you will always have a model capable of generating a CSAM-like horror story. That's just how text and words work. The reality is that a useful model will probably include some content with each of these three subjects.
- jerf 2y ago
  As AIs improve, they won't even need CSAM or fetish content in their training set. Explaining what they are in a handful of words using normal English is not that difficult. Users would trade prompts freely. As windows grow, you'll be able to stick more info in them. And as I like to remind people, LLMs are not "AI", in the sense that they are not the last word in AI. Better is coming. I don't know when; could be next month, could be 15 years, but we're going to get AIs that "know" things in some more direct and less "technically just a very high probability guess" way.
  - thomastjeffery 2y ago
    What everyone needs to know about LLMs is that they do not perform objectivity. An LLM does not work with categories: it stumbles blindly around a graph of tokens that usually happens to align with real semantic structures. It's like a coloring book: we perceive the lines, and the space between them, to be true representation, but that is a feature of human perception: it does not exist on the page itself.
Terr_ 2y ago
> Of all of the AI safety concerns, I think this is one of the least compelling. It's especially low-priority if nobody's put forward evidence to show that a software-assisted {fictional X} promotes more {actual X} that would harm actual people. I trust a lot of us are old enough to have lived through the failed prophecies that FPS-games needed to be categorically banned to prevent players becoming homicidal shooters in real life.
- lovethevoid 2y ago
  Ah the good old manhunt controversy. The problem with where you place your trust is that this then has to repeat every generation that has not had to deal with such controversies on a large scale, and that when people are emotionally motivated it turns off that rational part temporarily so they don't care about what was previously claimed. Until both of those are adequately accounted for, this is going to repeat endlessly as people love controlling others.

by jchw 2y ago

tcdent 2y ago

Really hard to quantify the demand out there, since, thankfully, most of these people keep it out of my feed. But I have a feeling it's significantly more popular than we expect.

orbital-decay 2y ago

Just a reminder that AI safety is all of the following, and many other things: - Rogue AI scenario, which increasingly looks like a figment of collective imagination of certain extremely smart people who discovered religions in their tech tree - Instructions on how to make nuclear weapons (are they scraping classified materials now?..) - Geopolitical games (don't let the adversary have what we have, "for the benefit of all humanity" is a red herring). - Spam/manipulation/botting/astroturfing (legit one, not nearly enough attention paid compared to others). - Erotic roleplay (prudish/thought policing), disturbing erotic roleplay (arguably a nothingburger, division is understandable). Turns out if you shove all that into one huge category of AI safety, the term becomes overloaded and meaningless.

gs17 2y ago
> Instructions on how to make nuclear weapons (are they scraping classified materials now?..) Presumably, a "smart enough" AI could work it the physics out the same way humans did to write those classified materials. It's still not a realistic threat unless we're banning physics textbooks as well, AFAIK the barrier more is the materials and equipment required than the principles.
- deleted 2y ago
  [deleted]
- ThrowawayTestr 2y ago
  If an LLM can figure out nukes from first principles I think we have bigger problems.
  - jchw 2y ago
    The idea that an LLM will spontaneously use its super-intelligence to somehow develop perfect plans for building weapons of mass destruction seem greatly misplaced. Think of all of the things that are not about intelligence that go into building something like that. It is of course feasible for someone to pull it off, which we know because we already did, but all we needed for that is ordinary human intelligence, the right knowledge, oh, and also (presumably) access to kilograms of weapons-grade plutonium, among many other things. Nobody ever really explains why normal nuclear non-proliferation efforts are insufficient to address the concerns. I get that the fear isn't always rational but it is rather mind-bending that these types of arguments are actually used in the real world in favor of some crazy regulation. I don't even really care that much about LLMs and I find it pretty perplexing.

jchw 2y ago

In most cases commenting on votes is boring and pointless, as per the guidelines. However, rather unusually, I've found it really quite interesting to watch the votes on this comment. It paints a picture that people are actually quite split on this matter. I kind of figured it might wind up in the gray (I don't say "am I the only one" without good cause usually) but on the other hand it leaves me genuinely curious exactly how people disagree with this. (To be clear, I'm probably not actually going to engage much more since this is not really a topic I care deeply about but it's more a morbid curiosity.)

thomastjeffery 2y ago
The article we are talking about wants to be about CSAM stories. That alone is a topic that most people have a strong opinion about. A strong enough opinion to say that anything even adjacent to the topic is not worth even a little consideration. CSAM is the penultimate taboo subject, and for good reason. But this article isn't really about CSAM. It's about the taboo itself. This article taunts the reader: if CSAM truly deserves to be taboo, then it logically follows that anything resembling CSAM should be censored, and its creators punished. If we take this argument seriously, then we must actually consider what it means to resemble CSAM. That's a path that no one is interested in exploring, so the argument itself just vanishes. -- The real argument is about the threat of story. Every writer has the power to write any story that they can imagine. There is nothing new about this: it's been true since prehistory, since language itself.
throwanem 2y ago
I describe what I have learned in later life about what was done to me in earlier. One may "groom" a child to accept sexual abuse in large part by portraying this as an entirely normal aspect of their present phase of life. To do so requires the presentation of what appears to be true evidence. Such images are invariably lies, but remember that the victim is a child as naïve to lies as to all else, yet. What he sees he will also believe, and not notice all the lies behind it. AI-generated CSAM makes this a much, much easier process. It relieves the prerequisite of acquiring genuine child pornography. Now, all that's required is unsupervised access, not even both at once, to both an AI and a child. You have now expanded the threat radius by several orders of magnitude. This alone suffices to justify AI-generated CSAM as a crime. In the US you may own many types of rifle. You may not, though, own an artillery rifle. It is far too dangerous a weapon, and you no more than any other civilian can have any possible lawful use of such a thing. Therefore its simple possession is a crime. The same principle applies here.

thomastjeffery 2y ago

This is just the natural conclusion to a narrative that conflates hallucination with [un]safety. Nightmares are not danger. LLMs will never be able to filter out specific categories of content. That is because ambiguity is an LLM's core feature. The entire narrative of "LLM safety" implies otherwise. The narrative continues with "guardrails", which don't guard anything. The only thing a "guardrail" can do is be "loud" enough to "talk over" undesired continuations. So long as the content exists in the model, the right permutation of tokens will be able to find it. Unless you want a model trained on content that completely excludes all content about, any sexuality, any violence, or any children; you will always have a model capable of generating a CSAM-like horror story. That's just how text and words work. The reality is that a useful model will probably include some content with each of these three subjects.

jerf 2y ago
As AIs improve, they won't even need CSAM or fetish content in their training set. Explaining what they are in a handful of words using normal English is not that difficult. Users would trade prompts freely. As windows grow, you'll be able to stick more info in them. And as I like to remind people, LLMs are not "AI", in the sense that they are not the last word in AI. Better is coming. I don't know when; could be next month, could be 15 years, but we're going to get AIs that "know" things in some more direct and less "technically just a very high probability guess" way.
- thomastjeffery 2y ago
  What everyone needs to know about LLMs is that they do not perform objectivity. An LLM does not work with categories: it stumbles blindly around a graph of tokens that usually happens to align with real semantic structures. It's like a coloring book: we perceive the lines, and the space between them, to be true representation, but that is a feature of human perception: it does not exist on the page itself.

Terr_ 2y ago

> Of all of the AI safety concerns, I think this is one of the least compelling. It's especially low-priority if nobody's put forward evidence to show that a software-assisted {fictional X} promotes more {actual X} that would harm actual people. I trust a lot of us are old enough to have lived through the failed prophecies that FPS-games needed to be categorically banned to prevent players becoming homicidal shooters in real life.

lovethevoid 2y ago
Ah the good old manhunt controversy. The problem with where you place your trust is that this then has to repeat every generation that has not had to deal with such controversies on a large scale, and that when people are emotionally motivated it turns off that rational part temporarily so they don't care about what was previously claimed. Until both of those are adequately accounted for, this is going to repeat endlessly as people love controlling others.