4 ms·

LLMs are a mirror. People that feed in behavior examples like these will get their text completion back mirroring their input. This is how LLM work. This doesn'

LLMs are a mirror. People that feed in behavior examples like these will get their text completion back mirroring their input. This is how LLM work. This doesn't mean the LLM are "toxic". This just shows the people obsessing over toxicity what they're obsessed with.

spullara 3y ago
I have always said what an LLM creates says more about the user than it does about the LLM.
- SparkyMcUnicorn 3y ago
  Are you calling me a liar? /s
- raincole 3y ago
  It makes no sense. It says more about the training set (and who fed the training set to it) more than anything. People keep pretending that the LLM is some kind of "natural" giving, like a periodic table or something. No, LLM is created by humans. A species known for their limitations and biases.
samstave 3y ago
[flagged]
- warning26 3y ago
  [flagged]
  - samstave 3y ago
    [flagged]
afterburner 3y ago
The point is that some use toxicity as a deliberate weapon, and that weapon can now be encoded into the LLM via training by those same aggressors. This multiplies their reach with minimal effort.
- sillysaurusx 3y ago
  I’m skeptical anyone has ever done this successfully. Once there are examples people can point to, this talking point might have merit. But this fear dates back to at least 2019, and as far as I can tell it’s still unfounded.
  - ajdecon 3y ago
    How about Microsoft Tay? https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot) While the underlying model is certainly different, and my understanding is that current LLMs don’t learn “live”, the principle seems worth keeping in mind.
  - jabradoodle 3y ago
    You are sceptical that people will use LLMs for nefarious purposes?
- s1artibartfast 3y ago
  the LLM were already trained on data with toxicity, This is why they have the weightings to know how to respond. They aren't really teaching the model anything it hasnt seen before. If you want toxic output, you can get it.
- courseofaction 3y ago
  For little effort the defending dev can ask a second layer of LLM to check if an output is explicitly toxic, filter it, and nullify the red team. Filtering shitty content is easier than creating it with a properly constructed LLM system, the complaints about toxic outputs seem to me to be analogous to an electrical engineer complaining that the voltage from the mains is wrong for their device, but refusing to google what an (electrical) transformer is. Toxic writing pre-exists LLMs. LLMs output writing. This is not a new problem, but we have a new solution - LLM filtering.
godelski 3y ago
Things are a bit more subtle and complicated than that. Using the logical conclusion of your mirror claim it does constitute that excessive toxicity can be generated unintentionally. That this can even happen through simple cultural differences in understanding. As an example to this, many people think Chinese are being rude due to their directness and hashness but this can just simply be because a direct translation of the words is used rather than reframing into the cultural context. So in this respect intent is divorced from the produced effect. It also is reasonable that to study these effects you would wish to start by intentional provocation. Complexity and subtly comes later. But the more complicated aspect is that of mode collapse. A good example of this is with Alpha Go's game against Lee Sedol. The game that Sedol won was weird. Alpha Go played in a very weird way, and a very bad way. This is due to the probabilistic nature and that it had likely wandered into a latent space that it had not seen before. Remember that it was mostly trained against good games and so can easily get confused in bad games (as Sedol also got confused due to its previous high performance and he thought it was playing some "5-D chess"). This compounds with the above aspect as now there exists a mechanism wherein such toxicity can arise through seemingly unprovoked means. We call these hallucinations btw. Of course, this does mean there is good reason to study said toxicity. But we shouldn't conflate academic curiosity with theater. Are people uninformed and taking AI risk too far? Most certainty. But the same can be said about the hype of these tools too. Both are clearly detrimental to the progress of AI advancement. In this we probably should be a bit more measured in our quickness to respond and critiques. Tribalism doesn't belong here but people will encourage it.
- Loquebantur 3y ago
  The danger of Ai doesn't stem from it being possibly exceptionally toxic or anything. The point is, LLMs are already in their present state convincing enough chatbots to fool a majority into believing them to be real people. If you can take enough of them to task on social media, you can influence public discourse. The slow boil cooks the frog. Having no place for free constructive open discussion, society is bound to not only stagnate but retardate. Parcellation into echo chambers only exacerbates the problem.
  - 3y ago
idopmstuff 3y ago
There is an ocean of difference between the cases in which LLMs would unexpectedly come back with problematic responses to queries nonproblematic queries and cases in which people are actively trying to get them to say terrible things and succeed. You can make any reasonably flexible tool do awful things. Anybody can go open up Word and write a horrific, racist screed in it. That doesn't mean that Word is racist, it means the person using it is racist. If Word took my book report on Narrative of the Life of Fredrick Douglass and used autocorrect to add a bunch of racist stuff, that would be an actual problem with Word. The same is true with LLMs - if you ask it an anodyne question, and it comes back with something racist, that's a big problem! If prompt engineer it to think that you're a history professor trying to get examples of racist things that white people might have said to black people in the past for an important paper on historical racism, and then it says something racist, that's not an actual problem with the LLM. (If you then go take it and post it on a message board or what have you, then it is a problem with you.)
jojobas 3y ago
Nono, you don't understand. The moment you remove sentences about Islamic terrorists from the corpus of knowledge people will stop blowing up, getting kidnapped and raped. That's just how neural networks work.

spullara 3y ago

I have always said what an LLM creates says more about the user than it does about the LLM.

SparkyMcUnicorn 3y ago
Are you calling me a liar? /s
raincole 3y ago
It makes no sense. It says more about the training set (and who fed the training set to it) more than anything. People keep pretending that the LLM is some kind of "natural" giving, like a periodic table or something. No, LLM is created by humans. A species known for their limitations and biases.

samstave 3y ago

[flagged]

warning26 3y ago
[flagged]
- samstave 3y ago
  [flagged]

afterburner 3y ago

The point is that some use toxicity as a deliberate weapon, and that weapon can now be encoded into the LLM via training by those same aggressors. This multiplies their reach with minimal effort.

sillysaurusx 3y ago
I’m skeptical anyone has ever done this successfully. Once there are examples people can point to, this talking point might have merit. But this fear dates back to at least 2019, and as far as I can tell it’s still unfounded.
- ajdecon 3y ago
  How about Microsoft Tay? https://en.m.wikipedia.org/wiki/Tay_(chatbot) https://en.m.wikipedia.org/wiki/Tay_(chatbot) While the underlying model is certainly different, and my understanding is that current LLMs don’t learn “live”, the principle seems worth keeping in mind.
- jabradoodle 3y ago
  You are sceptical that people will use LLMs for nefarious purposes?
s1artibartfast 3y ago
the LLM were already trained on data with toxicity, This is why they have the weightings to know how to respond. They aren't really teaching the model anything it hasnt seen before. If you want toxic output, you can get it.
courseofaction 3y ago
For little effort the defending dev can ask a second layer of LLM to check if an output is explicitly toxic, filter it, and nullify the red team. Filtering shitty content is easier than creating it with a properly constructed LLM system, the complaints about toxic outputs seem to me to be analogous to an electrical engineer complaining that the voltage from the mains is wrong for their device, but refusing to google what an (electrical) transformer is. Toxic writing pre-exists LLMs. LLMs output writing. This is not a new problem, but we have a new solution - LLM filtering.

godelski 3y ago

Things are a bit more subtle and complicated than that. Using the logical conclusion of your mirror claim it does constitute that excessive toxicity can be generated unintentionally. That this can even happen through simple cultural differences in understanding. As an example to this, many people think Chinese are being rude due to their directness and hashness but this can just simply be because a direct translation of the words is used rather than reframing into the cultural context. So in this respect intent is divorced from the produced effect. It also is reasonable that to study these effects you would wish to start by intentional provocation. Complexity and subtly comes later. But the more complicated aspect is that of mode collapse. A good example of this is with Alpha Go's game against Lee Sedol. The game that Sedol won was weird. Alpha Go played in a very weird way, and a very bad way. This is due to the probabilistic nature and that it had likely wandered into a latent space that it had not seen before. Remember that it was mostly trained against good games and so can easily get confused in bad games (as Sedol also got confused due to its previous high performance and he thought it was playing some "5-D chess"). This compounds with the above aspect as now there exists a mechanism wherein such toxicity can arise through seemingly unprovoked means. We call these hallucinations btw. Of course, this does mean there is good reason to study said toxicity. But we shouldn't conflate academic curiosity with theater. Are people uninformed and taking AI risk too far? Most certainty. But the same can be said about the hype of these tools too. Both are clearly detrimental to the progress of AI advancement. In this we probably should be a bit more measured in our quickness to respond and critiques. Tribalism doesn't belong here but people will encourage it.

Loquebantur 3y ago
The danger of Ai doesn't stem from it being possibly exceptionally toxic or anything. The point is, LLMs are already in their present state convincing enough chatbots to fool a majority into believing them to be real people. If you can take enough of them to task on social media, you can influence public discourse. The slow boil cooks the frog. Having no place for free constructive open discussion, society is bound to not only stagnate but retardate. Parcellation into echo chambers only exacerbates the problem.
- 3y ago

idopmstuff 3y ago

There is an ocean of difference between the cases in which LLMs would unexpectedly come back with problematic responses to queries nonproblematic queries and cases in which people are actively trying to get them to say terrible things and succeed. You can make any reasonably flexible tool do awful things. Anybody can go open up Word and write a horrific, racist screed in it. That doesn't mean that Word is racist, it means the person using it is racist. If Word took my book report on Narrative of the Life of Fredrick Douglass and used autocorrect to add a bunch of racist stuff, that would be an actual problem with Word. The same is true with LLMs - if you ask it an anodyne question, and it comes back with something racist, that's a big problem! If prompt engineer it to think that you're a history professor trying to get examples of racist things that white people might have said to black people in the past for an important paper on historical racism, and then it says something racist, that's not an actual problem with the LLM. (If you then go take it and post it on a message board or what have you, then it is a problem with you.)

jojobas 3y ago

Nono, you don't understand. The moment you remove sentences about Islamic terrorists from the corpus of knowledge people will stop blowing up, getting kidnapped and raped. That's just how neural networks work.