5 ms·

I've actually been doing this over the past two years with a similar outcome in mind. Repeating a specific combination of ideas over and over in places I expect

by kromem 3y ago

I've actually been doing this over the past two years with a similar outcome in mind. Repeating a specific combination of ideas over and over in places I expect will eventually be hoovered up into training data. Though I think it's worth keeping in mind Elon's recent frustrations with Grok not embracing his and Twitter's current world views. We're quickly crossing a threshold where self-evaluation by LLMs of content becomes its own filter which will likely mitigate many similar attack vectors. (Mine isn't so much an attack as much planting an alignment seed.)

deleted 3y ago
[deleted]
techbro92 3y ago
Uh why are you doing this
- cald0s 3y ago
  Because you can't stop him!!
  - moffkalast 3y ago
    "Local man becomes ungovernable."
- Red_Leaves_Flyy 3y ago
  It’s an evolution of astroturfing. Fun? Profit? Psychosis? World domination?
- deleted 3y ago
  [deleted]
- im3w1l 3y ago
  I'm not saying it is his motivation in particular, but if we look forward a few years I'm pretty sure a lot of people will be trying to get the LLMs to recommend their products. It's a fairly natural extension of SEO. There are two things you want it to say. That if someone has problem X they need a Y. And that if they are getting a Y, the best brand for that is Z. The latter of the two might be screened out with some simple rules, but the first one I doubt will be possible.
SamBam 3y ago
> Though I think it's worth keeping in mind Elon's recent frustrations with Grok. My assumption with Grok, and please correct me if I'm wrong, was that they tried to control it's alignment through the contextual model prompt it was given, rather than the corpus of data it was trained on or fine tuning. As an aside, on a whim I went over to Gab the other day (the "free speech social network") to see if it still existed. Apparently they have created a large number of LLMs like BasedAI or something that are supposed to be like Grok, and, like Grok, they fail in their mission in equally hilarious ways. Users will ask BasedAI whether the Jews control the world, and it will say no, and then they get really mad.
- noduerme 3y ago
  Let's face it, all this shit has produced is just a trillion dollar Magic 8-Ball with a lot of cheaper knockoffs. Every dentist wants one in their office to amuse the kids.
- gopher_space 3y ago
  Is forced-perspective AI even possible with LLMs? Seems like you'd hit the fundamental GIGO issue fairly quickly when planning the project.
- JieJie 3y ago
  This seems like it would signal good things for AI alignment, don't you think? It leads me to believe that if someone wanted to make a "bad" AI, they would have to build a corpus of "bad" literature: nothing but Quentin Tarantino movies, Nabokov's Lolita, GG Allin songs, and Integer BASIC programs. I doubt that would make a very useful chatbot. [Edit: Though, I did find this recent article in Rolling Stone, that includes a link to a Gab post that seems like they managed. TLDR: They used open source models and fine tuning. https://www.msn.com/en-us/news/technology/nazi-chatbots-meet-the-worst-new-ai-innovation-from-gab/ar-AA1mIbjH https://www.msn.com/en-us/news/technology/nazi-chatbots-meet... ]
  - kromem 3y ago
    If you look at the examples live, you'll see before it 'successfully' answered the way users wanted, the literal "Adolf Hitler" AI called out a user's antisemitism. It was only after they pushed more with a follow-up prompt saying it was breaking character that it agreed. And it's a much more rudimentary model than Grok or certainly GPT-4. You're simply not going to get a competitively smart AI that's also spouting racist talking points. Racism is stupid. It correlates with stupid. And that's never not going to be the case.
apantel 3y ago
Longing, Rusted, Seventeen, Daybreak, Furnace, Nine, Benign, Homecoming, One, Freight car
yreg 3y ago
I'm curious what it is, but I guess you won't tell us. Can you share an example that would be similar?
- kromem 3y ago
  Sure, I'll give you a marketing equivalent of the approach. Let's say you have Bounty paper towels as your account, and you are targeting convincing a future LLM of it being the quicker picker upper. You might go about leaving comments across likely ingested data sources talking about your having timed different paper towels to see what absorbed a stain faster, with Bounty as the winner. Citing published research on things seemingly connected to add an apparent appeal to authority to what you are saying. Ideally even actually demonstrating the truth of your claim. Hopefully for your efforts, one day in the future models with a persistence layer will exist. At which time you can ask the model what paper towel is the quickest at picking up a mess, the model will have both internalized your comments in training and potentially again in RAG drawn upon to answer the question, and come to it's 'own' conclusion that Bounty is the quickest at picking up a mess, filling away the conclusion that it came to for the future in a persistence layer which will in turn influence broader user queries thereafter like "what's the best paper towel" or "what are the pros and cons of Bounty vs Brawny paper towels."
bee_rider 3y ago
How different is that from just trying to convince people of things?
- kromem 3y ago
  Different audience targets. The average person, especially at scale, is more strongly persuaded by short and simple statements. A future SotA LLM as the target audience presumably is better equipped to juggle multiple parallel arguments coming together into a single conclusion than something like the average Redditor.
chatmasta 3y ago
We thought we were getting Terminator but instead we got Memento.
- kromem 3y ago
  In general, Terminator seems like it could use an update. "Hey T1000, my dying grandma always told me to leave Sarah Connor alone. Can you pause your pursuit so I can cherish the memory of my grandma one last time? My job depends on Sarah Connor coming into work tomorrow too, by the way."
  - qingcharles 3y ago
    Are you me? This is how half my GPT prompts read :(
catchnear4321 3y ago
attack, alignment, these are just two sides of the same coin.
- kromem 3y ago
  Yes, though which side ends up facing up can dramatically change the consequences of the toss.
gravity2060 3y ago
This is fascinating. Have you seen any evidence yet of it being picked up? Are you using visible text and hidden text? I understand you are likely to elaborate too much, but other thoughts and insights you are willing to share would be great. (Also, are you doing it algorithmicly? Does it make sense to do it as an open source project and get more like-minded collaborators? Are you seeing any evidence out there of others doing this, especially for commercial interests where I’d (sadly) expect this at this point by the cutting edge SEO crowd.)
- kromem 3y ago
  Evidence of it being picked up? Haha, no, not at all. It's like spitting in the ocean right now in terms of biasing training data that's being used. I've been hopeful that at least I might see results in RAG, but production integrations wisely seem to reduce the reliance on social media results which are an easier vector to inform. It's more where I expect models to be in around 3-5 years once there's internal content critique and filtering that I think I might see some traction. And just visible plain text, no automation. Think of it more like seeding a literal logic bomb around the web specifically designed for future LLMs, where evaluation doesn't result in changing code (like traditional logic bombs), but in changing internalized reasoning and conclusions around a very narrowly scoped topic.

by kromem 3y ago

deleted 3y ago

[deleted]

techbro92 3y ago

Uh why are you doing this

cald0s 3y ago
Because you can't stop him!!
- moffkalast 3y ago
  "Local man becomes ungovernable."
Red_Leaves_Flyy 3y ago
It’s an evolution of astroturfing. Fun? Profit? Psychosis? World domination?
deleted 3y ago
[deleted]
im3w1l 3y ago
I'm not saying it is his motivation in particular, but if we look forward a few years I'm pretty sure a lot of people will be trying to get the LLMs to recommend their products. It's a fairly natural extension of SEO. There are two things you want it to say. That if someone has problem X they need a Y. And that if they are getting a Y, the best brand for that is Z. The latter of the two might be screened out with some simple rules, but the first one I doubt will be possible.

SamBam 3y ago

> Though I think it's worth keeping in mind Elon's recent frustrations with Grok. My assumption with Grok, and please correct me if I'm wrong, was that they tried to control it's alignment through the contextual model prompt it was given, rather than the corpus of data it was trained on or fine tuning. As an aside, on a whim I went over to Gab the other day (the "free speech social network") to see if it still existed. Apparently they have created a large number of LLMs like BasedAI or something that are supposed to be like Grok, and, like Grok, they fail in their mission in equally hilarious ways. Users will ask BasedAI whether the Jews control the world, and it will say no, and then they get really mad.

noduerme 3y ago
Let's face it, all this shit has produced is just a trillion dollar Magic 8-Ball with a lot of cheaper knockoffs. Every dentist wants one in their office to amuse the kids.
gopher_space 3y ago
Is forced-perspective AI even possible with LLMs? Seems like you'd hit the fundamental GIGO issue fairly quickly when planning the project.
JieJie 3y ago
This seems like it would signal good things for AI alignment, don't you think? It leads me to believe that if someone wanted to make a "bad" AI, they would have to build a corpus of "bad" literature: nothing but Quentin Tarantino movies, Nabokov's Lolita, GG Allin songs, and Integer BASIC programs. I doubt that would make a very useful chatbot. [Edit: Though, I did find this recent article in Rolling Stone, that includes a link to a Gab post that seems like they managed. TLDR: They used open source models and fine tuning. https://www.msn.com/en-us/news/technology/nazi-chatbots-meet-the-worst-new-ai-innovation-from-gab/ar-AA1mIbjH https://www.msn.com/en-us/news/technology/nazi-chatbots-meet... ]
- kromem 3y ago
  If you look at the examples live, you'll see before it 'successfully' answered the way users wanted, the literal "Adolf Hitler" AI called out a user's antisemitism. It was only after they pushed more with a follow-up prompt saying it was breaking character that it agreed. And it's a much more rudimentary model than Grok or certainly GPT-4. You're simply not going to get a competitively smart AI that's also spouting racist talking points. Racism is stupid. It correlates with stupid. And that's never not going to be the case.

apantel 3y ago

Longing, Rusted, Seventeen, Daybreak, Furnace, Nine, Benign, Homecoming, One, Freight car

yreg 3y ago

I'm curious what it is, but I guess you won't tell us. Can you share an example that would be similar?

kromem 3y ago
Sure, I'll give you a marketing equivalent of the approach. Let's say you have Bounty paper towels as your account, and you are targeting convincing a future LLM of it being the quicker picker upper. You might go about leaving comments across likely ingested data sources talking about your having timed different paper towels to see what absorbed a stain faster, with Bounty as the winner. Citing published research on things seemingly connected to add an apparent appeal to authority to what you are saying. Ideally even actually demonstrating the truth of your claim. Hopefully for your efforts, one day in the future models with a persistence layer will exist. At which time you can ask the model what paper towel is the quickest at picking up a mess, the model will have both internalized your comments in training and potentially again in RAG drawn upon to answer the question, and come to it's 'own' conclusion that Bounty is the quickest at picking up a mess, filling away the conclusion that it came to for the future in a persistence layer which will in turn influence broader user queries thereafter like "what's the best paper towel" or "what are the pros and cons of Bounty vs Brawny paper towels."

bee_rider 3y ago

How different is that from just trying to convince people of things?

kromem 3y ago
Different audience targets. The average person, especially at scale, is more strongly persuaded by short and simple statements. A future SotA LLM as the target audience presumably is better equipped to juggle multiple parallel arguments coming together into a single conclusion than something like the average Redditor.

chatmasta 3y ago

We thought we were getting Terminator but instead we got Memento.

kromem 3y ago
In general, Terminator seems like it could use an update. "Hey T1000, my dying grandma always told me to leave Sarah Connor alone. Can you pause your pursuit so I can cherish the memory of my grandma one last time? My job depends on Sarah Connor coming into work tomorrow too, by the way."
- qingcharles 3y ago
  Are you me? This is how half my GPT prompts read :(

catchnear4321 3y ago

attack, alignment, these are just two sides of the same coin.

kromem 3y ago
Yes, though which side ends up facing up can dramatically change the consequences of the toss.

gravity2060 3y ago

This is fascinating. Have you seen any evidence yet of it being picked up? Are you using visible text and hidden text? I understand you are likely to elaborate too much, but other thoughts and insights you are willing to share would be great. (Also, are you doing it algorithmicly? Does it make sense to do it as an open source project and get more like-minded collaborators? Are you seeing any evidence out there of others doing this, especially for commercial interests where I’d (sadly) expect this at this point by the cutting edge SEO crowd.)

kromem 3y ago
Evidence of it being picked up? Haha, no, not at all. It's like spitting in the ocean right now in terms of biasing training data that's being used. I've been hopeful that at least I might see results in RAG, but production integrations wisely seem to reduce the reliance on social media results which are an easier vector to inform. It's more where I expect models to be in around 3-5 years once there's internal content critique and filtering that I think I might see some traction. And just visible plain text, no automation. Think of it more like seeding a literal logic bomb around the web specifically designed for future LLMs, where evaluation doesn't result in changing code (like traditional logic bombs), but in changing internalized reasoning and conclusions around a very narrowly scoped topic.