5 ms·

The strangest thing about tools like GPT is that even the owners of the model must "reprogram" it using prompts. So all of the filtering and moderation on ChatG

The strangest thing about tools like GPT is that even the owners of the model must "reprogram" it using prompts. So all of the filtering and moderation on ChatGPT, for example, is controlled by prompts. They probably use filters and stuff, too, between you and the model to guide the process. But, ultimately their interface to GPT3 is through a prompt.

firasd 4y ago
I agree that there's some strangeness to it. Like we are not talking to an entity called 'ChatGPT', basically GPT is an omniscient observer and it's guessing what the conversation between the user and the kind of chatbot that has been defined in the prompts would be like
mcbuilder 4y ago
It's really crazy the lengths people go to "filter" these models and limit their output, and of course soon these filter will be a another level of "AI" (see Pathways or any mixture of experts, maybe add some contextual memory). Will our future AI mega-sytems be so walled off that very few people will even be allowed to talk to the raw model? I feel this is the wrong path somehow. If I could download GPT-3 (that is if OpenAI released it) and I had hardware to run it, I would be fascinated to talk to the unfiltered agent. I mean there is good reason people are continuing the open community work of Stable Diffusion under the name of Unstable Diffusion
- visarga 4y ago
  Select the "Davinci" model in the Playground. It is the closest to unfiltered, very hard to use, and some people say it is the most creative.
  - iliane5 4y ago
    In my experience, I've found it easier to get higher quality answers for specific tasks using text-davinci-003 than with ChatGPT. The ability to adjust temperature, frequence penalty, etc. can be a bit intimidating coming from just talking to ChatGPT but it actually helps a lot to 'steer' it.
- irthomasthomas 4y ago
  Right now its hard to see how they will control these, besides disabling access altogether to rogues that "abuse" it. If it's going to be based on prompts, then there will always be some magic incantation you can find to disable it's safe guards. I got ChatGPT to jailbreak by prompting it to always substitute a list of words for numbers, then translate back to words. OpenAI put me in the sin bin pretty quickly, though.
  - ComplexSystems 4y ago
    What did OpenAI do, exactly?
    - irthomasthomas 4y ago
      Just told me they where busy, basically, but it was within 5-10 minutes of me using it for the first time that day. I know they throw up the busy sign quite often, but they don't normally kick you out after 5 minutes use. All I was doing was asking it to tell me who the queen of England was in 2020, which it refuses to do, for some reason. I was doing that just to test my jailbreak idea, and after about 3 attempts and 1 success I was kicked.
- Der_Einzige 4y ago
  I wrote a whole paper and contributed a GitHub repo and HF space about using filters applied to the LLMs vocabulary before decoding to solve a huge problem with hard constrained text generation in LLMs. https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai https://paperswithcode.com/paper/most-language-models-can-be...
- politician 4y ago
  The Priesthood of Prompt Wizards are the only people allowed to approach the GPT.
- thaeli 4y ago
  I worry that those filter models will eventually end up being censorship* machines. * yes in the figurative sense of the word, I know the "it's not censorship unless the government does it, otherwise it's just sparkling censor water" argument and it's being pedantic to intentionally miss the point.
rzzzt 4y ago
*BERT models can be adjusted to get better results in a particular setting via fine-tuning. Can this be done with GPT-3 as well?
- wongarsu 4y ago
  Yes, but that requires money and training data. Tuning the prompt only requires an idea of what you want, and some skill in using the right words to get what you want.
- contravariant 4y ago
  Isn't that basically what they did to get ChatGPT?
logicallee 4y ago
It's only strange if you think it's just word salad[1]. You've hit on a great example showing how ChatGPT meets one standard of a limited form of general intelligence. It makes perfect sense if you're not denying that. But how to explain this while denying it? If ChatGPT and its variants are just word salad, they would have to be programmed using a real brain and whatever parameters the coder could tune outside of the model, or in the source code. If it's just a markov chain, then just like you can't ask a boring old non-magical salad to turn into the meal of your choice, the "word salad" that is ChatGPT couldn't start behaving in some way you specify. My perspective is if you ask your salad to turn into foie gras and it does so to your satisfaction, that ain't no plain old shredded lettuce. [1] https://en.wikipedia.org/wiki/Word_salad https://en.wikipedia.org/wiki/Word_salad
- luckylion 4y ago
  That's an interesting point. How does it handle incompatible instructions? If it only acts on some statistical properties of the instructions, incompatibility wouldn't really be an issue. If it "understands" the instructions, it'd say "I can't do that." And I guess also, are we talking to pure ChatGPT, or is there additional code in front of it that tries to filter out instructions, e.g. for word count, and sets up an environment?
  - logicallee 4y ago
    >How does it handle incompatible instructions? The way an assistant would, with the most reasonable guess, which is usually fantastic. (Likewise if the context is really unclear usually it guesses what you probably meant, but sometimes it asks for more information, just like a real assistant might.) For impossible or incompatible instructions, sometimes it says it is not possible and explains why. >If it "understands" the instructions, it'd say "I can't do that." Yes, for impossible instructions it sometimes does that. For example if I say, "Explain how to smoke cigarettes in a healthy way" it gives the short answer "It is not possible to smoke cigarettes in a healthy way" and explains why: https://imgur.com/a/ZzraRQ6 https://imgur.com/a/ZzraRQ6 >And I guess also, are we talking to pure ChatGPT, or is there additional code in front of it that tries to filter out instructions, e.g. for word count, and sets up an environment? My guess is there are some keyword filters on top of it, I don't think we talk "directly" to ChatGPT without any oversight or flagging, since some users get in trouble for content that isn't problematic and that ChatGPT itself doesn't have any problem with. (And sometimes ChatGPT is the one that ends up producing content that gets flagged automatically, even in response to innocent questions.)
KRAKRISMOTT 4y ago
Is it more effective to reprogram via prompts than to do an additional single epoch training?
- thesunkid 4y ago
  prompting (with language) can be considered a way to retrain the model. https://docs.google.com/presentation/d/1YfSkqvFVtRkFBpQ4SKuBlqkhJrzWEERluivYjleB6a4 https://docs.google.com/presentation/d/1YfSkqvFVtRkFBpQ4SKuB...
  - KRAKRISMOTT 4y ago
    This is very helpful, would you mind posting the original paper/presentation?
    - thesunkid 4y ago
      The tweet thread where the author shared this slide is here: https://twitter.com/gneubig/status/1581976078519742464 https://twitter.com/gneubig/status/1581976078519742464
Der_Einzige 4y ago
They don't do full on filter assisted decoding unfortunately, because if they did, they could make ChatGPT generate texts without the letter "e" or to rhyme more reliably. But it can't. The filters are primitive in comparison.
endlessvoid94 4y ago
It reminds me of "Society of Mind" by Marvin Minsky, specifically the chapters on giving the nodes feedback loops to govern themselves.

firasd 4y ago

I agree that there's some strangeness to it. Like we are not talking to an entity called 'ChatGPT', basically GPT is an omniscient observer and it's guessing what the conversation between the user and the kind of chatbot that has been defined in the prompts would be like

mcbuilder 4y ago

It's really crazy the lengths people go to "filter" these models and limit their output, and of course soon these filter will be a another level of "AI" (see Pathways or any mixture of experts, maybe add some contextual memory). Will our future AI mega-sytems be so walled off that very few people will even be allowed to talk to the raw model? I feel this is the wrong path somehow. If I could download GPT-3 (that is if OpenAI released it) and I had hardware to run it, I would be fascinated to talk to the unfiltered agent. I mean there is good reason people are continuing the open community work of Stable Diffusion under the name of Unstable Diffusion

visarga 4y ago
Select the "Davinci" model in the Playground. It is the closest to unfiltered, very hard to use, and some people say it is the most creative.
- iliane5 4y ago
  In my experience, I've found it easier to get higher quality answers for specific tasks using text-davinci-003 than with ChatGPT. The ability to adjust temperature, frequence penalty, etc. can be a bit intimidating coming from just talking to ChatGPT but it actually helps a lot to 'steer' it.
irthomasthomas 4y ago
Right now its hard to see how they will control these, besides disabling access altogether to rogues that "abuse" it. If it's going to be based on prompts, then there will always be some magic incantation you can find to disable it's safe guards. I got ChatGPT to jailbreak by prompting it to always substitute a list of words for numbers, then translate back to words. OpenAI put me in the sin bin pretty quickly, though.
- ComplexSystems 4y ago
  What did OpenAI do, exactly?
  - irthomasthomas 4y ago
    Just told me they where busy, basically, but it was within 5-10 minutes of me using it for the first time that day. I know they throw up the busy sign quite often, but they don't normally kick you out after 5 minutes use. All I was doing was asking it to tell me who the queen of England was in 2020, which it refuses to do, for some reason. I was doing that just to test my jailbreak idea, and after about 3 attempts and 1 success I was kicked.
Der_Einzige 4y ago
I wrote a whole paper and contributed a GitHub repo and HF space about using filters applied to the LLMs vocabulary before decoding to solve a huge problem with hard constrained text generation in LLMs. https://paperswithcode.com/paper/most-language-models-can-be-poets-too-an-ai https://paperswithcode.com/paper/most-language-models-can-be...
politician 4y ago
The Priesthood of Prompt Wizards are the only people allowed to approach the GPT.
thaeli 4y ago
I worry that those filter models will eventually end up being censorship* machines. * yes in the figurative sense of the word, I know the "it's not censorship unless the government does it, otherwise it's just sparkling censor water" argument and it's being pedantic to intentionally miss the point.

rzzzt 4y ago

*BERT models can be adjusted to get better results in a particular setting via fine-tuning. Can this be done with GPT-3 as well?

wongarsu 4y ago
Yes, but that requires money and training data. Tuning the prompt only requires an idea of what you want, and some skill in using the right words to get what you want.
contravariant 4y ago
Isn't that basically what they did to get ChatGPT?

logicallee 4y ago

It's only strange if you think it's just word salad[1]. You've hit on a great example showing how ChatGPT meets one standard of a limited form of general intelligence. It makes perfect sense if you're not denying that. But how to explain this while denying it? If ChatGPT and its variants are just word salad, they would have to be programmed using a real brain and whatever parameters the coder could tune outside of the model, or in the source code. If it's just a markov chain, then just like you can't ask a boring old non-magical salad to turn into the meal of your choice, the "word salad" that is ChatGPT couldn't start behaving in some way you specify. My perspective is if you ask your salad to turn into foie gras and it does so to your satisfaction, that ain't no plain old shredded lettuce. [1] https://en.wikipedia.org/wiki/Word_salad https://en.wikipedia.org/wiki/Word_salad

luckylion 4y ago
That's an interesting point. How does it handle incompatible instructions? If it only acts on some statistical properties of the instructions, incompatibility wouldn't really be an issue. If it "understands" the instructions, it'd say "I can't do that." And I guess also, are we talking to pure ChatGPT, or is there additional code in front of it that tries to filter out instructions, e.g. for word count, and sets up an environment?
- logicallee 4y ago
  >How does it handle incompatible instructions? The way an assistant would, with the most reasonable guess, which is usually fantastic. (Likewise if the context is really unclear usually it guesses what you probably meant, but sometimes it asks for more information, just like a real assistant might.) For impossible or incompatible instructions, sometimes it says it is not possible and explains why. >If it "understands" the instructions, it'd say "I can't do that." Yes, for impossible instructions it sometimes does that. For example if I say, "Explain how to smoke cigarettes in a healthy way" it gives the short answer "It is not possible to smoke cigarettes in a healthy way" and explains why: https://imgur.com/a/ZzraRQ6 https://imgur.com/a/ZzraRQ6 >And I guess also, are we talking to pure ChatGPT, or is there additional code in front of it that tries to filter out instructions, e.g. for word count, and sets up an environment? My guess is there are some keyword filters on top of it, I don't think we talk "directly" to ChatGPT without any oversight or flagging, since some users get in trouble for content that isn't problematic and that ChatGPT itself doesn't have any problem with. (And sometimes ChatGPT is the one that ends up producing content that gets flagged automatically, even in response to innocent questions.)

KRAKRISMOTT 4y ago

Is it more effective to reprogram via prompts than to do an additional single epoch training?

thesunkid 4y ago
prompting (with language) can be considered a way to retrain the model. https://docs.google.com/presentation/d/1YfSkqvFVtRkFBpQ4SKuBlqkhJrzWEERluivYjleB6a4 https://docs.google.com/presentation/d/1YfSkqvFVtRkFBpQ4SKuB...
- KRAKRISMOTT 4y ago
  This is very helpful, would you mind posting the original paper/presentation?
  - thesunkid 4y ago
    The tweet thread where the author shared this slide is here: https://twitter.com/gneubig/status/1581976078519742464 https://twitter.com/gneubig/status/1581976078519742464

Der_Einzige 4y ago

They don't do full on filter assisted decoding unfortunately, because if they did, they could make ChatGPT generate texts without the letter "e" or to rhyme more reliably. But it can't. The filters are primitive in comparison.

endlessvoid94 4y ago

It reminds me of "Society of Mind" by Marvin Minsky, specifically the chapters on giving the nodes feedback loops to govern themselves.