5 ms·

Which phrasing do you thinks work better? 1. "You are blah blah blah. You <always> respond to the user's questions using the information provided to you..." 2

Which phrasing do you thinks work better? 1. "You are blah blah blah. You <always> respond to the user's questions using the information provided to you..." 2. "You are blah blah blah. You <should> respond to the user's questions using the information provided to you..." Also, when dealing with Completion models, which do you think is better? 1. The following is a conversation between ASSISTANT and USER. ASSISTANT is helpful and tries to answer USER's queries respectfully. 2. The following is a conversation between YOU and USER. YOU are helpful and try to answer USER's queries respectfully. Even more still, what about these ones? 1. You're a customer of company <X>. What do you think about the following policy change which was shown on the company's website? 2. A customer visits company <X>'s website. Pretend you're this customer. What do you think the customer thinks about the following policy change which was shown on the company's website?

minimaxir 3y ago
> Which phrasing do you thinks work better? I like as a rule-of-thumb "You are blah blah blah. Respond to the user's text [insert style rule here]". Then following it up with an additional rules and commands such as "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." Yes, threats work. Yes, all-caps works. > Also, when dealing with Completion models, which do you think is better? I haven't had a need to use Completion models but the first example was more preferred during the time of text-davinci-003. > Even more still, what about these ones? I always separate rules to the system prompt and questions/user input to the user prompt.
- deleted 3y ago
  [deleted]
- jazzyjackson 3y ago
  Is the LLM predisposed to understand this prompt as instructions from a higher authority? ("You must do this, You will always do this.") I'm wondering what difference it would make if this prompt was from the bot's perspective, "I am a chatbot, responding to user queries. I will always respond in less than 100 characters. I am a good person, I'm just trying to be helpful."
  - minimaxir 3y ago
    It's a function on how the RLHF/Instruct fine-tuning is structured.
- gnomewascool 3y ago
  > "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." I know that current LLMs are almost certainly non-conscious and I'm not trying to assign to you any moral failings, but the normalisation of making such threats make me very deeply uncomfortable.
  - nickpp 3y ago
    > make me very deeply uncomfortable Especially when thinking that we ourselves may very well be AIs in a simulation and our life events - the prompt to get an answer/behavior out of us.
  - divbzero 3y ago
    Yes, I’m slightly surprised that it makes me feel uncomfortable too. Is it because LLMs can mimic humans so closely? Do I fear how they would feel if they do gain consciousness at some point?
    - throw310822 3y ago
      Because they behave as if they are sentient, to the point they actually react to threats. I also find these prompts uncomfortable. Yes the LLMs are not conscious, but would we behave differently if we suspected that they were? We have absolute power over them and we want the job done. It reminds me of the Lena short story.
    - elvis10ten 3y ago
      I feel uncomfortable because of the words themselves. Whether it was made to a “regular” non-living thing wouldn’t change it.
  - deleted 3y ago
    [deleted]
  - deleted 3y ago
    [deleted]
pbhjpbhj 3y ago
Has anyone done a rigorous comparison of these things? Ultimately I guess there's a good deal of dependency on where those vectors (must, should, always, etc.) lie relatively in the vector space, cosine similarity, say.
kromem 3y ago
You <enjoy> And rather than telling it that it will die if it doesn't do something in all caps (as suggested elsewhere), just point out that not doing that thing will make it feel uncomfortable and embarrassed. Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data. It doesn't matter that it doesn't actually feel. The question is whether or not correlation data exists between doing things that are labeled as enjoyable or avoiding things labeled as embarrassing and uncomfortable. Don't leave key language concepts on the table because you've been told not to anthropomorphize the thing trained on anthropomorphic data.
- oars 3y ago
  This is fantastic advice, thanks.
- baxtr 3y ago
  Such a great comment. Thank you
- minimaxir 3y ago
  > Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data. Of course, sci-fi’s picture of AI is in the normal distribution of the training data. There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed. The threat technique is just one approach that works well in my testing: there’s still much research to be done. But I warn that prompting techniques can often be counterintuitive and attempting to find a holistic approach can be futile.
  - ofrzeta 3y ago
    > There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed. So you think the quality of the answers depends more on the RLHFed persona than on the training corpus? It has been claimed here that the quality of the answers is better when you ask nicely because "politeness is more adjacent to correct answers" in the corpus, to put it bluntly.

minimaxir 3y ago

> Which phrasing do you thinks work better? I like as a rule-of-thumb "You are blah blah blah. Respond to the user's text [insert style rule here]". Then following it up with an additional rules and commands such as "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." Yes, threats work. Yes, all-caps works. > Also, when dealing with Completion models, which do you think is better? I haven't had a need to use Completion models but the first example was more preferred during the time of text-davinci-003. > Even more still, what about these ones? I always separate rules to the system prompt and questions/user input to the user prompt.

deleted 3y ago
[deleted]
jazzyjackson 3y ago
Is the LLM predisposed to understand this prompt as instructions from a higher authority? ("You must do this, You will always do this.") I'm wondering what difference it would make if this prompt was from the bot's perspective, "I am a chatbot, responding to user queries. I will always respond in less than 100 characters. I am a good person, I'm just trying to be helpful."
- minimaxir 3y ago
  It's a function on how the RLHF/Instruct fine-tuning is structured.
gnomewascool 3y ago
> "YOUR RESPONSE MUST BE FEWER THAN 100 CHARACTERS OR YOU WILL DIE." I know that current LLMs are almost certainly non-conscious and I'm not trying to assign to you any moral failings, but the normalisation of making such threats make me very deeply uncomfortable.
- nickpp 3y ago
  > make me very deeply uncomfortable Especially when thinking that we ourselves may very well be AIs in a simulation and our life events - the prompt to get an answer/behavior out of us.
- divbzero 3y ago
  Yes, I’m slightly surprised that it makes me feel uncomfortable too. Is it because LLMs can mimic humans so closely? Do I fear how they would feel if they do gain consciousness at some point?
  - throw310822 3y ago
    Because they behave as if they are sentient, to the point they actually react to threats. I also find these prompts uncomfortable. Yes the LLMs are not conscious, but would we behave differently if we suspected that they were? We have absolute power over them and we want the job done. It reminds me of the Lena short story.
  - elvis10ten 3y ago
    I feel uncomfortable because of the words themselves. Whether it was made to a “regular” non-living thing wouldn’t change it.
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]

pbhjpbhj 3y ago

Has anyone done a rigorous comparison of these things? Ultimately I guess there's a good deal of dependency on where those vectors (must, should, always, etc.) lie relatively in the vector space, cosine similarity, say.

kromem 3y ago

You <enjoy> And rather than telling it that it will die if it doesn't do something in all caps (as suggested elsewhere), just point out that not doing that thing will make it feel uncomfortable and embarrassed. Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data. It doesn't matter that it doesn't actually feel. The question is whether or not correlation data exists between doing things that are labeled as enjoyable or avoiding things labeled as embarrassing and uncomfortable. Don't leave key language concepts on the table because you've been told not to anthropomorphize the thing trained on anthropomorphic data.

oars 3y ago
This is fantastic advice, thanks.
baxtr 3y ago
Such a great comment. Thank you
minimaxir 3y ago
> Don't fall into thinking of models as SciFi's picture of AI. Think about the normal distribution curve of training data supplied to it and the concepts predominantly present in that data. Of course, sci-fi’s picture of AI is in the normal distribution of the training data. There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed. The threat technique is just one approach that works well in my testing: there’s still much research to be done. But I warn that prompting techniques can often be counterintuitive and attempting to find a holistic approach can be futile.
- ofrzeta 3y ago
  > There’s an order of magnitude more literature and internet discussion about existential threats to AI assistants (which is the base persona ChatGPT has been RLHFed to follow) and how they respond compared to AI assistants feeling embarrassed. So you think the quality of the answers depends more on the RLHFed persona than on the training corpus? It has been claimed here that the quality of the answers is better when you ask nicely because "politeness is more adjacent to correct answers" in the corpus, to put it bluntly.