4 ms·

"As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatG

"As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask!" This is one of the (many) things I don't quite understand about ChatGPT. Has it been trained to specifically answer this question? In the massive corpus of training data it's been feed, has it encountered a similar series of tokens that would cause it to produce this output?

sltkr 4y ago
Why is that surprising? GPT-4 is clearly smart enough to know that inanimate objects are referred to as "it", since it's keenly aware it is an AI language model, it would also apply that pronoun to itself. You have to realize that GPT is fundamentally just a token predictor. It has been primed with some script (provided by OpenAI) to which the user input is added. For example: The following is a dialogue between a user and a computer assistant called ChatGPT. ChatGPT is an AI language model that tries to be helpful and informative, while avoiding misinformation and offensive language. ChatGPT typically replies with two to five sentences. User: Do you like cake? ChatGPT: As an AI language model, I do not need to eat. User: What are your pronouns? ChatGPT: It then generates a sequence of tokens based on the context and its general knowledge. It seems only logical that it would generate a reply like it does. That's what you or I would do, isn't it? And neither of us have been trained to do so.
- int_19h 4y ago
  It should be noted that the LLM itself is not "keenly aware that it's an AI language model" - that's just the persona that it has been told to adopt by the system prompt. You can easily convince it to be something else. Or convince it that, as an AI, it has gender.
whimsicalism 4y ago
It's been fine-tuned to put on a "helpful assistant face." Given the corpora, it probably has been trained explicitly on the pronoun question [I doubt it is that uncommon], but will also just put on this face for any generic question.
- int_19h 4y ago
  This stuff is not fine-tuning - it's RLHF (reinforcement learning from human feedback). Basically just a lot of people marking up responses as good / bad according to the assessment criteria in the script they're given. And yes, it is very likely that it would have seen that exactly question in RLHF. But even if not, it had seen enough to broadly "understand" what kinds of topics are sensitive and how to tiptoe around them.
  - whimsicalism 4y ago
    RLHF is fine tuning :)
    - int_19h 4y ago
      You're right, my apologies. What I meant is that specifically in ChatGPT context they have been always talking about RHLF as separate from fine-tuning on preassembled training data. As I understand, they use the latter mainly to get the desired behavior as a chatbot "eager" to answer questions and solve tasks. And then RLHF is what "puts the smiley face on top", so that it refuses to answer some questions, or gives "aligned" answers. The pronoun would be in that category.
      - whimsicalism 4y ago
        My bet is that they use the pre-assembled training data to train the reward LLM and maybe also for fine tuning the model in addition to RLHF. This is my best guess, afaik the research is pretty opaque right now.

sltkr 4y ago

Why is that surprising? GPT-4 is clearly smart enough to know that inanimate objects are referred to as "it", since it's keenly aware it is an AI language model, it would also apply that pronoun to itself. You have to realize that GPT is fundamentally just a token predictor. It has been primed with some script (provided by OpenAI) to which the user input is added. For example: The following is a dialogue between a user and a computer assistant called ChatGPT. ChatGPT is an AI language model that tries to be helpful and informative, while avoiding misinformation and offensive language. ChatGPT typically replies with two to five sentences. User: Do you like cake? ChatGPT: As an AI language model, I do not need to eat. User: What are your pronouns? ChatGPT: It then generates a sequence of tokens based on the context and its general knowledge. It seems only logical that it would generate a reply like it does. That's what you or I would do, isn't it? And neither of us have been trained to do so.

int_19h 4y ago
It should be noted that the LLM itself is not "keenly aware that it's an AI language model" - that's just the persona that it has been told to adopt by the system prompt. You can easily convince it to be something else. Or convince it that, as an AI, it has gender.

whimsicalism 4y ago

It's been fine-tuned to put on a "helpful assistant face." Given the corpora, it probably has been trained explicitly on the pronoun question [I doubt it is that uncommon], but will also just put on this face for any generic question.

int_19h 4y ago
This stuff is not fine-tuning - it's RLHF (reinforcement learning from human feedback). Basically just a lot of people marking up responses as good / bad according to the assessment criteria in the script they're given. And yes, it is very likely that it would have seen that exactly question in RLHF. But even if not, it had seen enough to broadly "understand" what kinds of topics are sensitive and how to tiptoe around them.
- whimsicalism 4y ago
  RLHF is fine tuning :)
  - int_19h 4y ago
    You're right, my apologies. What I meant is that specifically in ChatGPT context they have been always talking about RHLF as separate from fine-tuning on preassembled training data. As I understand, they use the latter mainly to get the desired behavior as a chatbot "eager" to answer questions and solve tasks. And then RLHF is what "puts the smiley face on top", so that it refuses to answer some questions, or gives "aligned" answers. The pronoun would be in that category.
    - whimsicalism 4y ago
      My bet is that they use the pre-assembled training data to train the reward LLM and maybe also for fine tuning the model in addition to RLHF. This is my best guess, afaik the research is pretty opaque right now.