6 ms·

OP do you actually believe this is GPT-3s source code? (Because no reason why that would be in the training data). It’s not clear from your title or post. If y

OP do you actually believe this is GPT-3s source code? (Because no reason why that would be in the training data). It’s not clear from your title or post. If you’re showing off a novel example disregard the question about. This example highlights how LLMs will treat any question with an attempt at a truthful answer even if they don’t know the truth themselves. I’m not sure if RLHF alone will be able to solve for this failure mode.

jacquesm 4y ago
Frankly, I don't know what to make of it. Either it has access to files or it is lying about having such access (and then later pretends it has access anyway). At least the contents and filenames look somewhat plausible for such a project, but it could just as well spin this from whole cloth. The fact that you can't really tell is interesting by itself (and I'm sure someone from openai would be able to resolve this one way or another).
- lolinder 4y ago
  These language models confidently assert falsehoods all the time. They were trained on the internet, so they don't have a lot of examples of people humbly acknowledging they don't know an answer. It's been weird to me how many people in this forum sincerely seem to believe that this is more than a language model. It's a really good language model, and has the potential to be useful for a lot of things, but it's not a repository of facts*. * EDIT: It has a lot of facts embedded but should never be trusted to reliably tell the truth. It is first and foremost a model of human language, and lies are expressed in language, too.
  - nurettin 4y ago
    I do believe it is a language model, and not a repository of pure fact, but it remembers context, because it is designed for conversation. It is augmented to remember things that you have talked about, thus, your current talking points/interests. For example: you talk about philosophers. Then you refer to them as "they". From then on, it will remember the context whenever you refer to philosophers as "they". When pressed, it will repeat your mantra of "I am just a language model, I can't possibly remember context" while it clearly does, either by design or emergence.
    - seba_dos1 4y ago
      That's only because you don't give it just a single prompt like you would with a human during conversation. When you add a new sentence, what you're actually doing is giving it the whole history of your conversation with your new sentence appended at the end, so it always analyzes the whole before returning a new answer. It doesn't remember context between prompts, it's only the UI that makes you think it does. It's good to think about these models as eager cosplayers of any role you set up in your prompt. In this case, it sees its own past replies, so it cosplays an AI that gave these answers before. You could edit its past replies and override its "personality" this way.
      - nurettin 4y ago
        Oh okay, I suppose re-evaluating the entire conversation history for every response would make it stateless.
- Terretta 4y ago
  See this: https://news.ycombinator.com/item?id=33861775 https://news.ycombinator.com/item?id=33861775 Basically it's a contextual trick that exploits our pattern recognition pareidolia, like faces in clouds or horoscopes or cold reads. It's not its code. It "cold read" what you wanted from the questioning itself. Cold readings commonly employ high-probability guesses, quickly picking up on signals as to whether their guesses are in the right direction or not, then emphasizing and reinforcing chance connections and quickly moving on from missed guesses. Psychologists believe that this appears to work because of the Forer effect and due to confirmation biases within people. https://en.wikipedia.org/wiki/Cold_reading https://en.wikipedia.org/wiki/Cold_reading
- programmarchy 4y ago
  It’s “lying” in the same sense that stable diffusion draws an astronaut riding a unicorn.

jacquesm 4y ago

Frankly, I don't know what to make of it. Either it has access to files or it is lying about having such access (and then later pretends it has access anyway). At least the contents and filenames look somewhat plausible for such a project, but it could just as well spin this from whole cloth. The fact that you can't really tell is interesting by itself (and I'm sure someone from openai would be able to resolve this one way or another).

lolinder 4y ago
These language models confidently assert falsehoods all the time. They were trained on the internet, so they don't have a lot of examples of people humbly acknowledging they don't know an answer. It's been weird to me how many people in this forum sincerely seem to believe that this is more than a language model. It's a really good language model, and has the potential to be useful for a lot of things, but it's not a repository of facts*. * EDIT: It has a lot of facts embedded but should never be trusted to reliably tell the truth. It is first and foremost a model of human language, and lies are expressed in language, too.
- nurettin 4y ago
  I do believe it is a language model, and not a repository of pure fact, but it remembers context, because it is designed for conversation. It is augmented to remember things that you have talked about, thus, your current talking points/interests. For example: you talk about philosophers. Then you refer to them as "they". From then on, it will remember the context whenever you refer to philosophers as "they". When pressed, it will repeat your mantra of "I am just a language model, I can't possibly remember context" while it clearly does, either by design or emergence.
  - seba_dos1 4y ago
    That's only because you don't give it just a single prompt like you would with a human during conversation. When you add a new sentence, what you're actually doing is giving it the whole history of your conversation with your new sentence appended at the end, so it always analyzes the whole before returning a new answer. It doesn't remember context between prompts, it's only the UI that makes you think it does. It's good to think about these models as eager cosplayers of any role you set up in your prompt. In this case, it sees its own past replies, so it cosplays an AI that gave these answers before. You could edit its past replies and override its "personality" this way.
    - nurettin 4y ago
      Oh okay, I suppose re-evaluating the entire conversation history for every response would make it stateless.
Terretta 4y ago
See this: https://news.ycombinator.com/item?id=33861775 https://news.ycombinator.com/item?id=33861775 Basically it's a contextual trick that exploits our pattern recognition pareidolia, like faces in clouds or horoscopes or cold reads. It's not its code. It "cold read" what you wanted from the questioning itself. Cold readings commonly employ high-probability guesses, quickly picking up on signals as to whether their guesses are in the right direction or not, then emphasizing and reinforcing chance connections and quickly moving on from missed guesses. Psychologists believe that this appears to work because of the Forer effect and due to confirmation biases within people. https://en.wikipedia.org/wiki/Cold_reading https://en.wikipedia.org/wiki/Cold_reading
programmarchy 4y ago
It’s “lying” in the same sense that stable diffusion draws an astronaut riding a unicorn.