3 ms·

Again, you have memory but models don't. If it's not in the context, it doesn't exist. This is really easy to prove: delete your question from the context and a

Again, you have memory but models don't. If it's not in the context, it doesn't exist. This is really easy to prove: delete your question from the context and ask the model why it answered the way it did. It will not know, because the model prediction service is stateless. It only takes the context as input. It's like if you and I weren't able to remember anything that wasn't written down. That's why you can get models to approximate thought by telling it to write down its intermediate steps.

stavros 3y ago
Right, exactly, the context is the model's memory. All you're saying is "except for their memory, they have no memory", which goes for humans as well.
- phailhaus 3y ago
  If you say that the models have memory and can think, you are heavily implying that there is a brain in the cloud that is capable of "thinking" or "reasoning" through your request. That's what it means for humans: we're self-contained. Models aren't like people. The context is not "part" of the model, the context is given to the model when you ask it for a prediction. It's like you cut a small part of a person's brain out. Alone, it can't think. It's a pedantic distinction, but an important one. The model itself isn't capable of thinking, but if you package it up with a context that it can manipulate at-will, that combination of parts can be said to "think". That's probably going to be the next step in LLM tech.
  - stavros 3y ago
    By that definition, ChatGPT can already think.
    - water-your-self 3y ago
      If this is your conclusion, I would recommend reading more of the research.
    - phailhaus 3y ago
      Uh, no? ChatGPT does not have a memory outside of the context that you submit.
      - stavros 3y ago
        https://openai.com/blog/memory-and-new-controls-for-chatgpt https://openai.com/blog/memory-and-new-controls-for-chatgpt
      - zoogeny 3y ago
        I think your use of the word "memory" here is imprecise. For example, I can ask ChatGPT "Give me a 200 word summary of George Orwell's book Animal Farm". It gives me a pretty cogent description of the novel. That knowledge of Animal Farm is somewhere, not in the context. If we don't call that memory, I'm not sure what to call it. Why should I think of this as different than my own memories of the book?
        phailhaus 3y ago
        That's encoded in the model weights, not "memory". Basically, there is no context outside of the context that you give the model. When you ask it a question, those model weights don't change. It doesn't "remember" what you asked. This is why chain-of-thought reasoning works so effectively: it lets the model "use" the context as a sort of scratch pad to build up a response. Without it, the model isn't capable of mimicking thought because it's only capable of predicting based on the current context.
        esafak 3y ago
        That is memory. Ask ChatGPT a basic fact about universe and it will tell you because it has memorized the answer. You are asking it to learn, or incrementally create new memories. That just has not been implemented yet, due to costs.
kelseyfrog 3y ago
If models had memory would it change your mind about whether they could think or not?

stavros 3y ago

Right, exactly, the context is the model's memory. All you're saying is "except for their memory, they have no memory", which goes for humans as well.

phailhaus 3y ago
If you say that the models have memory and can think, you are heavily implying that there is a brain in the cloud that is capable of "thinking" or "reasoning" through your request. That's what it means for humans: we're self-contained. Models aren't like people. The context is not "part" of the model, the context is given to the model when you ask it for a prediction. It's like you cut a small part of a person's brain out. Alone, it can't think. It's a pedantic distinction, but an important one. The model itself isn't capable of thinking, but if you package it up with a context that it can manipulate at-will, that combination of parts can be said to "think". That's probably going to be the next step in LLM tech.
- stavros 3y ago
  By that definition, ChatGPT can already think.
  - water-your-self 3y ago
    If this is your conclusion, I would recommend reading more of the research.
  - phailhaus 3y ago
    Uh, no? ChatGPT does not have a memory outside of the context that you submit.
    - stavros 3y ago
      https://openai.com/blog/memory-and-new-controls-for-chatgpt https://openai.com/blog/memory-and-new-controls-for-chatgpt
    - zoogeny 3y ago
      I think your use of the word "memory" here is imprecise. For example, I can ask ChatGPT "Give me a 200 word summary of George Orwell's book Animal Farm". It gives me a pretty cogent description of the novel. That knowledge of Animal Farm is somewhere, not in the context. If we don't call that memory, I'm not sure what to call it. Why should I think of this as different than my own memories of the book?
      - phailhaus 3y ago
        That's encoded in the model weights, not "memory". Basically, there is no context outside of the context that you give the model. When you ask it a question, those model weights don't change. It doesn't "remember" what you asked. This is why chain-of-thought reasoning works so effectively: it lets the model "use" the context as a sort of scratch pad to build up a response. Without it, the model isn't capable of mimicking thought because it's only capable of predicting based on the current context.
        esafak 3y ago
        That is memory. Ask ChatGPT a basic fact about universe and it will tell you because it has memorized the answer. You are asking it to learn, or incrementally create new memories. That just has not been implemented yet, due to costs.

kelseyfrog 3y ago

If models had memory would it change your mind about whether they could think or not?