5 ms·

Correct me if I’m wrong but this concept of state is across a prediction which is a series of token proposals. Not a sense of persistent state over, say, a chat

Correct me if I’m wrong but this concept of state is across a prediction which is a series of token proposals. Not a sense of persistent state over, say, a chat session.

comfypotato 3y ago
Correct, the state is during propagation. You could really even propose that a basic feedforward network has state during propagation. I think the proposition is that it’s conscious/sentient during propagation. I’ve got a graduate level background in cellular neurophysiology, and I’ll admit that I consider this a lot. In the end, I don’t think it will ever matter because a computer’s consciousness/sentience necessarily depends on humans not wiping memory.
- turtleyacht 3y ago
  > conscious/sentient during propagation Interesting. We've instituted a micro unit of sentience within an observable time slice: Not for nothing have-- We put in a mind within In conscious seconds
  - comfypotato 3y ago
    It’s begging for new sub-genres of the philosophy of the mind.
GistNoesis 3y ago
At each new token processed (edit: either if the token comes from the user or has been generated), this dynamic internal state is updated via a formula based on the model parameters and latest processed token. To make this more clear, if you process 1000 tokens the neural network will have gone through a sequence of 1000 states. Each of these states can be probed and analysed, by training a classifier on top of this state, for the content they contain (whether thoughts or emotion : some states can be classified as "happy", "sad",...). This is not yet mainstream view but viewed through this prism and anthropomorphising a little, it can been seen as a stream of proto-consciousness, where during the conversation the inner state of the neural network has gone through various thoughts and emotions. At the end of the chat session, this internal state is not persisted (but could be recreated from the produced conversation as it is deterministic). This internal state size is big and proportional to the length of the context window (If you want to persist between sessions you can by simply keeping the last "context window size" tokens produced and recompute the features). At the next session you start with a fresh new internal state. The conversation produced is persisted for use as input for future training where good conversation will be encouraged and bad conversation will be discouraged via Reinforcement Learning with Human Feedback. The dynamics of this internal state is what Large Language Models learn.
- fragsworth 3y ago
  > At the end of the chat session, this internal state is not persisted (but could be recreated from the produced conversation as it is deterministic) You're mistaken about internal state in a chat session. It loses all state after each response. There is internal state for a moment during a single response, but it's not persisted once it's done. Between messages, it has no memory of previous messages in your conversation. The way it's implemented is by the seemingly silly/expensive method of pre-pending the conversation history to your request to the LLM. The "internal state" is usually stored on your local machine.
  - MacsHeadroom 3y ago
    I believe you misunderstood the comment you're replying to. It says the internal state could be recreated because it is deterministic, and this is correct. Assuming you have control of the parameters (seed etc.) you can reproduce the exact internal states by submitting the same tokens with the same parameters over again step-by-step. With GPT-4 you can do this in the OpenAI Dialogue Playground or via the API.
    - fragsworth 3y ago
      > I believe you misunderstood the comment you're replying to. No, did you even read my quote? How did I misunderstand this: > At the end of the chat session, this internal state is not persisted It's misinformed at best, misleading at worst. The non-misleading truth is at the end of each message, the internal state is not persisted. I didn't disagree about how you can reproduce things. You have to do that to continue a conversation.
      - MacsHeadroom 3y ago
        >I didn't disagree about how you can reproduce things. You have to do that to continue a conversation. You do not have to reproduce the internal states to continue a conversation. When prior parts of the conversation are loaded into context the hidden states which generated those prior tokens are not reproduced. They are only reproduced if resubmitted in a piecemeal fashion, which does not happen in normal conversation continuation. You seem to not understand what the internal state is or how it is differentiated from the external state of the overall conversation.
lucubratory 3y ago
LLMs form an internal world model that is persistent over a chat session: https://thegradient.pub/othello/ https://thegradient.pub/othello/

Correct me if I’m wrong but this concept of state is across a prediction which is a series of token proposals. Not a sense of persistent state over, say, a chat session.

comfypotato 3y ago

Correct, the state is during propagation. You could really even propose that a basic feedforward network has state during propagation. I think the proposition is that it’s conscious/sentient during propagation. I’ve got a graduate level background in cellular neurophysiology, and I’ll admit that I consider this a lot. In the end, I don’t think it will ever matter because a computer’s consciousness/sentience necessarily depends on humans not wiping memory.

turtleyacht 3y ago
> conscious/sentient during propagation Interesting. We've instituted a micro unit of sentience within an observable time slice: Not for nothing have-- We put in a mind within In conscious seconds
- comfypotato 3y ago
  It’s begging for new sub-genres of the philosophy of the mind.

GistNoesis 3y ago

At each new token processed (edit: either if the token comes from the user or has been generated), this dynamic internal state is updated via a formula based on the model parameters and latest processed token. To make this more clear, if you process 1000 tokens the neural network will have gone through a sequence of 1000 states. Each of these states can be probed and analysed, by training a classifier on top of this state, for the content they contain (whether thoughts or emotion : some states can be classified as "happy", "sad",...). This is not yet mainstream view but viewed through this prism and anthropomorphising a little, it can been seen as a stream of proto-consciousness, where during the conversation the inner state of the neural network has gone through various thoughts and emotions. At the end of the chat session, this internal state is not persisted (but could be recreated from the produced conversation as it is deterministic). This internal state size is big and proportional to the length of the context window (If you want to persist between sessions you can by simply keeping the last "context window size" tokens produced and recompute the features). At the next session you start with a fresh new internal state. The conversation produced is persisted for use as input for future training where good conversation will be encouraged and bad conversation will be discouraged via Reinforcement Learning with Human Feedback. The dynamics of this internal state is what Large Language Models learn.

fragsworth 3y ago
> At the end of the chat session, this internal state is not persisted (but could be recreated from the produced conversation as it is deterministic) You're mistaken about internal state in a chat session. It loses all state after each response. There is internal state for a moment during a single response, but it's not persisted once it's done. Between messages, it has no memory of previous messages in your conversation. The way it's implemented is by the seemingly silly/expensive method of pre-pending the conversation history to your request to the LLM. The "internal state" is usually stored on your local machine.
- MacsHeadroom 3y ago
  I believe you misunderstood the comment you're replying to. It says the internal state could be recreated because it is deterministic, and this is correct. Assuming you have control of the parameters (seed etc.) you can reproduce the exact internal states by submitting the same tokens with the same parameters over again step-by-step. With GPT-4 you can do this in the OpenAI Dialogue Playground or via the API.
  - fragsworth 3y ago
    > I believe you misunderstood the comment you're replying to. No, did you even read my quote? How did I misunderstand this: > At the end of the chat session, this internal state is not persisted It's misinformed at best, misleading at worst. The non-misleading truth is at the end of each message, the internal state is not persisted. I didn't disagree about how you can reproduce things. You have to do that to continue a conversation.
    - MacsHeadroom 3y ago
      >I didn't disagree about how you can reproduce things. You have to do that to continue a conversation. You do not have to reproduce the internal states to continue a conversation. When prior parts of the conversation are loaded into context the hidden states which generated those prior tokens are not reproduced. They are only reproduced if resubmitted in a piecemeal fashion, which does not happen in normal conversation continuation. You seem to not understand what the internal state is or how it is differentiated from the external state of the overall conversation.

lucubratory 3y ago

LLMs form an internal world model that is persistent over a chat session: https://thegradient.pub/othello/ https://thegradient.pub/othello/