3 ms·

Okay, this isn't quite what you asked but it's close: we can decide to go on an errand and remember what we were doing even if we didn't write it down. (Though

Okay, this isn't quite what you asked but it's close: we can decide to go on an errand and remember what we were doing even if we didn't write it down. (Though sometimes we do forget, so shopping lists are useful.) This doesn't mean our justifications are accurate, but at least we do sometimes have "debug logs," some memories (perhaps faint and inaccurate) of internal thoughts that we can use to try to reconstruct what we were thinking. An LLM cannot remember anything it didn't write down. It can't do that at all. They would need to change the API.

TeMPOraL 4y ago
LLM doesn't have any other runtime state but for what it wrote down (assuming the user even provides it). For comparing with humans, I think LLMs are not like our minds, but like the inner voice. Chat history is the working memory.
- skybrian 4y ago
  Yep, and we have the text. It’s very transparent that way. But it might be worth thinking about what a satisfying explanation of the LLM’s behavior would look like. It would have to be a high-level summary of the calculations that were done to predict each new token. Knowing which previous tokens got attention and what higher-level concepts it calculated would help. (Also, if the temperature isn’t zero, there’s a random number generator involved.) The LLM doesn’t have access to that, once the calculation is done. It has the text, same as us. It can guess what might have happened, same as us. When it guesses it’s going to anthropomorphize by picking an explanation that would make sense for a human, because it’s trained to write justifications that a human would write.
naasking 4y ago
> An LLM cannot remember anything it didn't write down. It can't do that at all. Ok, but I'm not sure what that has to do with knowledge. You can ask ChatGPT to reflect on and correct its own answers, so it clearly has access to its own outputs, which is a form of memory.
- jochem9 4y ago
  By default ChatGPT keeps a context of 8000 tokens, which includes user provided prompts. That's the size of the working memory.

TeMPOraL 4y ago

LLM doesn't have any other runtime state but for what it wrote down (assuming the user even provides it). For comparing with humans, I think LLMs are not like our minds, but like the inner voice. Chat history is the working memory.

skybrian 4y ago
Yep, and we have the text. It’s very transparent that way. But it might be worth thinking about what a satisfying explanation of the LLM’s behavior would look like. It would have to be a high-level summary of the calculations that were done to predict each new token. Knowing which previous tokens got attention and what higher-level concepts it calculated would help. (Also, if the temperature isn’t zero, there’s a random number generator involved.) The LLM doesn’t have access to that, once the calculation is done. It has the text, same as us. It can guess what might have happened, same as us. When it guesses it’s going to anthropomorphize by picking an explanation that would make sense for a human, because it’s trained to write justifications that a human would write.

naasking 4y ago

> An LLM cannot remember anything it didn't write down. It can't do that at all. Ok, but I'm not sure what that has to do with knowledge. You can ask ChatGPT to reflect on and correct its own answers, so it clearly has access to its own outputs, which is a form of memory.

jochem9 4y ago
By default ChatGPT keeps a context of 8000 tokens, which includes user provided prompts. That's the size of the working memory.