6 ms·

No, the session history is all in the prompt, including the LLM's previous responses.

• tayo42 1y ago
  Appending more context to the existing prompt means it's a different prompt still... The text isn't the same
  • thunky 1y ago
    I'm not sure what your point is? Think of it like an append only journal. To correct an entry you add a new one with the correction. The LLM sees the mistake and the correction. That's no worse then mutating the history.
    • tayo42 1y ago
      thats not how it works. You put in its context window some more information, then roll the dice again. And it produces text again token by token. Its still not planning ahead and its not looking back at what was just generated. There's no guarantee everything stays the same except the mistake. This is different then editing in place. you are rolling the dice again.
      • thunky 1y ago
        > its not looking back at what was just generated It is, though. The LLM gets the full history in every prompt until you start a new session. That's why it gets slower as the conversation/context gets big. The developer could choose to rewrite or edit the history before sending it back to the LLM but the user typically can't. > There's no guarantee everything stays the same except the mistake Sure, but there's no guarantee about anything it will generate. But that's a separate issue.