4 ms·

Clue on conversation "history"-- "While ChatGPT is able to remember what the user has said earlier in the conversation, there is a limit to how much information

Clue on conversation "history"-- "While ChatGPT is able to remember what the user has said earlier in the conversation, there is a limit to how much information it can retain. The model is able to reference up to approximately 3000 words (or 4000 tokens) from the current conversation - any information beyond that is not stored. Please note that ChatGPT is not able to access past conversations to inform its responses." https://help.openai.com/en/articles/6787051-does-chatgpt-remember-what-happened-earlier-in-the-conversation https://help.openai.com/en/articles/6787051-does-chatgpt-rem... Some interesting techniques I've seen involve essentially a ring-buffer and after each turn a call is made to summarize the conversation up to that point and use that as context for subsequent prompt

roxgib 4y ago
Presumably one of the benefits of running your own model is that you can feed extra data into it via training rather than purely through inference? I.e. if you're a software company you could fine-tune it on your codebase, improving its answers without increasing inference time?
- MagicMoonlight 4y ago
  ChatGPT is itself just a GPT that has been finetuned to make it act as a chat bot and try not to be offensive
ProllyInfamous 4y ago
>essentially a ring-buffer and after each turn a call is made to summarize the conversation up to that point and use that as context for subsequent prompt Although people unfamiliar with "active listening" often find it annoying, this is something I attempt to implement within my conversations (quickly summarizing what I understand after somebody explains something technical). Oftentimes people feel like I'm `interrupting them`, but it is really just a way to stay engaged in two-way conversation. Eerie how LLM's benefit from this "focus" as similarly as this autistic commenter.

roxgib 4y ago

Presumably one of the benefits of running your own model is that you can feed extra data into it via training rather than purely through inference? I.e. if you're a software company you could fine-tune it on your codebase, improving its answers without increasing inference time?

MagicMoonlight 4y ago
ChatGPT is itself just a GPT that has been finetuned to make it act as a chat bot and try not to be offensive

ProllyInfamous 4y ago

>essentially a ring-buffer and after each turn a call is made to summarize the conversation up to that point and use that as context for subsequent prompt Although people unfamiliar with "active listening" often find it annoying, this is something I attempt to implement within my conversations (quickly summarizing what I understand after somebody explains something technical). Oftentimes people feel like I'm `interrupting them`, but it is really just a way to stay engaged in two-way conversation. Eerie how LLM's benefit from this "focus" as similarly as this autistic commenter.