11 ms·

As an example, what I tried right now: USER I'm responding to you roughly every second with current time, say "Stop!", after 10 seconds have passed. Respond on

As an example, what I tried right now: USER I'm responding to you roughly every second with current time, say "Stop!", after 10 seconds have passed. Respond only with silent when you do not wish to say anything. Current Time: 13:04:11.21 Say "OK!" now if you have understood the challenge. ASSISTANT OK! USER Current Time: 13:04:12.24 ASSISTANT silent USER Current Time: 13:04:16.21 ASSISTANT silent USER Current Time: 13:04:19.18 ASSISTANT silent USER Current Time: 13:04:23.12 ASSISTANT Stop!

Retric 3y ago
That’s almost but not actually waiting. Picking the nest token for a context window is something GPT-4 is quite good at. But the underlying approach has some real limitations which I discussed in response to your other comment. Using 1 hour instead of 10 seconds and then list every Current Time incrementing by .01 seconds and you overflow the context window. The solution isn’t to have a larger context window, the solution is to have an internal representation independent of that context window. The issue is you can’t train a system to use that internal memory by simply giving it a large corpus of text examples. The text lacks the internal representation associated with its creation.
- mewpmewp2 3y ago
  In this case you wouldn't spam the context window, you would perhaps just send latest time every time and lose portion of the previous times.
  - Retric 3y ago
    That solves this specific issue, but it isn’t a general solution to the finite size of a context window. Ideally the system should be able to decide for itself what information it needs to pay attention to, but LLM’s simply don’t operate in that fashion.
    - mewpmewp2 3y ago
      But you can, e.g. you allow GPT to call functions to store embeddings that it can later retrieve for the memory. Meanwhile default context memory window is just recent info more detailed and historical info gets more and more summarized, possibly summarized by GPT itself.

Retric 3y ago

That’s almost but not actually waiting. Picking the nest token for a context window is something GPT-4 is quite good at. But the underlying approach has some real limitations which I discussed in response to your other comment. Using 1 hour instead of 10 seconds and then list every Current Time incrementing by .01 seconds and you overflow the context window. The solution isn’t to have a larger context window, the solution is to have an internal representation independent of that context window. The issue is you can’t train a system to use that internal memory by simply giving it a large corpus of text examples. The text lacks the internal representation associated with its creation.

mewpmewp2 3y ago
In this case you wouldn't spam the context window, you would perhaps just send latest time every time and lose portion of the previous times.
- Retric 3y ago
  That solves this specific issue, but it isn’t a general solution to the finite size of a context window. Ideally the system should be able to decide for itself what information it needs to pay attention to, but LLM’s simply don’t operate in that fashion.
  - mewpmewp2 3y ago
    But you can, e.g. you allow GPT to call functions to store embeddings that it can later retrieve for the memory. Meanwhile default context memory window is just recent info more detailed and historical info gets more and more summarized, possibly summarized by GPT itself.