2 ms·
ChatGPT degrades precisely because they aren't doing anything special to extend their memory beyond the context length. There are trivial techniques to impleme
by Der_Einzige 3y ago
ChatGPT degrades precisely because they aren't doing anything special to extend their memory beyond the context length.
There are trivial techniques to implement "lossy" memory, such as just average pooling tokens (the same approach used by sentence transformers). Not sure why it's so rare to see this used for condensing a huge amount of context into a prompt. It is effectively "medium" term memory.
- visarga 3y agoAt the very least I would average vectors inside single words or word compounds getting a 2-3x reduction in length without much work.
- lgats 3y agohttps://chat.openai.com/share/e367a1de-c28b-4408-aa3d-2e4b85ea08b4 https://chat.openai.com/share/e367a1de-c28b-4408-aa3d-2e4b85... Fed chatGPT special numbers, then 3k tokens, then 2k tokens. after that, it was unable to understand any question about the special numbers provided.
- sharkjacobs 3y agoOn the other hand https://chat.openai.com/share/8a0675b6-2876-4606-ac79-646391dce370 https://chat.openai.com/share/8a0675b6-2876-4606-ac79-646391...