3 ms·
>tfw it's 2024 and people STILL aren't using span compression to implement a "medium term" memory (RAG is long term and the context length is short term) for LL
by Der_Einzige 3y ago
>tfw it's 2024 and people STILL aren't using span compression to implement a "medium term" memory (RAG is long term and the context length is short term) for LLMs.
>tfw it's 2024 and we just accept that the context "falls out" of the model if we push it beyond it's regular context length
So everyone forgot that we can put large N number of tokens into small N number of embeddings because???
- register 3y agoWhat do you mean by span compression? We have experimented with various embedding contes lengths and we have found that bigger embeddings aren't the ones providing the best recall. We have hit the best results with something between 65% to 75% of the maximum embedding context length. We have been using OpenAI embeddings models though.