3 ms·
Yes, I think it might be a good solution where you have a context up to 10M of tokens and you do a lot of requests with that context. It might be relevant for a
by killerstorm 1y ago
Yes, I think it might be a good solution where you have a context up to 10M of tokens and you do a lot of requests with that context. It might be relevant for agentic stuff which tends to produce long chat logs - especially with some gadgets on top, e.g. some 'episodes' might be completely removed as obsolete.
But I don't think it's a good solution for bigger amounts of data - as in that case it's more beneficial if that can be formed into independent memories.