3 ms·
This comment section is wild right now btw, no one is responding to the content, just the, i guess, misleading title. The OS in the title is making a compariso
by tayo42 3y ago
This comment section is wild right now btw, no one is responding to the content, just the, i guess, misleading title.
The OS in the title is making a comparison to how page caches work. Relevant data is moved into the context the llm needs to answer chat questions. When it runs out of memory it moves a condensed, searchable version of the content to another data store. Its trying to solve the problem of relatively low max token limits.
I wonder if this makes it easier to run on small devices?
- pacjam 3y agoThanks for checking out our work! Yeah, great point, this is even more critical in those scenarios when you have very limited memory. You can play around with different context sizes using the code on GitHub: https://github.com/cpacker/MemGPT/blob/main/memgpt/constants.py#L16 https://github.com/cpacker/MemGPT/blob/main/memgpt/constants...
- tayo42 3y agoThats cool, thanks for sharing the code!
- deleted 3y ago[deleted]