3 ms·
A gigabyte is a lot of memory. Even the largest context windows are a small fraction of that with any sane engineering discipline.
by overgard 5mo ago
A gigabyte is a lot of memory. Even the largest context windows are a small fraction of that with any sane engineering discipline.
- andriy_koval 5mo agoFor each LLM interaction they likely have bunch of thoughts traces, tool calls, etc, which don't go to context, but still can be retrieved. But I obviously don't know for sure.