3 ms·
32k tokens is a novella to short novel length, so it can handle an extremely long conversation, but, sure, its possible they are using summarization and/or stor
by dragonwriter 4y ago
32k tokens is a novella to short novel length, so it can handle an extremely long conversation, but, sure, its possible they are using summarization and/or storage-and-retrieval hacks to extend beyond what fits in the context space naively.