3 ms·
Why don't we have some equivalent of "fork" if we are talking the same context and tokens, you'd think that could all just be loaded into the gpu.
by stuaxo 3mo ago
Why don't we have some equivalent of "fork" if we are talking the same context and tokens, you'd think that could all just be loaded into the gpu.
- ricardobeat 3mo agoOpenCode, Crush and Pi do have the ability to fork a conversation. But cache reuse is up to the provider and not guaranteed. At some point you need to forward the cache to a more recent checkpoint, and you have a finite (unknown) number of parallel cached chats.