2 ms·
> why not cache user prompt/response pairs because of context. You can cache it if it is indeed the first sentence of a fresh dialogue, but that's it.
by Tepix 4y ago
> why not cache user prompt/response pairs
because of context. You can cache it if it is indeed the first sentence of a fresh dialogue, but that's it.