4 ms·
No, why would they. You are supposed to maintain that cache. What I really want to know is about caching the large prefixes for prompts. Do they let you manage
by EGreg 1y ago
No, why would they. You are supposed to maintain that cache.
What I really want to know is about caching the large prefixes for prompts. Do they let you manage this somehow? What about llama and deepseek?