3 ms·
You can keep around the KV cache from previous generations which lowers the cost of prompts significantly.
by why_only_15 3y ago
You can keep around the KV cache from previous generations which lowers the cost of prompts significantly.
loading thread…