3 ms·
Think of it as a very useful prefix match. If all of your threads start with the same system prompt, you will reap benefits from prompt caching.
by babelfish 10mo ago
Think of it as a very useful prefix match. If all of your threads start with the same system prompt, you will reap benefits from prompt caching.