4 ms·
Under 1,000 tokens for the system prompt and tool defs is the number that matters here. Everything else is downstream of not crowding the context.
by sage981 2mo ago
Under 1,000 tokens for the system prompt and tool defs is the number that matters here. Everything else is downstream of not crowding the context.