3 ms·
From the linked docs: > If you want to disable thinking, you can set the reasoning effort to "none". For other APIs, you can set the thinking tokens to 0 and
by minimaxir 2y ago
From the linked docs:
> If you want to disable thinking, you can set the reasoning effort to "none".
For other APIs, you can set the thinking tokens to 0 and that also works.
- mgraczyk 2y agoWow thanks I did not know
- logankilpatrick 1y agoWe added it to the docs. The downside of the OAI compat endpoint is we have to design the API twice, once for our API, then once through the OAI compat layer which makes it slower sometimes to have certain features, especially if we diverge at all.
- mgraczyk 1y agoThanks, yes makes sense. BTW, I have noticed that when tested outside GCP, the OpenAI compat endpoint has significantly lower latency for most requests (vs using the genai library). VertexAI is better than both. Any idea why or if that will change?