3 ms·
I spotted something interesting in the Python API library code: https://github.com/googleapis/python-genai/blob/473bf4b6b5a69e5324a5d4bac0fe852351338c43/google
by simonw 2y ago
I spotted something interesting in the Python API library code:
https://github.com/googleapis/python-genai/blob/473bf4b6b5a69e5324a5d4bac0fe852351338c43/google/genai/types.py#L2070-L2082 https://github.com/googleapis/python-genai/blob/473bf4b6b5a6...
class ThinkingConfig(_common.BaseModel):
"""The thinking features configuration."""
include_thoughts: Optional[bool] = Field(
default=None,
description="""Indicates whether to include thoughts in the response. If true, thoughts are returned only if the model supports thought and thoughts are available.
""",
)
thinking_budget: Optional[int] = Field(
default=None,
description="""Indicates the thinking budget in tokens.
""",
)
That thinking_budget thing is documented, but what's the deal with include_thoughts? It sounds like it's an option to have the API return the thought summary... but I can't figure out how to get it to work, and I've not found documentation or example code that uses it.
Anyone managed to get Gemini to spit out thought summaries in its API using this option?
- phillypham 2y agoThey removed the docs and support for it https://github.com/googleapis/python-genai/commit/af3b339a9d58e25cb3baa6fd3d0b1d078c620f9d https://github.com/googleapis/python-genai/commit/af3b339a9d.... You can see the thoughts in AI Studio UI as per https://ai.google.dev/gemini-api/docs/thinking#debugging-and-steering https://ai.google.dev/gemini-api/docs/thinking#debugging-and....
- lemming 2y agoI maintain an alternative client which I build from the API definitions at https://github.com/googleapis/googleapis https://github.com/googleapis/googleapis, which according to https://github.com/googleapis/python-genai/issues/345 https://github.com/googleapis/python-genai/issues/345 should be the right place. But neither the AI Studio nor the Vertex definitions even have ThinkingConfig yet - very frustrating. In general it's amazing how much API munging is required to get a working client from the public API definitions.
- qwertox 2y agoIn AI Studio the flash moddels has two toggles: Enable thinking and Set thinking budget. If thinking budget is enabled, you can set tue max number of tokens it can use to think, else it's Auto.
- Deathmax 2y agoIt is gated behind the GOOGLE_INTERNAL visibility flag, which only internal Google projects and Cursor have at the moment as far as I know.
- msp26 2y agoThe API won't give you the "thinking" tokens, those are only visible on AI studio. Probably to try to stop distillation, very disappointing. I find reading the cot to be incredibly informative to identify failure modes. > Hey Everyone, > Moving forward, our team has made a decision to only show thoughts in Google AI Studio. Meaning, we no longer return thoughts via the Gemini API. Here is the updated doc to reflect that. https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not-included-anymore/63653/10 https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not... --- After I wrote all of that I see that the API docs page looks different today and now says: >Note that a summarized version of the thinking process is available through both the API and Google AI Studio. https://ai.google.dev/gemini-api/docs/thinking https://ai.google.dev/gemini-api/docs/thinking Maybe they just updated it? Or people aren't on the same page at Google idk Previously it said > Models with thinking capabilities are available in Google AI Studio and through the Gemini API. Note that the thinking process is visible within Google AI Studio but is not provided as part of the API output. https://web.archive.org/web/20250409174840/https://ai.google.dev/gemini-api/docs/thinking https://web.archive.org/web/20250409174840/https://ai.google...