4 ms·

Gemini was good when the thinking tokens were shown to the user. As soon as Google replaced those with some thought summary, I stopped finding it as useful. Pre

Gemini was good when the thinking tokens were shown to the user. As soon as Google replaced those with some thought summary, I stopped finding it as useful. Previously, the thoughts were so organized that I would often read those instead of the final answer.

dwringer 1y ago
These were extremely helpful to read for insights on how to go back and retry different prompts instead, IMHO. I find it to be a significant step back in usability to lose those although I can understand the argument that they weren't directly useful on their own outside of that use case.
kridsdale3 1y ago
In the API, the thinking tokens are just a different stream. You can still read them.
- deaux 1y ago
  They aren't, in the API too they're a thought summary, not nearly as useful as actually showing the thoughts. Anthropic still does provide raw thoughts to this day, showing how it's not necessary to keep a moat. Google and OpenAI don't. [0] https://ai.google.dev/gemini-api/docs/thinking#summaries https://ai.google.dev/gemini-api/docs/thinking#summaries [1] https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not-included-anymore/63653/10 https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not... [2] https://discuss.ai.google.dev/t/massive-regression-detailed-gemini-thinking-process-vanished-from-ai-studio/83916 https://discuss.ai.google.dev/t/massive-regression-detailed-... [2]
  - behnamoh 1y ago
    to be fair, Anthropic reasoning models don't have long thinking to begin with, and I find their reasoning pretty useless compared to what Gemini used to do.
    - deaux 1y ago
      Long thinking seems to be a marketing term without clear definition, only applicable to the opaque chat frontends. If you give Anthropic models a hard problem and set the thinking budget high (API), it does plenty of reasoning and the CoT helps a lot with debugging. With Gemini and OpenAI you can't debug as the summaries tell you effectively nothing about why it's giving a wrong answer or going off the rails when it does som
      - behnamoh 1y ago
        yes but I was referring to the Claude app which only shows a few sentences of thinking.

dwringer 1y ago

These were extremely helpful to read for insights on how to go back and retry different prompts instead, IMHO. I find it to be a significant step back in usability to lose those although I can understand the argument that they weren't directly useful on their own outside of that use case.

kridsdale3 1y ago

In the API, the thinking tokens are just a different stream. You can still read them.

deaux 1y ago
They aren't, in the API too they're a thought summary, not nearly as useful as actually showing the thoughts. Anthropic still does provide raw thoughts to this day, showing how it's not necessary to keep a moat. Google and OpenAI don't. [0] https://ai.google.dev/gemini-api/docs/thinking#summaries https://ai.google.dev/gemini-api/docs/thinking#summaries [1] https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not-included-anymore/63653/10 https://discuss.ai.google.dev/t/thoughts-are-missing-cot-not... [2] https://discuss.ai.google.dev/t/massive-regression-detailed-gemini-thinking-process-vanished-from-ai-studio/83916 https://discuss.ai.google.dev/t/massive-regression-detailed-... [2]
- behnamoh 1y ago
  to be fair, Anthropic reasoning models don't have long thinking to begin with, and I find their reasoning pretty useless compared to what Gemini used to do.
  - deaux 1y ago
    Long thinking seems to be a marketing term without clear definition, only applicable to the opaque chat frontends. If you give Anthropic models a hard problem and set the thinking budget high (API), it does plenty of reasoning and the CoT helps a lot with debugging. With Gemini and OpenAI you can't debug as the summaries tell you effectively nothing about why it's giving a wrong answer or going off the rails when it does som
    - behnamoh 1y ago
      yes but I was referring to the Claude app which only shows a few sentences of thinking.