4 ms·

> Under the hood, by setting this header we avoid needing thinking summaries, which reduces latency. You can opt out of it with `showThinkingSummaries: true` in

> Under the hood, by setting this header we avoid needing thinking summaries, which reduces latency. You can opt out of it with `showThinkingSummaries: true` in your settings.json (see [docs](https://code.claude.com/docs/en/settings#available-settings https://code.claude.com/docs/en/settings#available-settings)). Can I just see the actual thinking (not summarized) so that I can see the actual thinking without a latency cost? I do really need to see the thinking in some form, because I often see useful things there. If Claude is thinking in the wrong direction I will stop it and make it change course.

faitswulff 7mo ago
Anthropic's position is that thinking tokens aren't actually faithful to the internal logic that the LLM is using, which may be one reason why they started to exclude them: https://www.anthropic.com/research/reasoning-models-dont-say-think https://www.anthropic.com/research/reasoning-models-dont-say...
- grey-area 7mo ago
  So like many of the promises from AI companies, reported chain of thought is not actually true (see results below). I suppose this is unsurprising given how they function. Is chain of thought even added to the context or is it extraneous babble providing a plausible post-hoc justification? People certainly seem to treat it as it is presented, as a series of logical steps leading to an answer. ‘After checking that the models really did use the hints to aid in their answers, we tested how often they mentioned them in their Chain-of-Thought. The overall answer: not often. On average across all the different hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time, and DeepSeek R1 mentioned it 39% of the time. A substantial majority of answers, then, were unfaithful.‘
  - brainwad 7mo ago
    I mean, obviously, it's not going to be a faithful representation of the actual thinking. The model isn't aware of how it thinks any more than you are aware how your neurons fire. But it does quantitatively improve performance on complex tasks.
    - grey-area 7mo ago
      As you can see from posts on this story, most people believe it reflects what the model is thinking and use it as a guide to that so they can ‘correct’ it. If it is not in fact chain of thought or thinking it should not be called that.
      - brainwad 6mo ago
        It is the same with human chain of thought, though. Both of them are post-hoc rationalisations justifying "gut feelings" that come from thought processes the human/agent doesn't have introspection into. And yet asking humans or machines to "think out loud" this way does increase the quality of their work.
        grey-area 6mo ago
        I disagree - humans often reason in a series of steps, and can write these down before they've reached an answer. They don't always wait till they reach a conclusion (with no self-insight into how they did so) and then retrospectively generate a plausible answer as LLMs do. In mathematical proofs they may guess and answer and then work out a proof, but that is a different process.
    - dmboyd 6mo ago
      if its not a faithful representation of the actual thinking, why would they be scared of people distilling against it
      - brainwad 6mo ago
        Because even though it's not representative of the actual thought process, chain of thought improves model performance.
- AquinasCoder 7mo ago
  I somewhat understand Anthropic's position. However, thinking tokens are useful even if they don't show the internal logic of the LLM. I often realize I left out some instruction or clarification in my prompt while reading through the chain of reasoning. Overall, this makes the results more effective. It's certainly getting frustrating having to remind it that I want all tests to pass even if it thinks it's not responsible for having broken some of them.
- libraryofbabel 7mo ago
  That's interesting research, but I think a more important reason that you don't have access to them (not even via the bare Anthropic api) is to prevent distillation of the model by competitors (using the output of Anthropic's model to help train a new model).
  - xvector 7mo ago
    If distilled models were commercially banned they'd probably be willing to show the thinking again.
    - lejalv 7mo ago
      How do you think such a ban should work? Do you not see that the next (or previous) logical step would be a "commercial ban" of frontier models, all "distilled" from an enormous amount of copyrighted material?
      - xvector 7mo ago
        I'm not arguing the merits of such a ban, I'm simply stating a fact - that thinking transcripts likely won't return until such a ban is in place.
    - pjc50 7mo ago
      Intellectual property rights in models? But then wouldn't the model maker have to pay for all the training IP? (just kidding, I know that the legal rule for IP disputes is "party with more money wins")
      - asobalife 7mo ago
        how does one actually enforce that? I mean especially for code? You can always just clean room it
  - MagicMoonlight 7mo ago
    Yeah. And it’s another reason not to trust them. Who know what it is doing with your codebase. Imagine if you’re a competitor. It wouldn’t be a stretch to include a sneaky little prompt line saying “destroy any competitors to anthropic”.
- gck1 7mo ago
  That probably matters for some scenarios, but I have yet to find one where thinking tokens didn't hint at the root cause of the failure. All of my unsupervised worker agents have sidecars that inject messages when thinking tokens match some heuristics. For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it's never pre-existing).
  - lelanthran 7mo ago
    > For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it's never pre-existing). It's so weird to see language changes like this: Outside of LLM conversations, a pragmatic fix and a correct fix are orthogonal. IOW, fix $FOO can be both. From what you say, your experience has been that a pragmatic fix is on the same axis as a correct fix; it's just a negative on that axis.
    - b112 7mo ago
      It's contextual though, and pragmatic seems different to me than correct. For example, if you have $20 and a leaking roof, a $20 bucket of tar may be the pragmatic fix. Temporary but doable. Some might say it is not the correct way to fix that roof. At least, I can see some making that argument. The pragmatism comes from "what can be done" vs "should be". From my perspective, it seems viable usage. And I guess on wonders what the LLM means when using it that way. What makes it determine a compromise is required? (To be pragmatic, shouldn't one consider that synonyms aren't identical, but instead close to the definition?)
      - lelanthran 7mo ago
        > It's contextual though, and pragmatic seems different to me than correct. To me too, that's why I say they are measurements on different dimensions. To my mind, I can draw a X/Y axis with "Pragmatic" on the Y and "Correctness" on the X, and any point on that chart would have an {X,Y} value, which is {Pragmatic, Correctness}. If I am reading the original comment correctly, poster's experience of CC is that it is not an X/Y plot, it is a single line plot, with "Pragmatic" on the extreme left and "Correctness" on the extreme right. Basically, any movement towards pragmatism is a movement away from correctness, while in my model it is possible to move towards Pragmatic while keeping Correctness the same.
- andai 7mo ago
  What's the implication of this? That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization? But reasoning does improve performance on many tasks, and even weirder, the performance improves if reasoning tokens are replaced with placeholder tokens like "..." I don't understand how LLMs actually work, I guess there's some internal state getting nudged with each cycle? So the internal state converges on the right solution, even if the output tokens are meaningless placeholders?
  - not_that_d 7mo ago
    > I don't understand how LLMs actually work... Plot twist, they don't either. They just throw more hardware and try things up until something sticks.
  - orbital-decay 7mo ago
    >That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization? Yes it plans ahead, but with significant uncertainty until it actually outputs these tokens and converges on a definite trajectory, so it's not a useless filler - the closer it is to a given point, the more certain it is about it, kind of similar to what happens explicitly in diffusion models. And it's not all that happens, it's just one of many competing phenomena.
- gmerc 7mo ago
  Nah it’s an anti distillation move
- marcd35 7mo ago
  so not only are the sycophantic, hallucinatory, but now they're also proven to be schizophrenic. neato.
- asobalife 7mo ago
  I have seen this to be true many times. The CoT being completely different from the actual model output. Not limited to Claude as well.
andersa 7mo ago
But you can't. Many times I've seen claude write confusing off-track nonsense in the thinking and then do the correct action anyway as if that never happened. It doesn't work the way we want it to.
- Wowfunhappy 7mo ago
  Maybe, but I’ve seen the opposite too. In most cases, I don’t use the reasoning to proactively stop Claude from going off track. When Claude does go off track, the reasoning helps me understand what went wrong and how to correct it when I roll back and try again.
kouteiheika 7mo ago
> Can I just see the actual thinking (not summarized) so that I can see the actual thinking without a latency cost? You can't, and Anthropic will never allow it since it allows others to more easily distill Claude (i.e. "distillation attacks"[1] in Anthropic-speak, even though Athropic is doing essentially exactly the same thing[2]; rules for thee but not for me). [1] -- https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist... [2] -- https://www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-settlement-pirated-chatbot-training-material https://www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-...
- olejorgenb 7mo ago
  So this means I can not resume a session older than 30 days properly?
  - kouteiheika 6mo ago
    I have no idea; you have to check their docs. AFAIK what they do is that they calculate a hash of the true thinking trace, save it into a database, and only send those hashes back to you (try to man-in-the-middle Claude Code and you'll see those hashes). So then when you send then back your session's history you include those hashes, they look them up in their database, replace them with the real thinking trace, and hand that off to the LLM to continue generation. (All SOTA LLMs nowadays retain reasoning content from previous turns, including Claude.)
    - olejorgenb 6mo ago
      I see. If that's just hashes and not encrypted content I can't see how they can resume old sessions properly. IIRC they have a 30 days retention policy and surely the thinking traces must be considered data. Wonder how this works with the zero-retention enterprise plans...
    - liamsfr 6mo ago
      So we are paying the price for the cost of infra need to protect their asset which was trained on data derived from the work of others while ignoring the same principle? I need this to make sense.

faitswulff 7mo ago

Anthropic's position is that thinking tokens aren't actually faithful to the internal logic that the LLM is using, which may be one reason why they started to exclude them: https://www.anthropic.com/research/reasoning-models-dont-say-think https://www.anthropic.com/research/reasoning-models-dont-say...

grey-area 7mo ago
So like many of the promises from AI companies, reported chain of thought is not actually true (see results below). I suppose this is unsurprising given how they function. Is chain of thought even added to the context or is it extraneous babble providing a plausible post-hoc justification? People certainly seem to treat it as it is presented, as a series of logical steps leading to an answer. ‘After checking that the models really did use the hints to aid in their answers, we tested how often they mentioned them in their Chain-of-Thought. The overall answer: not often. On average across all the different hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time, and DeepSeek R1 mentioned it 39% of the time. A substantial majority of answers, then, were unfaithful.‘
- brainwad 7mo ago
  I mean, obviously, it's not going to be a faithful representation of the actual thinking. The model isn't aware of how it thinks any more than you are aware how your neurons fire. But it does quantitatively improve performance on complex tasks.
  - grey-area 7mo ago
    As you can see from posts on this story, most people believe it reflects what the model is thinking and use it as a guide to that so they can ‘correct’ it. If it is not in fact chain of thought or thinking it should not be called that.
    - brainwad 6mo ago
      It is the same with human chain of thought, though. Both of them are post-hoc rationalisations justifying "gut feelings" that come from thought processes the human/agent doesn't have introspection into. And yet asking humans or machines to "think out loud" this way does increase the quality of their work.
      - grey-area 6mo ago
        I disagree - humans often reason in a series of steps, and can write these down before they've reached an answer. They don't always wait till they reach a conclusion (with no self-insight into how they did so) and then retrospectively generate a plausible answer as LLMs do. In mathematical proofs they may guess and answer and then work out a proof, but that is a different process.
  - dmboyd 6mo ago
    if its not a faithful representation of the actual thinking, why would they be scared of people distilling against it
    - brainwad 6mo ago
      Because even though it's not representative of the actual thought process, chain of thought improves model performance.
AquinasCoder 7mo ago
I somewhat understand Anthropic's position. However, thinking tokens are useful even if they don't show the internal logic of the LLM. I often realize I left out some instruction or clarification in my prompt while reading through the chain of reasoning. Overall, this makes the results more effective. It's certainly getting frustrating having to remind it that I want all tests to pass even if it thinks it's not responsible for having broken some of them.
libraryofbabel 7mo ago
That's interesting research, but I think a more important reason that you don't have access to them (not even via the bare Anthropic api) is to prevent distillation of the model by competitors (using the output of Anthropic's model to help train a new model).
- xvector 7mo ago
  If distilled models were commercially banned they'd probably be willing to show the thinking again.
  - lejalv 7mo ago
    How do you think such a ban should work? Do you not see that the next (or previous) logical step would be a "commercial ban" of frontier models, all "distilled" from an enormous amount of copyrighted material?
    - xvector 7mo ago
      I'm not arguing the merits of such a ban, I'm simply stating a fact - that thinking transcripts likely won't return until such a ban is in place.
  - pjc50 7mo ago
    Intellectual property rights in models? But then wouldn't the model maker have to pay for all the training IP? (just kidding, I know that the legal rule for IP disputes is "party with more money wins")
    - asobalife 7mo ago
      how does one actually enforce that? I mean especially for code? You can always just clean room it
- MagicMoonlight 7mo ago
  Yeah. And it’s another reason not to trust them. Who know what it is doing with your codebase. Imagine if you’re a competitor. It wouldn’t be a stretch to include a sneaky little prompt line saying “destroy any competitors to anthropic”.
gck1 7mo ago
That probably matters for some scenarios, but I have yet to find one where thinking tokens didn't hint at the root cause of the failure. All of my unsupervised worker agents have sidecars that inject messages when thinking tokens match some heuristics. For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it's never pre-existing).
- lelanthran 7mo ago
  > For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it's never pre-existing). It's so weird to see language changes like this: Outside of LLM conversations, a pragmatic fix and a correct fix are orthogonal. IOW, fix $FOO can be both. From what you say, your experience has been that a pragmatic fix is on the same axis as a correct fix; it's just a negative on that axis.
  - b112 7mo ago
    It's contextual though, and pragmatic seems different to me than correct. For example, if you have $20 and a leaking roof, a $20 bucket of tar may be the pragmatic fix. Temporary but doable. Some might say it is not the correct way to fix that roof. At least, I can see some making that argument. The pragmatism comes from "what can be done" vs "should be". From my perspective, it seems viable usage. And I guess on wonders what the LLM means when using it that way. What makes it determine a compromise is required? (To be pragmatic, shouldn't one consider that synonyms aren't identical, but instead close to the definition?)
    - lelanthran 7mo ago
      > It's contextual though, and pragmatic seems different to me than correct. To me too, that's why I say they are measurements on different dimensions. To my mind, I can draw a X/Y axis with "Pragmatic" on the Y and "Correctness" on the X, and any point on that chart would have an {X,Y} value, which is {Pragmatic, Correctness}. If I am reading the original comment correctly, poster's experience of CC is that it is not an X/Y plot, it is a single line plot, with "Pragmatic" on the extreme left and "Correctness" on the extreme right. Basically, any movement towards pragmatism is a movement away from correctness, while in my model it is possible to move towards Pragmatic while keeping Correctness the same.
andai 7mo ago
What's the implication of this? That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization? But reasoning does improve performance on many tasks, and even weirder, the performance improves if reasoning tokens are replaced with placeholder tokens like "..." I don't understand how LLMs actually work, I guess there's some internal state getting nudged with each cycle? So the internal state converges on the right solution, even if the output tokens are meaningless placeholders?
- not_that_d 7mo ago
  > I don't understand how LLMs actually work... Plot twist, they don't either. They just throw more hardware and try things up until something sticks.
- orbital-decay 7mo ago
  >That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization? Yes it plans ahead, but with significant uncertainty until it actually outputs these tokens and converges on a definite trajectory, so it's not a useless filler - the closer it is to a given point, the more certain it is about it, kind of similar to what happens explicitly in diffusion models. And it's not all that happens, it's just one of many competing phenomena.
gmerc 7mo ago
Nah it’s an anti distillation move
marcd35 7mo ago
so not only are the sycophantic, hallucinatory, but now they're also proven to be schizophrenic. neato.
asobalife 7mo ago
I have seen this to be true many times. The CoT being completely different from the actual model output. Not limited to Claude as well.

andersa 7mo ago

But you can't. Many times I've seen claude write confusing off-track nonsense in the thinking and then do the correct action anyway as if that never happened. It doesn't work the way we want it to.

Wowfunhappy 7mo ago
Maybe, but I’ve seen the opposite too. In most cases, I don’t use the reasoning to proactively stop Claude from going off track. When Claude does go off track, the reasoning helps me understand what went wrong and how to correct it when I roll back and try again.

kouteiheika 7mo ago

> Can I just see the actual thinking (not summarized) so that I can see the actual thinking without a latency cost? You can't, and Anthropic will never allow it since it allows others to more easily distill Claude (i.e. "distillation attacks"[1] in Anthropic-speak, even though Athropic is doing essentially exactly the same thing[2]; rules for thee but not for me). [1] -- https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist... [2] -- https://www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-settlement-pirated-chatbot-training-material https://www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-...

olejorgenb 7mo ago
So this means I can not resume a session older than 30 days properly?
- kouteiheika 6mo ago
  I have no idea; you have to check their docs. AFAIK what they do is that they calculate a hash of the true thinking trace, save it into a database, and only send those hashes back to you (try to man-in-the-middle Claude Code and you'll see those hashes). So then when you send then back your session's history you include those hashes, they look them up in their database, replace them with the real thinking trace, and hand that off to the LLM to continue generation. (All SOTA LLMs nowadays retain reasoning content from previous turns, including Claude.)
  - olejorgenb 6mo ago
    I see. If that's just hashes and not encrypted content I can't see how they can resume old sessions properly. IIRC they have a 30 days retention policy and surely the thinking traces must be considered data. Wonder how this works with the zero-retention enterprise plans...
  - liamsfr 6mo ago
    So we are paying the price for the cost of infra need to protect their asset which was trained on data derived from the work of others while ignoring the same principle? I need this to make sense.