3 ms·

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understan

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

JamesSwift 3mo ago
> all major vendors throw out the reasoning tokens between turns That would be surprising to me. The reasoning _is_ the model intelligence in a lot of respects, and so dropping those from the context would affect its output pretty significantly. I assume that instead they just have a lot of guardrails in place and multiple runtime environments that an individual turns ping-pong between in order to dehydrate/rehydrate the reasoning to keep it hidden from the end user.
- irthomasthomas 3mo ago
  Yep they store them encrypted https://blog.cryptographyengineering.com/2026/05/29/fooling-around-with-encrypted-reasoning-blobs/ https://blog.cryptographyengineering.com/2026/05/29/fooling-...
- Roritharr 3mo ago
  Anthropic very explicitly says below their diagrams ( https://platform.claude.com/docs/en/build-with-claude/context-windows https://platform.claude.com/docs/en/build-with-claude/contex... ) on this: "Stripping extended thinking: Extended thinking blocks (shown in dark gray) are generated during each turn's output phase, but are not carried forward as input tokens for subsequent turns. You do not need to strip the thinking blocks yourself. The Claude API automatically does this for you if you pass them back." It's more nuanced in the various modes, but i haven't seen it boil down towards Thinking Tokens surviving more than two turns.
  - JamesSwift 3mo ago
    Thats really surprising, I stand corrected. I have had a lot of issues with hallucinations I attributed to adaptive thinking, but I wonder if those were actually due to this behavior instead. I also wonder if they actually do a hybrid of "standard reasoning" and then classify this stripped chain of thought as "extended thinking".
  - haus20xx 3mo ago
    https://platform.claude.com/docs/en/build-with-claude/extended-thinking#thinking-block-preservation-by-model https://platform.claude.com/docs/en/build-with-claude/extend... default depends on the model class. Opus: Claude Opus 4.5 and later Opus models keep all prior thinking blocks; Claude Opus 4.1 (deprecated) and earlier Opus models keep only the last assistant turn's thinking. Sonnet: Claude Sonnet 4.6 and later Sonnet models keep all; Claude Sonnet 4.5 and earlier Sonnet models keep only the last turn. Haiku: all Haiku models through Claude Haiku 4.5 keep only the last turn. Claude Mythos Preview also keeps all prior thinking blocks.
    - JamesSwift 3mo ago
      Now Im even more confused : D That would also explain the issue I mention in my other comment. And would also reinforce how much output would degrade without this. Opus 4.5 was a step above previous models in my experience. At some point it degraded and only got better when I disabled adaptive thinking. Adaptive thinking is always on for 4.6 and above.
vesterde 3mo ago
Gemini models return a thinking signature that you, I think, must send back when invoking further, so they seem to keep them?
btown 3mo ago
For Claude, at least, "throw out the reasoning tokens" is only true when a session has been idle for more than an hour, and is new since March. The basic concept is that for a session active recently, interleaved thinking tokens are already in KV cache, so it's more efficient to keep using them than not! But when resuming an older session where KV cache has been evicted, it's more expensive to restore the thinking tokens, so they're silently dropped from prior turns. It's 2026 and stateful servers are back on the menu! https://www.anthropic.com/engineering/april-23-postmortem https://www.anthropic.com/engineering/april-23-postmortem describes this as an intended optimization: > The design should have been simple: if a session has been idle for more than an hour, we could reduce users’ cost of resuming that session by clearing old thinking sections. Since the request would be a cache miss anyway, we could prune unnecessary messages from the request to reduce the number of uncached tokens sent to the API. We’d then resume sending full reasoning history. To do this we used the clear_thinking_20251015 API header along with keep:1. > The implementation had a bug. Instead of clearing thinking history once, it cleared it on every turn for the rest of the session... This surfaced as the forgetfulness, repetition, and odd tool choices people reported. And https://news.ycombinator.com/item?id=47879561 https://news.ycombinator.com/item?id=47879561 is a thread with a Claude team member's further rationale. > Eliding parts of the context after idle: old tool results, old messages, thinking. Of these, thinking performed the best, and when we shipped it, that's when we unintentionally introduced the bug in the blog post. (Also, https://news.ycombinator.com/item?id=47884517 https://news.ycombinator.com/item?id=47884517 indicates OpenAI drops reasoning tokens "smartly" at its own election, which is likely a similar performance optimization.) I've experimented with rules to have Claude Code be explicit about recapping its thinking tokens, including tool choices and approaches chosen and rejected, into actual message output, but this is lossy at best. And sometimes dropping reasoning tokens can give a session "fresh eyes" in a good way. I just really don't like the lack of control, and it's a reminder of how ephemeral the current landscape is. The Claude giveth, and the Claude taketh away.
- Roritharr 3mo ago
  Thank you! This is much more nuanced than my understanding so far!
- chacham15 3mo ago
tough 3mo ago
OAI is now implementing encrypted CoT that you can store and pass back between turns (harness call), so new models have it https://developers.openai.com/api/docs/guides/reasoning#encrypted-reasoning-items https://developers.openai.com/api/docs/guides/reasoning#encr...
- sigmoid10 3mo ago
  You could also use the responses api which stores all message contents (including reasoning) on OAI servers. This has been possible for quite a while now. Encryption is only necessary if you really care about local storage (which is different from privacy concerns, because the data gets sent to their servers anyway).
  - tough 3mo ago
    well the encryption part is also mostly about OAI wanting to avoid others to distill from their COT/reasoning traces, since this is not ever displayed to devs or final users, and as you say lives on their servers. but yes you're correct on the responses api already baking it in too supposedly keeping these between tool calls should help the model reason and have better overall outputs etc

JamesSwift 3mo ago

> all major vendors throw out the reasoning tokens between turns That would be surprising to me. The reasoning _is_ the model intelligence in a lot of respects, and so dropping those from the context would affect its output pretty significantly. I assume that instead they just have a lot of guardrails in place and multiple runtime environments that an individual turns ping-pong between in order to dehydrate/rehydrate the reasoning to keep it hidden from the end user.

irthomasthomas 3mo ago
Yep they store them encrypted https://blog.cryptographyengineering.com/2026/05/29/fooling-around-with-encrypted-reasoning-blobs/ https://blog.cryptographyengineering.com/2026/05/29/fooling-...
Roritharr 3mo ago
Anthropic very explicitly says below their diagrams ( https://platform.claude.com/docs/en/build-with-claude/context-windows https://platform.claude.com/docs/en/build-with-claude/contex... ) on this: "Stripping extended thinking: Extended thinking blocks (shown in dark gray) are generated during each turn's output phase, but are not carried forward as input tokens for subsequent turns. You do not need to strip the thinking blocks yourself. The Claude API automatically does this for you if you pass them back." It's more nuanced in the various modes, but i haven't seen it boil down towards Thinking Tokens surviving more than two turns.
- JamesSwift 3mo ago
  Thats really surprising, I stand corrected. I have had a lot of issues with hallucinations I attributed to adaptive thinking, but I wonder if those were actually due to this behavior instead. I also wonder if they actually do a hybrid of "standard reasoning" and then classify this stripped chain of thought as "extended thinking".
- haus20xx 3mo ago
  https://platform.claude.com/docs/en/build-with-claude/extended-thinking#thinking-block-preservation-by-model https://platform.claude.com/docs/en/build-with-claude/extend... default depends on the model class. Opus: Claude Opus 4.5 and later Opus models keep all prior thinking blocks; Claude Opus 4.1 (deprecated) and earlier Opus models keep only the last assistant turn's thinking. Sonnet: Claude Sonnet 4.6 and later Sonnet models keep all; Claude Sonnet 4.5 and earlier Sonnet models keep only the last turn. Haiku: all Haiku models through Claude Haiku 4.5 keep only the last turn. Claude Mythos Preview also keeps all prior thinking blocks.
  - JamesSwift 3mo ago
    Now Im even more confused : D That would also explain the issue I mention in my other comment. And would also reinforce how much output would degrade without this. Opus 4.5 was a step above previous models in my experience. At some point it degraded and only got better when I disabled adaptive thinking. Adaptive thinking is always on for 4.6 and above.

vesterde 3mo ago

Gemini models return a thinking signature that you, I think, must send back when invoking further, so they seem to keep them?

btown 3mo ago

For Claude, at least, "throw out the reasoning tokens" is only true when a session has been idle for more than an hour, and is new since March. The basic concept is that for a session active recently, interleaved thinking tokens are already in KV cache, so it's more efficient to keep using them than not! But when resuming an older session where KV cache has been evicted, it's more expensive to restore the thinking tokens, so they're silently dropped from prior turns. It's 2026 and stateful servers are back on the menu! https://www.anthropic.com/engineering/april-23-postmortem https://www.anthropic.com/engineering/april-23-postmortem describes this as an intended optimization: > The design should have been simple: if a session has been idle for more than an hour, we could reduce users’ cost of resuming that session by clearing old thinking sections. Since the request would be a cache miss anyway, we could prune unnecessary messages from the request to reduce the number of uncached tokens sent to the API. We’d then resume sending full reasoning history. To do this we used the clear_thinking_20251015 API header along with keep:1. > The implementation had a bug. Instead of clearing thinking history once, it cleared it on every turn for the rest of the session... This surfaced as the forgetfulness, repetition, and odd tool choices people reported. And https://news.ycombinator.com/item?id=47879561 https://news.ycombinator.com/item?id=47879561 is a thread with a Claude team member's further rationale. > Eliding parts of the context after idle: old tool results, old messages, thinking. Of these, thinking performed the best, and when we shipped it, that's when we unintentionally introduced the bug in the blog post. (Also, https://news.ycombinator.com/item?id=47884517 https://news.ycombinator.com/item?id=47884517 indicates OpenAI drops reasoning tokens "smartly" at its own election, which is likely a similar performance optimization.) I've experimented with rules to have Claude Code be explicit about recapping its thinking tokens, including tool choices and approaches chosen and rejected, into actual message output, but this is lossy at best. And sometimes dropping reasoning tokens can give a session "fresh eyes" in a good way. I just really don't like the lack of control, and it's a reminder of how ephemeral the current landscape is. The Claude giveth, and the Claude taketh away.

Roritharr 3mo ago
Thank you! This is much more nuanced than my understanding so far!
chacham15 3mo ago

tough 3mo ago

OAI is now implementing encrypted CoT that you can store and pass back between turns (harness call), so new models have it https://developers.openai.com/api/docs/guides/reasoning#encrypted-reasoning-items https://developers.openai.com/api/docs/guides/reasoning#encr...

sigmoid10 3mo ago
You could also use the responses api which stores all message contents (including reasoning) on OAI servers. This has been possible for quite a while now. Encryption is only necessary if you really care about local storage (which is different from privacy concerns, because the data gets sent to their servers anyway).
- tough 3mo ago
  well the encryption part is also mostly about OAI wanting to avoid others to distill from their COT/reasoning traces, since this is not ever displayed to devs or final users, and as you say lives on their servers. but yes you're correct on the responses api already baking it in too supposedly keeping these between tool calls should help the model reason and have better overall outputs etc