3 ms·

This article was more true than not a year ago but now the harnesses are so far past the simple agent loop that I'd argue that this is not even close to an accu

This article was more true than not a year ago but now the harnesses are so far past the simple agent loop that I'd argue that this is not even close to an accurate mental model of what claude code is doing.

splike 9mo ago
I'm interested, could you expand on that?
- prodigycorp 9mo ago
  Off the top of my head: parallel subagents, hooks, skills, and a much better plan mode. These features enable way better steering than we had last year. Subagents are a huge boon to productivity.
  - rtgfhyuj 9mo ago
    are subagents just tools that are agents themselves?
    - dkdcio 9mo ago
      pretty much…they have their own system prompts, you can customize the model, the tools they use, etc. CC has built in subagents (including at least one not listed) that work very well: https://code.claude.com/docs/en/sub-agents#built-in-subagents https://code.claude.com/docs/en/sub-agents#built-in-subagent... this was not the case in the past, I swore off subagents, but they got good at some point
dkdcio 9mo ago
it seems to have changed a ton in recent versions too — I would love more details on what exactly I find it doing what I in the past had to interrupt and tell it to do fairly frequently now
- terminalshort 9mo ago
  For one thing it seems to splitting up the work and making some determination of complexity, then allocating it out to a model based on that complexity to save resources. When I run Claude with Opus 4.5 and run /cost I see tokens for Opus 4.5, but also a lot in Sonnet and Haiku, with the majority of tokens actually being used by Haiku.
  - nyellin 9mo ago
    Haiku is called often, but not always the way you think. E.g. every time you write something CC invokes Haiku multiple times to generate the 'delightful 1-2 word phrase used to indicate progress to the user' (Doing Stuff, Wizarding, etc)
    - dkdcio 9mo ago
      it’s also used in the Explore agent and for other things too
lukan 9mo ago
The article was also published one year ago on january 2025. (Should have 2025 in the title? Time flies)
- llmslave2 9mo ago
  Claude Code didn't exist in January 2025. I think it's a typo and should be 2026.
  - prodigycorp 9mo ago
    You’re right. No wonder the date felt odd. iirc Claude code was released around march.
    - dkdcio 9mo ago
      late Feb. 2025: https://www.anthropic.com/news/claude-3-7-sonnet https://www.anthropic.com/news/claude-3-7-sonnet
alright2565 9mo ago
But does that extra complexity actually improve performance? https://www.tbench.ai/leaderboard/terminal-bench/2.0 https://www.tbench.ai/leaderboard/terminal-bench/2.0 says yes, but not as much as you'd think. "Terminus" is basically just a tmux session and LLM in a loop.
- prodigycorp 9mo ago
  I'm not a good representative for claude code because I'm primarily a codex user now, but I know that if codex had subagents it would be at least twice as productive. Time spent is an important aspect of performance so yup, the complexity improved performance.
  - nyellin 9mo ago
    Not necessarily true. Subagents allow for parallelization but they can decrease accuracy dramatically if you're not careful because there are often dependencies between tasks and swapping context windows with a summary is extremely lossy. For the longest time, Claude Code itself didnt really use subagents much by default, other than supporting them as a feature eager users could configure. (Source is reverse engineering we did on Claude code using the fantastic CC tracing tool Simon Willison wrote about once. This is also no longer true on latest versions that have e.g. an Explore subagent that is actively used.)
    - prodigycorp 9mo ago
      You’re right that subagents were more likely to cause issues than be helpful. But, when properly understood, lead to so much time saved through parallelization for tasks that warranted it. I was having codex organize my tv/movie library the other day by having it generate. most of the files were not properly labeled. I had codex generate transcripts, manually search the movie db to find descriptions of show episodes, and match the show descriptions against the transcripts to figure out which episode/season the show belonged to. Claude Code could have parallelized those manual checks and finished that task at 8x the speed.
  - 9mo ago
qsort 9mo ago
Obviously modern harnesses have better features but I wouldn't say it invalidates the mental model. Simpler agents aren't that far behind in performance if the underlying model is the same, including very minimal ones with basic tools. I'd say it's similar to how a "make your own relational DB" article might feature a basic B-tree with merge-joins. Yeah, obviously real engines have sophisticated planners, multiple join methods, bloom filters, etc., but the underlying mental model is still accurate.
- prodigycorp 9mo ago
  You’re not wrong but I still think that the harness matters a lot when trying to accurately describe Claude Code. Here’s a reframing: If you asked people “what would you rather work with, today’s Claude Code harness with sonnet 3.7, or the 200 line agentic loop in the article with Opus 4.5, which would you choose?” I suspect many people would choose 3.7 with the harness. Moreover, that is true, then I’d say the article is no longer useful for a modern understanding of Claude Code.
  - rfw300 9mo ago
    Any person who would choose 3.7 with a fancy harness has a very poor memory about how dramatically the model capabilities have improved between then and now.
    - prodigycorp 9mo ago
      I’d be very interested in the performance of 3.7 decked out with web search, context7, a full suite of skills, and code quality hooks against opus 4.5 with none of those. I suspect it’s closer than you think!
      - CuriouslyC 9mo ago
        Skills don't make any difference above having markdown files to point an agent to with instructions as needed. Context7 isn't any better than telling your agent to use trafilatura to scrape web docs for your libs, and having a linting/static analysis suite isn't a harness thing. 3.7 was kinda dumb, it was good at vibe UIs but really bad at a lot of things and it would lie and hack rewards a LOT. The difference with Opus 4.5 is that when you go off the Claude happy path, it holds together pretty well. With Sonnet (particularly <=4) if you went off the happy path things got bad in a hurry.
CuriouslyC 9mo ago
Less true than you think. A lot of the progress in the last year has been tightening agentic prompts/tools and getting out of the way so the model can flex. Subagents/MCP/Skills are all pretty mid, and while there has been some context pruning optimization to avoid carrying tool output along forever, that's mainly a benefit to long running agents and for short tasks you won't notice.
- prodigycorp 9mo ago
  All of these things you mentioned are put into a footnote of the article.
pama 9mo ago
Agreed. You can get a better model using the codex-cli repo and having an agent help you analyze the core functionality.

splike 9mo ago

I'm interested, could you expand on that?

prodigycorp 9mo ago
Off the top of my head: parallel subagents, hooks, skills, and a much better plan mode. These features enable way better steering than we had last year. Subagents are a huge boon to productivity.
- rtgfhyuj 9mo ago
  are subagents just tools that are agents themselves?
  - dkdcio 9mo ago
    pretty much…they have their own system prompts, you can customize the model, the tools they use, etc. CC has built in subagents (including at least one not listed) that work very well: https://code.claude.com/docs/en/sub-agents#built-in-subagents https://code.claude.com/docs/en/sub-agents#built-in-subagent... this was not the case in the past, I swore off subagents, but they got good at some point

dkdcio 9mo ago

it seems to have changed a ton in recent versions too — I would love more details on what exactly I find it doing what I in the past had to interrupt and tell it to do fairly frequently now

terminalshort 9mo ago
For one thing it seems to splitting up the work and making some determination of complexity, then allocating it out to a model based on that complexity to save resources. When I run Claude with Opus 4.5 and run /cost I see tokens for Opus 4.5, but also a lot in Sonnet and Haiku, with the majority of tokens actually being used by Haiku.
- nyellin 9mo ago
  Haiku is called often, but not always the way you think. E.g. every time you write something CC invokes Haiku multiple times to generate the 'delightful 1-2 word phrase used to indicate progress to the user' (Doing Stuff, Wizarding, etc)
  - dkdcio 9mo ago
    it’s also used in the Explore agent and for other things too

lukan 9mo ago

The article was also published one year ago on january 2025. (Should have 2025 in the title? Time flies)

llmslave2 9mo ago
Claude Code didn't exist in January 2025. I think it's a typo and should be 2026.
- prodigycorp 9mo ago
  You’re right. No wonder the date felt odd. iirc Claude code was released around march.
  - dkdcio 9mo ago
    late Feb. 2025: https://www.anthropic.com/news/claude-3-7-sonnet https://www.anthropic.com/news/claude-3-7-sonnet

alright2565 9mo ago

But does that extra complexity actually improve performance? https://www.tbench.ai/leaderboard/terminal-bench/2.0 https://www.tbench.ai/leaderboard/terminal-bench/2.0 says yes, but not as much as you'd think. "Terminus" is basically just a tmux session and LLM in a loop.

prodigycorp 9mo ago
I'm not a good representative for claude code because I'm primarily a codex user now, but I know that if codex had subagents it would be at least twice as productive. Time spent is an important aspect of performance so yup, the complexity improved performance.
- nyellin 9mo ago
  Not necessarily true. Subagents allow for parallelization but they can decrease accuracy dramatically if you're not careful because there are often dependencies between tasks and swapping context windows with a summary is extremely lossy. For the longest time, Claude Code itself didnt really use subagents much by default, other than supporting them as a feature eager users could configure. (Source is reverse engineering we did on Claude code using the fantastic CC tracing tool Simon Willison wrote about once. This is also no longer true on latest versions that have e.g. an Explore subagent that is actively used.)
  - prodigycorp 9mo ago
    You’re right that subagents were more likely to cause issues than be helpful. But, when properly understood, lead to so much time saved through parallelization for tasks that warranted it. I was having codex organize my tv/movie library the other day by having it generate. most of the files were not properly labeled. I had codex generate transcripts, manually search the movie db to find descriptions of show episodes, and match the show descriptions against the transcripts to figure out which episode/season the show belonged to. Claude Code could have parallelized those manual checks and finished that task at 8x the speed.
- 9mo ago

qsort 9mo ago

Obviously modern harnesses have better features but I wouldn't say it invalidates the mental model. Simpler agents aren't that far behind in performance if the underlying model is the same, including very minimal ones with basic tools. I'd say it's similar to how a "make your own relational DB" article might feature a basic B-tree with merge-joins. Yeah, obviously real engines have sophisticated planners, multiple join methods, bloom filters, etc., but the underlying mental model is still accurate.

prodigycorp 9mo ago
You’re not wrong but I still think that the harness matters a lot when trying to accurately describe Claude Code. Here’s a reframing: If you asked people “what would you rather work with, today’s Claude Code harness with sonnet 3.7, or the 200 line agentic loop in the article with Opus 4.5, which would you choose?” I suspect many people would choose 3.7 with the harness. Moreover, that is true, then I’d say the article is no longer useful for a modern understanding of Claude Code.
- rfw300 9mo ago
  Any person who would choose 3.7 with a fancy harness has a very poor memory about how dramatically the model capabilities have improved between then and now.
  - prodigycorp 9mo ago
    I’d be very interested in the performance of 3.7 decked out with web search, context7, a full suite of skills, and code quality hooks against opus 4.5 with none of those. I suspect it’s closer than you think!
    - CuriouslyC 9mo ago
      Skills don't make any difference above having markdown files to point an agent to with instructions as needed. Context7 isn't any better than telling your agent to use trafilatura to scrape web docs for your libs, and having a linting/static analysis suite isn't a harness thing. 3.7 was kinda dumb, it was good at vibe UIs but really bad at a lot of things and it would lie and hack rewards a LOT. The difference with Opus 4.5 is that when you go off the Claude happy path, it holds together pretty well. With Sonnet (particularly <=4) if you went off the happy path things got bad in a hurry.

CuriouslyC 9mo ago

Less true than you think. A lot of the progress in the last year has been tightening agentic prompts/tools and getting out of the way so the model can flex. Subagents/MCP/Skills are all pretty mid, and while there has been some context pruning optimization to avoid carrying tool output along forever, that's mainly a benefit to long running agents and for short tasks you won't notice.

prodigycorp 9mo ago
All of these things you mentioned are put into a footnote of the article.

pama 9mo ago

Agreed. You can get a better model using the codex-cli repo and having an agent help you analyze the core functionality.