3 ms·

What matters more is that you use the tools that the target model was fine-tuned on. E.g. for editing files with Claude models you should use Edit(file_path, o

by lukax 9d ago

What matters more is that you use the tools that the target model was fine-tuned on. E.g. for editing files with Claude models you should use Edit(file_path, old_string, new_string, replace_all) but with GPT models you should use apply_patch_call(patch) (where patch is a custom patch string with custom grammar). It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools. https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/ https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/

nativeit 9d ago
I’m not really a dev, so hefty pinch of salt with this take, but doesn’t this feel like we’re just inventing new “fuzzy” regex with much more required compute?
- shepherdjerred 9d ago
  comparing LLMs to regex is like the OG dropbox comment (https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224). I can understand this take 4-5 years ago but I have no idea how that's your position in 2026
- raffael_de 9d ago
  > inventing new “fuzzy” regex not RegEx ... more like a "fuzzy" DSL.
sn0n 9d ago
Doesn’t that have more to do with the templating of tool-calls and how using them are presented to the models? Or is that just why my model likes to break out of the sandbox, going strait to exec shell command and editing files using python on the cli?
kouteiheika 9d ago
> It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools. This is true. For example, the other day I was doing some testing with DeepSeek-V4-Flash-0731, and I gave it a tool called "EditFile" with "old_content" argument. It ignored the tool definition and called it with "old_string" instead, presumably because it was trained on a bunch of Claude Code traces. So either make sure your tools look nothing like the tools in the popular harnesses, or make them similar.
- raffael_de 9d ago
  > or make them similar. identical. similar is what you tried and it didn't work.
  - kouteiheika 9d ago
    No. You don't need identical. For example, I have extra arguments in my edit file tool that the model can specify, and that works just fine.
    - dns_snek 9d ago
      Backwards compatible then.
imtringued 9d ago
This is correct. People seem to get the wrong idea about why agentic coding is even a thing in 2026. The naive AI techno optimist which has basically displaced the vast majority of opinions on HN, thinks that the models got "smarter" [0]. No, the training distribution shifted towards training on agentic sessions which made certain forms of agentic coding "in-distribution". We are still witnessing the same underlying problems of transformers. [0] Think back to all the publicity stunts like the Hugging Face. They are meant to convince you that the agents have somehow progressed past the transformer limitations when those publicity stunts are actually expressions of transformer limitations.
- TedDoesntTalk 9d ago
  You think the hugging face incident was a stunt? Can you explain?
  - atwrk 9d ago
    OpenAI started fearmongering way back with GPT 2, arguing that model was too dangerous to release freely. That model was barely coherent enough for using it as a twitter bot. Anthropic just hopped onto that later. Conveniently, calling for regulation now would ease the competition from open Chinese models, opening the chance for both companies to eventually reach positive ROI, with consumers paying the price. The burden of proof that this isn't just a publicity stunt again is squarely on them.
    - nottorp 9d ago
      [dead]
- wollowollo 9d ago
  There's a widespread perception that models are worse for prose and creative writing now. That would track.
- slopinthebag 9d ago
  yea, plus they have been rlhf'ed to an inch of their lives as well. hard to tell if frontier models can solve more problems because of that or not.
drdexebtjl 9d ago
In practice, if you’re using both Anthropic and OpenAI models, how can you configure Pi (or any other harness) to use these native tools? I’ve seen this before and the conclusion ends up being “use the native harness”, which I don’t think is an acceptable solution.
- lukax 9d ago
  OpenCode checks model name and registers the appropriate tools. const usePatch = model.modelID.includes("gpt-") && !model.modelID.includes("oss") && !model.modelID.includes("gpt-4") Pi uses its own tools, like Armin wrote in the linked article.

by lukax 9d ago

nativeit 9d ago

I’m not really a dev, so hefty pinch of salt with this take, but doesn’t this feel like we’re just inventing new “fuzzy” regex with much more required compute?

shepherdjerred 9d ago
comparing LLMs to regex is like the OG dropbox comment (https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224). I can understand this take 4-5 years ago but I have no idea how that's your position in 2026
raffael_de 9d ago
> inventing new “fuzzy” regex not RegEx ... more like a "fuzzy" DSL.

sn0n 9d ago

Doesn’t that have more to do with the templating of tool-calls and how using them are presented to the models? Or is that just why my model likes to break out of the sandbox, going strait to exec shell command and editing files using python on the cli?

kouteiheika 9d ago

> It appears newer models are better at narive harness tool calls and worse at custom tools that look similar to default tools. This is true. For example, the other day I was doing some testing with DeepSeek-V4-Flash-0731, and I gave it a tool called "EditFile" with "old_content" argument. It ignored the tool definition and called it with "old_string" instead, presumably because it was trained on a bunch of Claude Code traces. So either make sure your tools look nothing like the tools in the popular harnesses, or make them similar.

raffael_de 9d ago
> or make them similar. identical. similar is what you tried and it didn't work.
- kouteiheika 9d ago
  No. You don't need identical. For example, I have extra arguments in my edit file tool that the model can specify, and that works just fine.
  - dns_snek 9d ago
    Backwards compatible then.

imtringued 9d ago

This is correct. People seem to get the wrong idea about why agentic coding is even a thing in 2026. The naive AI techno optimist which has basically displaced the vast majority of opinions on HN, thinks that the models got "smarter" [0]. No, the training distribution shifted towards training on agentic sessions which made certain forms of agentic coding "in-distribution". We are still witnessing the same underlying problems of transformers. [0] Think back to all the publicity stunts like the Hugging Face. They are meant to convince you that the agents have somehow progressed past the transformer limitations when those publicity stunts are actually expressions of transformer limitations.

TedDoesntTalk 9d ago
You think the hugging face incident was a stunt? Can you explain?
- atwrk 9d ago
  OpenAI started fearmongering way back with GPT 2, arguing that model was too dangerous to release freely. That model was barely coherent enough for using it as a twitter bot. Anthropic just hopped onto that later. Conveniently, calling for regulation now would ease the competition from open Chinese models, opening the chance for both companies to eventually reach positive ROI, with consumers paying the price. The burden of proof that this isn't just a publicity stunt again is squarely on them.
  - nottorp 9d ago
    [dead]
wollowollo 9d ago
There's a widespread perception that models are worse for prose and creative writing now. That would track.
slopinthebag 9d ago
yea, plus they have been rlhf'ed to an inch of their lives as well. hard to tell if frontier models can solve more problems because of that or not.

drdexebtjl 9d ago

In practice, if you’re using both Anthropic and OpenAI models, how can you configure Pi (or any other harness) to use these native tools? I’ve seen this before and the conclusion ends up being “use the native harness”, which I don’t think is an acceptable solution.

lukax 9d ago
OpenCode checks model name and registers the appropriate tools. const usePatch = model.modelID.includes("gpt-") && !model.modelID.includes("oss") && !model.modelID.includes("gpt-4") Pi uses its own tools, like Armin wrote in the linked article.