4 ms·

> building your own harness is a huge undertaking, a deep rabbit hole. I eventually gave up on this task. It's not possible to fight OpenAI or Anthropic's engi

by bob1029 16d ago

> building your own harness is a huge undertaking, a deep rabbit hole. I eventually gave up on this task. It's not possible to fight OpenAI or Anthropic's engineering teams. Their reasoning models have all kinds of undocumented back door access to the base models that you'd never be able to replicate from the outside. Even if you had full access you would not have the engineering man hours or experience to keep up. I think this Agents API thing is a step too far, but Chat Completion is too cold now. Something approximating Responses API seems like the happy medium. You still get most of the control with the only blackbox part being the reasoning loop / tokens. Building agents using the GPT5.6 family w/ Responses API feels pretty close to Star Trek computer shit to me. I thought I was being clever with my DIY contraption on top of chat completion, but it wasn't even close. I have embraced the reality that I will need to use opaque reasoning tokens to give my clients the experiences they are paying me to provide.

surgical_fire 15d ago
> It's not possible to fight OpenAI or Anthropic's engineering teams And yet, I find Pi so much better to work with than Claude Code.
- teruakohatu 15d ago
  I don’t find it better than Claude Code but much better than OpenCode. So I am curious why less is more in that context.
lukebuehler 15d ago
I've been working on a custom managed agent (see my other top-level comment), I find it is actually a manageable undertaking. It does feel herculean, but somehow doable. I do not find their hidden reasoning tokens to be insurmountable as long as you match the behavior of codex or CC (which takes work, but, again, is doable). My managed agent harness currently matches Codex on several benchmarks like Terminal Bench.
ernsheong 15d ago
I built my harness in pi within herdr, I cloned (zipped and downloaded) 0xRichardH/pi-herdr-subagents and went from there, and used pi to build itself, adding gate workflow state control, provider fallbacks (I use many token plans), subagent skill injection, etc. It is highly custom to my needs and wants, and I think every developer needs to do this. I only talk to my planner which plans, and it subs out to designer, oracle, coders, testers, and reviewers, etc. It is thus highly optimized for correctness. You can TDD or no TDD. You can fast track small changes. I tweak my harness dozens of times as I encounter new edge cases (esp when I switch models and encounter models not as good at following instructions). As you can start to see, it is better to own the harness because nobody can build something custom that 100% fits your needs or development philosophy.
- boredtofears 15d ago
  I'm very interested in this but I am confused on what Pi provides you if you are building the harness? What does Pi get you that writing from scratch doesn't? Any good starting points or tutorials you recommend?
  - kadoban 15d ago
    Pi is just a nice base and it has defined extension protocols and such. You might as well start there, it's just easier and going from nothing to working to adding whatever functionality is like 2 minutes.
- tokioyoyo 15d ago
  Correct me if I'm wrong, but the harness will always be dependent on the underlying model, and useless without it. All custom harnesses are being built, could be obsolete in the next big-generation-jump of the models. I might be absolutely wrong, but "harnesses" / cc-derivatives became "good enough" only maybe a year ago max. Before that, people were pushing for gigantic folder structures with custom documents and "pretend you're X" stuff. My point is, depended on what you're trying to achieve, testing out current-gen harnesses, and nudging your workflows towards them might be better RoI, rather than chasing something that might be throwaway code a quarter later. Obviously, this really depends on whether you believe model development will speed up or slow down in the upcoming future.
mingqiz 15d ago
Yet i find claude code worse than opencode.
- stymaar 15d ago
  I don't think anyone who's tried something else can like claude code. The only reason to use it is the subsidized plan and nothing else.
  - meetingthrower 15d ago
    But it does have some very cool features built in I haven't seen rolled in elsewhere. Sessions can talk to one another out of the box. Two left arrows and you see all the sessions. Automatic remote control to pick up a session from another computer or even on your phone. So convenient! I could patch this together with Herdr and OMP and some custom server stuff but it works right out of the box!?
    - anthonypasq 15d ago
      Cursor does all that and actually lets you look at the code
      - meetingthrower 13d ago
        Right but I do this in VS code and it does the same?
z2 15d ago
After being burned by the rug-pull of OpenAI retiring the Assistants API in favor of Responses last year, I swore off using heavily stateful APIs for language model access. I could be totally wrong, but at this point I'm more willing to use a proprietary harness headless than to abstract it into an API.
- bob1029 15d ago
  [dead]
reissbaker 15d ago
I feel like you could use an open-source harness like Pi and get 100+% of what these closed APIs offer without getting locked to OpenAI. What do you think is missing from them?
- bob1029 15d ago
  Lack of risk to the business. If it was just me and my tinker toys, the open path makes a lot of sense. I am in the business of provisioning custom AI robots for paying customers. I need to be able to provide certain assurances and offer operational simplicity or the whole thing falls apart. These systems need to survive in their environments. Accounting and compliance teams feel a lot more comfortable with the landscape around the frontier model companies. If I went to my favorite client on Monday and tried to sell them what HN seems to think is the most ideologically pure AI solution, I would probably be fired as a vendor.
_davide_ 15d ago
> > building your own harness is a huge undertaking, a deep rabbit hole. > I eventually gave up on this task. It's not trivial, but cmon, i did during weekends from my phone and FOR ME it's so much better than the codex or claude, it has every i need and want :D I'm using my own harness for work and hobby, has github integration, review mode, interactive voice mode, overlayed worktree, browser integration, mcp and much more. Using claude and codex feels like picking up a club, in-line with the caveman skill...
anon-3988 15d ago
A good harness is a universal harness. It should not depend on the model at all. I foresee harness being the edge that people have. For instance, if you are able to encode laws as a lean program, that would be an absolutely insane harness that would make you leap way ahead of your competitor. Right now, I am interested in creating a personal assistant. It is crucial that this is done privately and safely. To do this, I run a local gemma4 model that have access to a copy of my emails. The only outside world that it can interact with is a private Telegram chat that I have with it.
simonwsimonw 15d ago
[dead]
aniceperson 14d ago
pi + pi web on a raspberry pi. an agent is just a path with its own .pi/ folder. you can have clown/.pi with only jokes mcp enabled (no bash, no read/write) and for sessions spawn from this path, the agent will think it is in the circus reality and will never go rogue (no tool to do so)

by bob1029 16d ago

surgical_fire 15d ago

> It's not possible to fight OpenAI or Anthropic's engineering teams And yet, I find Pi so much better to work with than Claude Code.

teruakohatu 15d ago
I don’t find it better than Claude Code but much better than OpenCode. So I am curious why less is more in that context.

lukebuehler 15d ago

I've been working on a custom managed agent (see my other top-level comment), I find it is actually a manageable undertaking. It does feel herculean, but somehow doable. I do not find their hidden reasoning tokens to be insurmountable as long as you match the behavior of codex or CC (which takes work, but, again, is doable). My managed agent harness currently matches Codex on several benchmarks like Terminal Bench.

ernsheong 15d ago

I built my harness in pi within herdr, I cloned (zipped and downloaded) 0xRichardH/pi-herdr-subagents and went from there, and used pi to build itself, adding gate workflow state control, provider fallbacks (I use many token plans), subagent skill injection, etc. It is highly custom to my needs and wants, and I think every developer needs to do this. I only talk to my planner which plans, and it subs out to designer, oracle, coders, testers, and reviewers, etc. It is thus highly optimized for correctness. You can TDD or no TDD. You can fast track small changes. I tweak my harness dozens of times as I encounter new edge cases (esp when I switch models and encounter models not as good at following instructions). As you can start to see, it is better to own the harness because nobody can build something custom that 100% fits your needs or development philosophy.

boredtofears 15d ago
I'm very interested in this but I am confused on what Pi provides you if you are building the harness? What does Pi get you that writing from scratch doesn't? Any good starting points or tutorials you recommend?
- kadoban 15d ago
  Pi is just a nice base and it has defined extension protocols and such. You might as well start there, it's just easier and going from nothing to working to adding whatever functionality is like 2 minutes.
tokioyoyo 15d ago
Correct me if I'm wrong, but the harness will always be dependent on the underlying model, and useless without it. All custom harnesses are being built, could be obsolete in the next big-generation-jump of the models. I might be absolutely wrong, but "harnesses" / cc-derivatives became "good enough" only maybe a year ago max. Before that, people were pushing for gigantic folder structures with custom documents and "pretend you're X" stuff. My point is, depended on what you're trying to achieve, testing out current-gen harnesses, and nudging your workflows towards them might be better RoI, rather than chasing something that might be throwaway code a quarter later. Obviously, this really depends on whether you believe model development will speed up or slow down in the upcoming future.

mingqiz 15d ago

Yet i find claude code worse than opencode.

stymaar 15d ago
I don't think anyone who's tried something else can like claude code. The only reason to use it is the subsidized plan and nothing else.
- meetingthrower 15d ago
  But it does have some very cool features built in I haven't seen rolled in elsewhere. Sessions can talk to one another out of the box. Two left arrows and you see all the sessions. Automatic remote control to pick up a session from another computer or even on your phone. So convenient! I could patch this together with Herdr and OMP and some custom server stuff but it works right out of the box!?
  - anthonypasq 15d ago
    Cursor does all that and actually lets you look at the code
    - meetingthrower 13d ago
      Right but I do this in VS code and it does the same?

z2 15d ago

After being burned by the rug-pull of OpenAI retiring the Assistants API in favor of Responses last year, I swore off using heavily stateful APIs for language model access. I could be totally wrong, but at this point I'm more willing to use a proprietary harness headless than to abstract it into an API.

bob1029 15d ago
[dead]

reissbaker 15d ago

I feel like you could use an open-source harness like Pi and get 100+% of what these closed APIs offer without getting locked to OpenAI. What do you think is missing from them?

bob1029 15d ago
Lack of risk to the business. If it was just me and my tinker toys, the open path makes a lot of sense. I am in the business of provisioning custom AI robots for paying customers. I need to be able to provide certain assurances and offer operational simplicity or the whole thing falls apart. These systems need to survive in their environments. Accounting and compliance teams feel a lot more comfortable with the landscape around the frontier model companies. If I went to my favorite client on Monday and tried to sell them what HN seems to think is the most ideologically pure AI solution, I would probably be fired as a vendor.

_davide_ 15d ago

> > building your own harness is a huge undertaking, a deep rabbit hole. > I eventually gave up on this task. It's not trivial, but cmon, i did during weekends from my phone and FOR ME it's so much better than the codex or claude, it has every i need and want :D I'm using my own harness for work and hobby, has github integration, review mode, interactive voice mode, overlayed worktree, browser integration, mcp and much more. Using claude and codex feels like picking up a club, in-line with the caveman skill...

anon-3988 15d ago

A good harness is a universal harness. It should not depend on the model at all. I foresee harness being the edge that people have. For instance, if you are able to encode laws as a lean program, that would be an absolutely insane harness that would make you leap way ahead of your competitor. Right now, I am interested in creating a personal assistant. It is crucial that this is done privately and safely. To do this, I run a local gemma4 model that have access to a copy of my emails. The only outside world that it can interact with is a private Telegram chat that I have with it.

simonwsimonw 15d ago

[dead]

aniceperson 14d ago

pi + pi web on a raspberry pi. an agent is just a path with its own .pi/ folder. you can have clown/.pi with only jokes mcp enabled (no bash, no read/write) and for sessions spawn from this path, the agent will think it is in the circus reality and will never go rogue (no tool to do so)