8 ms·

What Is a Harness?

589 points by tosh 2mo ago

thelastgallon 2mo ago
Is harness load-bearing?
- 0x457 2mo ago
  No, its a smoking gun seam.
childofhedgehog 2mo ago
Clear, relevant, and easy to understand. Thank you for writing this up, I’ll be sharing this link with all my non-tech friends!
- ni10c 2mo ago
  Author here. Thanks - I appreciate the feedback
- phforms 2mo ago
  I second that. Not using agents myself but trying to get an idea on how this stuff works, so I always wondered what an "harness" even is, since anyone seems to assume that this is common knowledge. Now it is really clear to me!
theturtletalks 2mo ago
Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers. And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but none have extension functionality as good as Pi. Even if there is a new harness or agent project, I tell Pi to dig into the codebase and then make me an extension that brings that functionality into Pi. I did it with Prime Intellect’s and Deepseek’s harnesses and those are built on Pi.
- oceansky 2mo ago
  I want to move from Claude Desktop to Pi, but I found it a little unfriendly. Any tips to set it up?
  - zukzuk 2mo ago
    I haven’t tried it myself yet but I’m under the impression that Hermes Agent might be what you’re looking for?
  - theturtletalks 2mo ago
    Pi doesn’t have a UI like Claude Desktop. It also doesn’t work with the Claude subscription, only API key and pricing. So if you do want to use it, use the Codex sub. Once you install it, run Pi and /login and you’ll get login with ChatGPT. From there, Pi can tweak it’s settings if you ask. Check out their extensions (or ask Pi) and that will take you most of the way there. What hiccups were you having?
    - goosejuice 2mo ago
      > It also doesn’t work with the Claude subscription, only API key and pricing. Not out of the box, but you can add agent sdk. I'm not sure how great the results will be though.
  - ni10c 2mo ago
    Can you be more specific regarding unfriendliness?
- cyanydeez 2mo ago
  what have you built other than a harness?
  - theturtletalks 2mo ago
    I built a software factory and am now building a stock trader using opencandle extension[0] and a custom extension. For inspiration for how to tweak Pi, check out OMP, Prime Intellect, and Deepseek harnesses. 0. https://github.com/Kahtaf/OpenCandle https://github.com/Kahtaf/OpenCandle
jascha_eng 2mo ago
The ai hype word for 2026 after agent in 2025 for any LLM powered application. Well kind of, I wouldn't be surprised to see that some things marketed as agents are actually good old deterministic software.
- sph 2mo ago
  It's really funny (and a bit obnoxious) to watch the vocabulary from the outside. In 2023 everybody learned the word moat, then it's been agent(ic), from last year there's more talking about harnesses than at a bungee jumping convention. The mot du jour is frontier. It truly proves like there's a handful of thought leaders on Twitter that everybody follows blindly and start to copy down to the lexicon and parrot everywhere else.
  - okamiueru 2mo ago
    I find the vocabulary used to be disturbingly fascinating, to the degree concepts are being anthropomorphized. It's so pervasive that I cannot help but assume it is entirely deliberate. The principles by which LLMs functions haven't changed in the last four years. It is still a next-word-predictor, a statistical parrot, if you will. But if you don't understand the mechanisms behind it, you cannot be faulted for thinking this is something much more. Most of it, is pretty devious marketing. As an example, no LLM model does anything that can be considered "reasoning", or "intelligent" in the traditional sense, but these words are used extensively. "High reasoning model" is a pricing tier. The article in question has an anthropomorphized term in every single sentence. I'll pick a paragraph at random and highlight the cases. If both parties understand the mechanisms, these words are fine, and we do that all that time. The issue is when one sides is mislead to believe that these systems can be relied on in a way that they should not be, leading to people getting hurt. > > The *translation layer* is what *allows* a *harness* to *work* with different AI models. In some cases, a *harness may decide* to *use* different models within the same *agentic loop*, because different AI *models may excel* at different tasks. The *translation layer* is also a crucial aspect of *harnesses* because they *deliver control* to the end user. It means that someone can take their *AI harness* and use it with a model from Anthropic, or OpenAI, or explore one of the open weight AI models that often deliver great value-for-money (measured by cost-per-task). The underlying logic isn't remotely as mysterious or mystic as the language makes it seem. A different paragraph: > > Tools are a set of *capabilities*, written in code, that the model can *“call”*. The *harness describes* the tools and also *provides* the software that is the tool itself. Examples of these tools might include a web search tool, a tool that *allows the model* to write and execute software code, or a tool that *allows the model* to *compose* an email. Critically, the *harness usually* does not *dictate* when and how the *AI* model should *use* the tool. Instead, it simply *makes* the tools available, *describes* them clearly, and *allows* the *AI* model itself to *decide* when and how *it should use* them
driftproofhq 2mo ago
[flagged]
webdoodle 2mo ago
I thought this was the next evolution of the smartphone. One so smart that it does all the thinking for you. You don't even have to be conscious, you just do whatever it tells you too. Oh wait, that's what they do already.
tosh 2mo ago
i also like the backpack analogy the harness is what you take with you on a trip/task whatever you take with you is not free (system prompt, tools, skills …) some models are really good even if you bring almost no skills, tools or system prompt the harness is the complement to the model the better the model the more minimal the harness can be harnesses like pi [0] and smol [1]are on the more minimal end of things [0] https://github.com/earendil-works/pi https://github.com/earendil-works/pi [1] https://github.com/smol-env/smol https://github.com/smol-env/smol
- lebek 2mo ago
  Not a bad analogy because the bigger your backpack the slower you walk. With models a big context and tool set degrades performance. So you want the smallest harness/backpack that can do the job.
tingletech 2mo ago
A harness is "the code that gives a model an operational environment" according the the framing in https://github.com/shareAI-lab/learn-claude-code https://github.com/shareAI-lab/learn-claude-code -- which is a build your own python claude code tutorial shareAI made after the code leak a few months back.
stacktraceyo 2mo ago
To me, Before agentic programming a harness was like a mini framework in the app. Like for testing mostly. You’d set up the harness and configure it for your test and it would take care of boilerplate setup / optional reporting / benchmarks ect. Still works for both - but yea need a new word I guess
- asim 2mo ago
  From a naming terms yes, the closest equivalent of the past is the term "framework". Having written a Go service framework that's how I perceived it and as I started to work on agents, anything related to that became an "agent harness". I guess naming and terms change with different paradigms.
JBAnderson5 2mo ago
I have a similar mental model to the climbing harness. I think of LLMs as horses and harnesses as the saddle, reins, etc that you put on your horse. You might configure your harness for an individual rider or you might hook together several horses to pull a carriage.
myegorov 2mo ago
Written using a harness? Too verbose to be read.
jpitz 2mo ago
Harnesses are hands.
Syntaf 2mo ago
I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skills was often too prescriptive and limited to the authors own specific function in the company. A 2k line long skill suffers from the same gaps as we do, if an agent is just following a laundry list it’s less likely to reason about the request it’s doing. So we instead asked ourselves: what if we just _let_ the agent reason about the work to be done and only provided the tools + guardrails to gather context and perform accounting work? Turns out frontier models are GOOD at what they do, they outperformed our highly prescriptive skills and were able to work across a larger set of tasks even without instruction on how to do those tasks. It’s a breath of fresh air from the decade of CRUD I’ve worked on, harness engineering is very neat.
- pdhborges 2mo ago
  So you still have CLIs but they have I presume an help command that describes the capabilities right. Could you give an example of an accounting guardrail you created?
  - dpritchett 2mo ago
    I’ve also found that Claude and friends are eerily good at using classic Unix CLI tools so I build mine in the same style, not unlike the `gh` CLI from GitHub, though with an agent-first design shape. Usually I’m returning TSV as a default format and I add a `help-all` subcommand to list every available command at once when needed. Another thing that helps is adding just-in-time context-sensitive hints, such as: user has just run a list query with at least one result. Add a one-liner to the response explaining the command shape for getting the detail view of the first response. In terms of skill files, I like to have my CLI generate them dynamically at runtime by walking their own current command tree and then feeding that through a text template. Examples from a public project: https://github.com/radiusmethod/gitlab-kiosk/blob/main/skill/SKILL.md.tmpl https://github.com/radiusmethod/gitlab-kiosk/blob/main/skill...
jchip303 2mo ago
[dead]
dwheeler 2mo ago
Another AI harness is Goose, which is OSS and housed under the Linux Foundation (LF) Agentic AI Foundation (AAIF): https://github.com/aaif-goose/goose https://github.com/aaif-goose/goose Full disclosure: I work at the LF, but not the AAIF.
xrd 2mo ago
Does anyone have a suggestion for a harness that is good at handoff? When I say handoff, I mean: * handoff from a terminal CLI to webui (on a phone)? * handoff from one team member, to another? * handoff from one communication modality, like writing a prompt in a TUI, to email? * handoff from one model to another, or one provider (openrouter)( to another (llama.cpp) Does such a thing exist? I used to think that a PR would be a good place to centralize all this. Who cares what IDE, or developer, or location. But, now I feel like an agent harness might contain that better. Why do I want handoff? I keep losing context of where my harness is running. Sometimes I am inside an isolated VM. Sometimes I'm on my laptop, sometimes I'm on my home machine with the big GPU for local models. If I could spin up a harness that could identify itself inside my tailscale network, then I could probably have a single web UI which allows me to keep all that context straight. I'm tempted to experiment with Pi to configure such a thing. But, perhaps there are patterns out there already with a harness I have not considered.
- ds_opseeker 2mo ago
  Sounds like you want an orchestration. Let's assume handoff happens when one "agent" finishes its work on one task, i.e. "submit a PR". At that point you want to exit the agent/clear context etc (any context the next actor needs should be in the handoff artifact). And the orchestrator calls the next agent with the artifact. Claude can do this with subagents. If you want to get more serious, I'd look at "durable workflows" and check out what the pi people have to say: https://earendil-works.github.io/absurd/ https://earendil-works.github.io/absurd/ https://earendil-works.github.io/absurd/patterns/pi-ai-agent/ https://earendil-works.github.io/absurd/patterns/pi-ai-agent... you should also look at dbos https://www.dbos.dev/ https://www.dbos.dev/ And then do a search for these terms on HN and get some idea of their shortcomings vs a 'real' orchestration tool like Airflow or Dagster
- YZF 2mo ago
  I do this all the time in my workflow. Use any harness. Ask it to create a markdown file with the information required for the handoff. Use that downstream. Keep a "repo" of those markdown files. Are you trying to orchestrate or manage this sort of process?
- 2mo ago
freepiai 2mo ago
I think a harness is kind of anything around the intelligence that allows the intelligence to be applied towards something, some sort of task. A great (if off-color) example I remember hearing was how Steven Hawking was brilliant, but really needed that computer setup to be able to apply his intelligence. It really stands out to me as such a clear visual example of what a harness actually is. Anyway I've been building my own harness on top of pi- www.freepi.ai (it's based on Pi, but now I have an OpenAI compatible endpoint so I'm thinking of it more like free-api :-) ). Basically ad+training supported so I can offer completely free inference. It's really important to me that we don't have harnesses and intelligence trapped in a "have and have not" world. If we don't all have access to intelligence we will end up in a dark place. Thats again where the visual of Steven Hawking and the wheelchair really stand out in my mind. It's not enough to have the raw intelligence, we need a really good wheelchair too.
lovelybunch 2mo ago
[dead]
cyode 2mo ago
Great example of writing about AI that maintains a human voice. Starting off the post with a picture of the author + nod to real-world experience (climbing) is a reasonably strong “this is not slop” signal.
- jakzurr 2mo ago
  OK, I agree. FYI, the pic is Royal Robbins (https://en.wikipedia.org/wiki/Royal_Robbins https://en.wikipedia.org/wiki/Royal_Robbins), who was kind of a hero to some of my climbing buddies back in the 70's.
DarmokTanagra 2mo ago
apparently its an easy way to get on HN, seems like a great blogspam target
- alansaber 2mo ago
  >most popular harness writes a blog about harnesses
kmansm27 2mo ago
From these comments, it seems like people still don't understand what harnesses are... The point is you shouldn't build a harness, you should use a harness and change its system prompt, the tools it has, MCPs it has, give it skills, etc, to make it work for your usecase. You aren't "building a harness on top of pi" if all you're doing is the above. You're just using the harness to connect different things to the LLM.
ni10c 2mo ago
Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was: harness = chassis, model = engine, fuel = tokens, agent = car I’m curious what y’all might think and whether that analogy carries more explanatory power
- asQuirreL 2mo ago
  The first analogy that comes to mind, growing out of "harness", is more like harness = harness, model = horse (rather than harness as in climbing harness). I guess you could say that tokens = hay, and agent = horse and cart, from there? Not sure how useful the hay part is but you could observe from the second that there are many different things you could harness a horse to (also a plough, or a coach, or just a saddle) based on your goal.
  - johnmw 2mo ago
    I wonder if I'm the only person whose first analogy that came to mind was: model = toddler, harness = baby bouncing jumper harness
    - mijowi 2mo ago
      Although obviously unserious, I would argue a baby harness is mostly the opposite of what an agent harness does. A baby harness constrains and contains the baby, while an agent harness controls but also empowers the agent.
- troyvit 2mo ago
  I'm a climber so I'm biased but I really liked your climbing harness example because of the configuration you're able to easily make to the harness. Saying the harness is like a car's chassis doesn't work as well for me because the chassis isn't as configurable as a climbing harness for as little work. Getting deeper into the climbing analogy you can even swap out the harnesses themselves for wildly different climbs. Like using Claude Code with a bunch of agents for medical software (climbing K2 where that extra padding comes in super handy) and pi.dev with a local model for a respectable web project (sport route where you'll be back in a few hours and it's safe to be a little more exposed). I'm glad your article made HN, and thank you for pi!
neet_dev 2mo ago
[flagged]
vatsachak 2mo ago
[dead]
FinnLobsien 2mo ago
I feel like harnesses will become massively important for enterprise AI agents. Right now every tool is shipping some kind of AI agent, but I can’t help but feel that AI agents in large companies will eventually be some kind of internal app with internal MCPs, CLIs, APIs etc. There might be different harnesses for different use cases that different people have different levels of access to. This would make sense for the platform/infrastructure engineers who can build a modular harness that a person or team can get access to. You could have agents team members use locally that have memory enabled for personalization and then agents that anyone can use to ask questions about company context, which wouldn’t personalize things.
_pdp_ 2mo ago
A harness is all that plus compensating for bad model behaviour - which unfortunately few harnesses actually do
luciana1u 2mo ago
[flagged]
tedggh 2mo ago
I feel I’ve been left behind because all I use is Agents.md / Claude.md and Project.md. No skills, no tools. Everything I want my agent to do is specified in those two files. I also have a particular directory and file structure for things like scripts and sources which is indexed in the mds.
- anon7000 2mo ago
  Fairly sure that your agent is using tools like “edit file” behind the scenes. The post is describing the fundamentals of how agent harnesses work, models don’t exactly have a way to edit files just sending an API request to and forth the provider in a chatbox.
- alansaber 2mo ago
  You don't need to. Underlying agent operations (how it compacts, utilises memory, etc) is not user navigable.
bitwize 2mo ago
A harness is the thing that turns sloppotron spew into action. That's... pretty much it. LLMs are still "just" token prediction algorithms, but you can coax them into outputting things that look like commands. The harness figures out which bits are supposed to be commands, executes them, and feeds the output back into the context of the LLM. They also have prompts of their own to guide LLM behavior.
- deleted 2mo ago
  [deleted]
phs318u 2mo ago
My 2c. A lot of what’s being discussed here may boil down to experience (or lack thereof) of writing good functional (and non-functional) requirements. Some developers are always thinking about implementation (or at least, implementation approach), and as a result they are very technically prescriptive. There’s nothing wrong with that per se if you’re actively driving towards a specific architecture (for your own reasons - which could be specified as non-functional reqs). What others are describing is the BA skill of capturing good _business_ requirements ie. specifying outcomes and qualities rather than methods. We should be clear about who’s being the BA, the architect and the developer here. At one end, a human vibe coder is acting as BA and the AI as architect, designer and engineer. At the other end, the human is a senior dev closely instructing a AI junior dev. All approaches are valid depending on what you need and the AI tools available (eg. SOTA model vs small self-hosted open-weights model).
tannerr_dev 2mo ago
super cool site design
effnorwood 2mo ago
[dead]
profsummergig 2mo ago
When SE's cannot agree on the meaning of a tool, it indicates the tool is a placeholder for desire.
- alexjurkiewicz 2mo ago
  Do you think there is disagreement about the meaning of "harness" in AI? I would say there is almost unanimous consensus.
  - Revanche1367 2mo ago
    Comments on this very thread would indicate otherwise.
codelion 2mo ago
[dead]
wangii 2mo ago
`harness` is a term invented/borrowed to describe a vague vision of applying llm in a more controlled manner, suitable for semi organized, complex work. the sad part is then people shift the attention from the real problem but seeking inspiration from the terminology. the real problem: is there a general framework to allow llm with limited context size to approach problems, just like a capable newbie integrated into a chaotic code base? how does it work? I think it's an epistemic problem, and my fork of pi: pie (github.com/wangii/pie)
ggangsir 2mo ago
It is a loop of action-thinking-action based on users'input and LLM response.
respectattentio 2mo ago
[dead]
JackCooper7 2mo ago
[flagged]
muh_gradle 2mo ago
Excellent post.
ChicagoDave 2mo ago
I have a software architecture and engineering harness that was first constructed from session summaries in ChatGPT and Claude web. Then Claude desktop, then Claude Code. It’s private and I charge for it and I am always looking at the list of skills and agents people create (I’ve never copied anyone else’s stuff) and so far I like mine better. It makes me roughly 20x faster at building systems and Claude itself closes gaps in my knowledge like networking and cloud devops. If you’re serious about producing quality code, it’s my belief that a strong harness is the best way. There are articles about it from Fowler and Redhat.
paidx 2mo ago
[flagged]
chromanoid 2mo ago
When I think about a harness in context of AI, pit pony harnesses come into my mind ( https://commons.wikimedia.org/wiki/File:Descente_d%27un_cheval_dans_la_mine_(1).jpg https://commons.wikimedia.org/wiki/File:Descente_d%27un_chev... ). That's also how I always thought about them, maybe the climber/tool belt analogy is better when thinking about capabilities.
rurban 2mo ago
I have a completely different view on that. Maybe because I work with harnesses for decades already. The perl test suite is called harness. It's a build and test system, which makes it easy to test and run all supported platform and config options, and app/lib features and regressions. Like a CI. Basically a CI is the best harness. The agent is just a feedback loop between the model and the harness. Since the feedback loop must be fast, a real CI is too costly, and only serves as end verification. So a good harness enables fast tests for features per arch. With proper cleanup and parallelization.
nnevatie 2mo ago
A harness is the heuristics you'll apply to a statistics-based core.
throwawayqqq11 2mo ago
Am i the only one that does not trust the ootb harnesses? My approach is a llama-server wrapped in a locked down systemd service and a goose client/harness with a from-scratch apparmor profile behind a filtered network interface. Getting this running was alot of work and is certainly not perfect but with qwen3.8 and expected price hikes, i think it might be worth it.
sourcecodeplz 2mo ago
its best you build your own harness.
jkwang 2mo ago
[flagged]
zarldev 2mo ago
A harness is a tool that allows the LLM to focus on the primary objective that the user wants, coding, research, reports, finance anything really but the harness structures the agent interactions with the supporting tools to achieve the narrow scope. Driving the LLM with correct feedback from the domain knowledge provided by the tools. TBH when I was writing my harness this is why i have the Guardrails concept around Any tool, pre and post tool call, pre can make sure the data is formatted correctly lowering incorrect tool calls and post tool calls for validaition of results of output. Harness: https://github.com/zarldev/zarlmono/tree/main/zarlcode https://github.com/zarldev/zarlmono/tree/main/zarlcode Guardrail Design: https://zarldev.github.io/zarlmono/guardrails/ https://zarldev.github.io/zarlmono/guardrails/ All Docs: https://zarldev.github.io/zarlmono/ https://zarldev.github.io/zarlmono/
catigula 2mo ago
Whomever invented this word for "AI agents" really nailed the marketing and, to me, the cringe-factor.
madmecodes 2mo ago
[flagged]
francisofascii 2mo ago
Still not sure I get it. Which of the below are harnesses? These all sound like they meet the definiton. - Codex / VSCode Github Copilot Plugin - ChatGPT Web application - Code I wrote to send/process REST API requests to an Antropic / OpenAI API endpoint
sysfiend 2mo ago
Model -> brain cells' conections Harness -> everything else I've been working with different setups in parallel for months (openclaw, pi, cursor per project harnesses and codex) and, even though using the same model most of the time, I can clearly see how the behave in very different ways deppending on the setup. As a language model, language is our way to communicate and build everything around the models, which makes my younger self (who loved writing stories) very very happy :)

What Is a Harness?

589 points by tosh 2mo ago

thelastgallon 2mo ago

Is harness load-bearing?

0x457 2mo ago
No, its a smoking gun seam.

childofhedgehog 2mo ago

Clear, relevant, and easy to understand. Thank you for writing this up, I’ll be sharing this link with all my non-tech friends!

ni10c 2mo ago
Author here. Thanks - I appreciate the feedback
phforms 2mo ago
I second that. Not using agents myself but trying to get an idea on how this stuff works, so I always wondered what an "harness" even is, since anyone seems to assume that this is common knowledge. Now it is really clear to me!

theturtletalks 2mo ago

Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” Right now, it’s like an AC vs DC between Claude and ChatGPT, but once that settles, the harnesses will be the actual value providers. And Pi is the best harness because of the amazing extension system. You can build extensions that turn Pi into a stock trader, software factory, anything. I tried switching to another harness but none have extension functionality as good as Pi. Even if there is a new harness or agent project, I tell Pi to dig into the codebase and then make me an extension that brings that functionality into Pi. I did it with Prime Intellect’s and Deepseek’s harnesses and those are built on Pi.

oceansky 2mo ago
I want to move from Claude Desktop to Pi, but I found it a little unfriendly. Any tips to set it up?
- zukzuk 2mo ago
  I haven’t tried it myself yet but I’m under the impression that Hermes Agent might be what you’re looking for?
- theturtletalks 2mo ago
  Pi doesn’t have a UI like Claude Desktop. It also doesn’t work with the Claude subscription, only API key and pricing. So if you do want to use it, use the Codex sub. Once you install it, run Pi and /login and you’ll get login with ChatGPT. From there, Pi can tweak it’s settings if you ask. Check out their extensions (or ask Pi) and that will take you most of the way there. What hiccups were you having?
  - goosejuice 2mo ago
    > It also doesn’t work with the Claude subscription, only API key and pricing. Not out of the box, but you can add agent sdk. I'm not sure how great the results will be though.
- ni10c 2mo ago
  Can you be more specific regarding unfriendliness?
cyanydeez 2mo ago
what have you built other than a harness?
- theturtletalks 2mo ago
  I built a software factory and am now building a stock trader using opencandle extension[0] and a custom extension. For inspiration for how to tweak Pi, check out OMP, Prime Intellect, and Deepseek harnesses. 0. https://github.com/Kahtaf/OpenCandle https://github.com/Kahtaf/OpenCandle

jascha_eng 2mo ago

The ai hype word for 2026 after agent in 2025 for any LLM powered application. Well kind of, I wouldn't be surprised to see that some things marketed as agents are actually good old deterministic software.

sph 2mo ago
It's really funny (and a bit obnoxious) to watch the vocabulary from the outside. In 2023 everybody learned the word moat, then it's been agent(ic), from last year there's more talking about harnesses than at a bungee jumping convention. The mot du jour is frontier. It truly proves like there's a handful of thought leaders on Twitter that everybody follows blindly and start to copy down to the lexicon and parrot everywhere else.
- okamiueru 2mo ago
  I find the vocabulary used to be disturbingly fascinating, to the degree concepts are being anthropomorphized. It's so pervasive that I cannot help but assume it is entirely deliberate. The principles by which LLMs functions haven't changed in the last four years. It is still a next-word-predictor, a statistical parrot, if you will. But if you don't understand the mechanisms behind it, you cannot be faulted for thinking this is something much more. Most of it, is pretty devious marketing. As an example, no LLM model does anything that can be considered "reasoning", or "intelligent" in the traditional sense, but these words are used extensively. "High reasoning model" is a pricing tier. The article in question has an anthropomorphized term in every single sentence. I'll pick a paragraph at random and highlight the cases. If both parties understand the mechanisms, these words are fine, and we do that all that time. The issue is when one sides is mislead to believe that these systems can be relied on in a way that they should not be, leading to people getting hurt. > > The *translation layer* is what *allows* a *harness* to *work* with different AI models. In some cases, a *harness may decide* to *use* different models within the same *agentic loop*, because different AI *models may excel* at different tasks. The *translation layer* is also a crucial aspect of *harnesses* because they *deliver control* to the end user. It means that someone can take their *AI harness* and use it with a model from Anthropic, or OpenAI, or explore one of the open weight AI models that often deliver great value-for-money (measured by cost-per-task). The underlying logic isn't remotely as mysterious or mystic as the language makes it seem. A different paragraph: > > Tools are a set of *capabilities*, written in code, that the model can *“call”*. The *harness describes* the tools and also *provides* the software that is the tool itself. Examples of these tools might include a web search tool, a tool that *allows the model* to write and execute software code, or a tool that *allows the model* to *compose* an email. Critically, the *harness usually* does not *dictate* when and how the *AI* model should *use* the tool. Instead, it simply *makes* the tools available, *describes* them clearly, and *allows* the *AI* model itself to *decide* when and how *it should use* them

driftproofhq 2mo ago

[flagged]

webdoodle 2mo ago

I thought this was the next evolution of the smartphone. One so smart that it does all the thinking for you. You don't even have to be conscious, you just do whatever it tells you too. Oh wait, that's what they do already.

tosh 2mo ago

i also like the backpack analogy the harness is what you take with you on a trip/task whatever you take with you is not free (system prompt, tools, skills …) some models are really good even if you bring almost no skills, tools or system prompt the harness is the complement to the model the better the model the more minimal the harness can be harnesses like pi [0] and smol [1]are on the more minimal end of things [0] https://github.com/earendil-works/pi https://github.com/earendil-works/pi [1] https://github.com/smol-env/smol https://github.com/smol-env/smol

lebek 2mo ago
Not a bad analogy because the bigger your backpack the slower you walk. With models a big context and tool set degrades performance. So you want the smallest harness/backpack that can do the job.

tingletech 2mo ago

A harness is "the code that gives a model an operational environment" according the the framing in https://github.com/shareAI-lab/learn-claude-code https://github.com/shareAI-lab/learn-claude-code -- which is a build your own python claude code tutorial shareAI made after the code leak a few months back.

stacktraceyo 2mo ago

To me, Before agentic programming a harness was like a mini framework in the app. Like for testing mostly. You’d set up the harness and configure it for your test and it would take care of boilerplate setup / optional reporting / benchmarks ect. Still works for both - but yea need a new word I guess

asim 2mo ago
From a naming terms yes, the closest equivalent of the past is the term "framework". Having written a Go service framework that's how I perceived it and as I started to work on agents, anything related to that became an "agent harness". I guess naming and terms change with different paradigms.

JBAnderson5 2mo ago

I have a similar mental model to the climbing harness. I think of LLMs as horses and harnesses as the saddle, reins, etc that you put on your horse. You might configure your harness for an individual rider or you might hook together several horses to pull a carriage.

myegorov 2mo ago

Written using a harness? Too verbose to be read.

jpitz 2mo ago

Harnesses are hands.

Syntaf 2mo ago

I’ve been working on a harness for accounting agents at my job recently and it’s been a pretty interesting experience. We originally started with building a CLI tool so our LLMs could more easily interact with our platform. I cannot recommend enough the value of having an internal CLI. It’s both fun to build and extremely useful for agents. We paired this with skills initially, but found that the way folks built skills was often too prescriptive and limited to the authors own specific function in the company. A 2k line long skill suffers from the same gaps as we do, if an agent is just following a laundry list it’s less likely to reason about the request it’s doing. So we instead asked ourselves: what if we just _let_ the agent reason about the work to be done and only provided the tools + guardrails to gather context and perform accounting work? Turns out frontier models are GOOD at what they do, they outperformed our highly prescriptive skills and were able to work across a larger set of tasks even without instruction on how to do those tasks. It’s a breath of fresh air from the decade of CRUD I’ve worked on, harness engineering is very neat.

pdhborges 2mo ago
So you still have CLIs but they have I presume an help command that describes the capabilities right. Could you give an example of an accounting guardrail you created?
- dpritchett 2mo ago
  I’ve also found that Claude and friends are eerily good at using classic Unix CLI tools so I build mine in the same style, not unlike the `gh` CLI from GitHub, though with an agent-first design shape. Usually I’m returning TSV as a default format and I add a `help-all` subcommand to list every available command at once when needed. Another thing that helps is adding just-in-time context-sensitive hints, such as: user has just run a list query with at least one result. Add a one-liner to the response explaining the command shape for getting the detail view of the first response. In terms of skill files, I like to have my CLI generate them dynamically at runtime by walking their own current command tree and then feeding that through a text template. Examples from a public project: https://github.com/radiusmethod/gitlab-kiosk/blob/main/skill/SKILL.md.tmpl https://github.com/radiusmethod/gitlab-kiosk/blob/main/skill...

jchip303 2mo ago

[dead]

dwheeler 2mo ago

Another AI harness is Goose, which is OSS and housed under the Linux Foundation (LF) Agentic AI Foundation (AAIF): https://github.com/aaif-goose/goose https://github.com/aaif-goose/goose Full disclosure: I work at the LF, but not the AAIF.

xrd 2mo ago

Does anyone have a suggestion for a harness that is good at handoff? When I say handoff, I mean: * handoff from a terminal CLI to webui (on a phone)? * handoff from one team member, to another? * handoff from one communication modality, like writing a prompt in a TUI, to email? * handoff from one model to another, or one provider (openrouter)( to another (llama.cpp) Does such a thing exist? I used to think that a PR would be a good place to centralize all this. Who cares what IDE, or developer, or location. But, now I feel like an agent harness might contain that better. Why do I want handoff? I keep losing context of where my harness is running. Sometimes I am inside an isolated VM. Sometimes I'm on my laptop, sometimes I'm on my home machine with the big GPU for local models. If I could spin up a harness that could identify itself inside my tailscale network, then I could probably have a single web UI which allows me to keep all that context straight. I'm tempted to experiment with Pi to configure such a thing. But, perhaps there are patterns out there already with a harness I have not considered.

ds_opseeker 2mo ago
Sounds like you want an orchestration. Let's assume handoff happens when one "agent" finishes its work on one task, i.e. "submit a PR". At that point you want to exit the agent/clear context etc (any context the next actor needs should be in the handoff artifact). And the orchestrator calls the next agent with the artifact. Claude can do this with subagents. If you want to get more serious, I'd look at "durable workflows" and check out what the pi people have to say: https://earendil-works.github.io/absurd/ https://earendil-works.github.io/absurd/ https://earendil-works.github.io/absurd/patterns/pi-ai-agent/ https://earendil-works.github.io/absurd/patterns/pi-ai-agent... you should also look at dbos https://www.dbos.dev/ https://www.dbos.dev/ And then do a search for these terms on HN and get some idea of their shortcomings vs a 'real' orchestration tool like Airflow or Dagster
YZF 2mo ago
I do this all the time in my workflow. Use any harness. Ask it to create a markdown file with the information required for the handoff. Use that downstream. Keep a "repo" of those markdown files. Are you trying to orchestrate or manage this sort of process?
2mo ago

freepiai 2mo ago

I think a harness is kind of anything around the intelligence that allows the intelligence to be applied towards something, some sort of task. A great (if off-color) example I remember hearing was how Steven Hawking was brilliant, but really needed that computer setup to be able to apply his intelligence. It really stands out to me as such a clear visual example of what a harness actually is. Anyway I've been building my own harness on top of pi- www.freepi.ai (it's based on Pi, but now I have an OpenAI compatible endpoint so I'm thinking of it more like free-api :-) ). Basically ad+training supported so I can offer completely free inference. It's really important to me that we don't have harnesses and intelligence trapped in a "have and have not" world. If we don't all have access to intelligence we will end up in a dark place. Thats again where the visual of Steven Hawking and the wheelchair really stand out in my mind. It's not enough to have the raw intelligence, we need a really good wheelchair too.

lovelybunch 2mo ago

[dead]

cyode 2mo ago

Great example of writing about AI that maintains a human voice. Starting off the post with a picture of the author + nod to real-world experience (climbing) is a reasonably strong “this is not slop” signal.

jakzurr 2mo ago
OK, I agree. FYI, the pic is Royal Robbins (https://en.wikipedia.org/wiki/Royal_Robbins https://en.wikipedia.org/wiki/Royal_Robbins), who was kind of a hero to some of my climbing buddies back in the 70's.

DarmokTanagra 2mo ago

apparently its an easy way to get on HN, seems like a great blogspam target

alansaber 2mo ago
>most popular harness writes a blog about harnesses

kmansm27 2mo ago

From these comments, it seems like people still don't understand what harnesses are... The point is you shouldn't build a harness, you should use a harness and change its system prompt, the tools it has, MCPs it has, give it skills, etc, to make it work for your usecase. You aren't "building a harness on top of pi" if all you're doing is the above. You're just using the harness to connect different things to the LLM.

ni10c 2mo ago

Author here. It’s ironic because this post was clearly geared towards non-hackers. But now that we’re here.. the other analogy I considered presenting was: harness = chassis, model = engine, fuel = tokens, agent = car I’m curious what y’all might think and whether that analogy carries more explanatory power

asQuirreL 2mo ago
The first analogy that comes to mind, growing out of "harness", is more like harness = harness, model = horse (rather than harness as in climbing harness). I guess you could say that tokens = hay, and agent = horse and cart, from there? Not sure how useful the hay part is but you could observe from the second that there are many different things you could harness a horse to (also a plough, or a coach, or just a saddle) based on your goal.
- johnmw 2mo ago
  I wonder if I'm the only person whose first analogy that came to mind was: model = toddler, harness = baby bouncing jumper harness
  - mijowi 2mo ago
    Although obviously unserious, I would argue a baby harness is mostly the opposite of what an agent harness does. A baby harness constrains and contains the baby, while an agent harness controls but also empowers the agent.
troyvit 2mo ago
I'm a climber so I'm biased but I really liked your climbing harness example because of the configuration you're able to easily make to the harness. Saying the harness is like a car's chassis doesn't work as well for me because the chassis isn't as configurable as a climbing harness for as little work. Getting deeper into the climbing analogy you can even swap out the harnesses themselves for wildly different climbs. Like using Claude Code with a bunch of agents for medical software (climbing K2 where that extra padding comes in super handy) and pi.dev with a local model for a respectable web project (sport route where you'll be back in a few hours and it's safe to be a little more exposed). I'm glad your article made HN, and thank you for pi!

neet_dev 2mo ago

[flagged]

vatsachak 2mo ago

[dead]

FinnLobsien 2mo ago

I feel like harnesses will become massively important for enterprise AI agents. Right now every tool is shipping some kind of AI agent, but I can’t help but feel that AI agents in large companies will eventually be some kind of internal app with internal MCPs, CLIs, APIs etc. There might be different harnesses for different use cases that different people have different levels of access to. This would make sense for the platform/infrastructure engineers who can build a modular harness that a person or team can get access to. You could have agents team members use locally that have memory enabled for personalization and then agents that anyone can use to ask questions about company context, which wouldn’t personalize things.

_pdp_ 2mo ago

A harness is all that plus compensating for bad model behaviour - which unfortunately few harnesses actually do

luciana1u 2mo ago

[flagged]

tedggh 2mo ago

I feel I’ve been left behind because all I use is Agents.md / Claude.md and Project.md. No skills, no tools. Everything I want my agent to do is specified in those two files. I also have a particular directory and file structure for things like scripts and sources which is indexed in the mds.

anon7000 2mo ago
Fairly sure that your agent is using tools like “edit file” behind the scenes. The post is describing the fundamentals of how agent harnesses work, models don’t exactly have a way to edit files just sending an API request to and forth the provider in a chatbox.
alansaber 2mo ago
You don't need to. Underlying agent operations (how it compacts, utilises memory, etc) is not user navigable.

bitwize 2mo ago

A harness is the thing that turns sloppotron spew into action. That's... pretty much it. LLMs are still "just" token prediction algorithms, but you can coax them into outputting things that look like commands. The harness figures out which bits are supposed to be commands, executes them, and feeds the output back into the context of the LLM. They also have prompts of their own to guide LLM behavior.

deleted 2mo ago
[deleted]

phs318u 2mo ago

My 2c. A lot of what’s being discussed here may boil down to experience (or lack thereof) of writing good functional (and non-functional) requirements. Some developers are always thinking about implementation (or at least, implementation approach), and as a result they are very technically prescriptive. There’s nothing wrong with that per se if you’re actively driving towards a specific architecture (for your own reasons - which could be specified as non-functional reqs). What others are describing is the BA skill of capturing good _business_ requirements ie. specifying outcomes and qualities rather than methods. We should be clear about who’s being the BA, the architect and the developer here. At one end, a human vibe coder is acting as BA and the AI as architect, designer and engineer. At the other end, the human is a senior dev closely instructing a AI junior dev. All approaches are valid depending on what you need and the AI tools available (eg. SOTA model vs small self-hosted open-weights model).

tannerr_dev 2mo ago

super cool site design

effnorwood 2mo ago

[dead]

profsummergig 2mo ago

When SE's cannot agree on the meaning of a tool, it indicates the tool is a placeholder for desire.

alexjurkiewicz 2mo ago
Do you think there is disagreement about the meaning of "harness" in AI? I would say there is almost unanimous consensus.
- Revanche1367 2mo ago
  Comments on this very thread would indicate otherwise.

codelion 2mo ago

[dead]

wangii 2mo ago

`harness` is a term invented/borrowed to describe a vague vision of applying llm in a more controlled manner, suitable for semi organized, complex work. the sad part is then people shift the attention from the real problem but seeking inspiration from the terminology. the real problem: is there a general framework to allow llm with limited context size to approach problems, just like a capable newbie integrated into a chaotic code base? how does it work? I think it's an epistemic problem, and my fork of pi: pie (github.com/wangii/pie)

ggangsir 2mo ago

It is a loop of action-thinking-action based on users'input and LLM response.

respectattentio 2mo ago

[dead]

JackCooper7 2mo ago

[flagged]

muh_gradle 2mo ago

Excellent post.

ChicagoDave 2mo ago

I have a software architecture and engineering harness that was first constructed from session summaries in ChatGPT and Claude web. Then Claude desktop, then Claude Code. It’s private and I charge for it and I am always looking at the list of skills and agents people create (I’ve never copied anyone else’s stuff) and so far I like mine better. It makes me roughly 20x faster at building systems and Claude itself closes gaps in my knowledge like networking and cloud devops. If you’re serious about producing quality code, it’s my belief that a strong harness is the best way. There are articles about it from Fowler and Redhat.

paidx 2mo ago

[flagged]

chromanoid 2mo ago

When I think about a harness in context of AI, pit pony harnesses come into my mind ( https://commons.wikimedia.org/wiki/File:Descente_d%27un_cheval_dans_la_mine_(1).jpg https://commons.wikimedia.org/wiki/File:Descente_d%27un_chev... ). That's also how I always thought about them, maybe the climber/tool belt analogy is better when thinking about capabilities.

rurban 2mo ago

I have a completely different view on that. Maybe because I work with harnesses for decades already. The perl test suite is called harness. It's a build and test system, which makes it easy to test and run all supported platform and config options, and app/lib features and regressions. Like a CI. Basically a CI is the best harness. The agent is just a feedback loop between the model and the harness. Since the feedback loop must be fast, a real CI is too costly, and only serves as end verification. So a good harness enables fast tests for features per arch. With proper cleanup and parallelization.

nnevatie 2mo ago

A harness is the heuristics you'll apply to a statistics-based core.

throwawayqqq11 2mo ago

Am i the only one that does not trust the ootb harnesses? My approach is a llama-server wrapped in a locked down systemd service and a goose client/harness with a from-scratch apparmor profile behind a filtered network interface. Getting this running was alot of work and is certainly not perfect but with qwen3.8 and expected price hikes, i think it might be worth it.

sourcecodeplz 2mo ago

its best you build your own harness.

jkwang 2mo ago

[flagged]

zarldev 2mo ago

A harness is a tool that allows the LLM to focus on the primary objective that the user wants, coding, research, reports, finance anything really but the harness structures the agent interactions with the supporting tools to achieve the narrow scope. Driving the LLM with correct feedback from the domain knowledge provided by the tools. TBH when I was writing my harness this is why i have the Guardrails concept around Any tool, pre and post tool call, pre can make sure the data is formatted correctly lowering incorrect tool calls and post tool calls for validaition of results of output. Harness: https://github.com/zarldev/zarlmono/tree/main/zarlcode https://github.com/zarldev/zarlmono/tree/main/zarlcode Guardrail Design: https://zarldev.github.io/zarlmono/guardrails/ https://zarldev.github.io/zarlmono/guardrails/ All Docs: https://zarldev.github.io/zarlmono/ https://zarldev.github.io/zarlmono/

catigula 2mo ago

Whomever invented this word for "AI agents" really nailed the marketing and, to me, the cringe-factor.

madmecodes 2mo ago

[flagged]

francisofascii 2mo ago

Still not sure I get it. Which of the below are harnesses? These all sound like they meet the definiton. - Codex / VSCode Github Copilot Plugin - ChatGPT Web application - Code I wrote to send/process REST API requests to an Antropic / OpenAI API endpoint

sysfiend 2mo ago

Model -> brain cells' conections Harness -> everything else I've been working with different setups in parallel for months (openclaw, pi, cursor per project harnesses and codex) and, even though using the same model most of the time, I can clearly see how the behave in very different ways deppending on the setup. As a language model, language is our way to communicate and build everything around the models, which makes my younger self (who loved writing stories) very very happy :)