5 ms·
Towards a harness that can do anything
- embedding-shape 3mo agoWhat has been the most helpful when developing harnesses: > When in doubt, simplify. Remove, trim and minimize. Reproduce issues in as small cases as possible, understand the full design completely, there is no shortcuts for this.
- lukasco 3mo agoI still find it hard to develop harnesses because you can't really test many turns with an llm in the middle. I suppose doing LLM as judge is one way to start to tackle this kind of thing.
- _superposition_ 3mo agoI really like this idea and the way you mapped the concepts to unix primitives. Indeed llms are already "unix native". I've been experimenting with similar event driven workflows using k8s primitives but that's one level up the stack. This makes a whole lot of sense to me in terms of organizing a shared mental model. Will definitely check it out. Thanks for the good work.
- ardatasci 3mo agothanks a lot! let me know what you think -- i'd love to share ideas with people doing similar stuff!
- _superposition_ 3mo agoI do have a gripe already... The readme says macos, however as of yet I don't see the actual dependency. Sort of defeats the *nix spirit as well, no?
- ardatasci 3mo agoYeah, a little... I was mostly building this for myself so I didn't really think about other platforms but I'll get to it asap :)
- mnky9800n 3mo agogood news. now you have a feature request.
- _superposition_ 3mo agoOr an incoming PR ;)
- ardatasci 3mo agoActually I was initially going to put this on my Arch machine because utilizing FUSE made a lot of sense to me, but I realized that I use my mac a lot more often so I suppose that's what caused that choice.
- FrattB 3mo agoWhy are we not just using Claude Code or Codex on our machine and using this thing? Real question...
- esafak 3mo agoThis one is open source, and potentially better, since it responds to (file system) events instead of polling.
- PhunkyPhil 3mo agoI'm kind of in the same boat and it's been pestering me for months. Every agent is simply a less capable Claude Code. If it had a lossless, massive context window (100m-1b tokens), then it will squash everything. Give it bash + r/w and it can in theory /goal anything. I think there's something to be gained in a production environment be siloing agents for reproducebility/auditability, but I suspect that will go away in the future. There's that video of a silly demo someone made of an OS that was just nested copilot instances that generated the HTML of each window, which allowed you to do whatever you could imagine. It was seen as silly because it was, but that seems truly transformative.
- mnky9800n 3mo agoi think the point is less about what agent you should or should not do and more about what is the natural harness for an agent to succeed in. And agents are often autonomously doing things right now, why would you want claude code doing such stuff for you?
- killix 3mo ago[flagged]
- brainless 3mo agoI kind of have a different idea of agents. I totally believe in a deterministic scaffold but I really think that an agent should be as deterministic as possible - the more code, the better. Think of a typical loop we may ask of Claude Code today (assume we are not using TDD): run some test suite with fail fast mode, diagnose if the failure is due to recent feature changes (pass reference to backend/frontend, github issues, PRD,...). Ask CC to decide if test failed due to feature change and then update the test. Perhaps ask CC to use sub-agent to investigate and fix (if deemed so). Commit each fix, move on to next. I know, this has so many ways to make blunder but I am talking about the agent here, not our error-prone test maintenance. What if we had an agent that had context of your codebase, deterministically ran test suite, linter, hooks, etc. The "English" prompt would become a code loop with the LLM only brought in to decide if a test has failed because of feature change. Also, we can extract git log, JIRA and what not. Each tool here is real code. Executable code that calls others and only prompts when they meet edge cases. Edge cases are defined but we can now accelerate the maintenance of these tools using agents themselves. But the system is built on "programs that do one thing and do it well" and then reach out to an LLM for its specific edge case. The agent is how these executables work with each other.
- alexpotato 3mo ago100% agree that the more deterministic code the better up to the limit where you need the LLM's ability to be non-deterministic to kick in. There is this ACM blog post called "Manual Work is a Bug" [0] that was originally written to help humans automate processes using code. I find it just as applicable today as when it was written. You and the LLM look at what has to be done and then figure out the scripts/tools to make it happen. You then tie those tools into a system. The more I use the above the more it makes sense and the worse the whole "just commit the prompt" seems like nonsense. 0 - https://queue.acm.org/detail.cfm?id=3197520 https://queue.acm.org/detail.cfm?id=3197520
- brainless 3mo agoThis is an interesting share, thanks. Yes, that is my mental model. Use coding agents to generate more "programs" (scripts) to automate everything. Have edge case handlers - and these handlers can develop/update the original scripts.
- rob 3mo agoWhat's with this "harness" word people have been trying to adopt lately? Are we all going rock climbing?
- ambicapter 3mo agoThe awesome power of the AI must be somehow contained by us mere mortals.
- erezsh 3mo agoframework is too 90s
- simlevesque 3mo agoWe're taming a beast.
- dexwiz 3mo agoI think a better term would be bridle, but most people don't know it. But they do put harnesses on their kids and pets.
- smartmic 3mo agoBuzzword bullshit bingo is a must for anyone who wants to ride the hype.
- Legend2440 3mo agoA harness is just a piece of software that exposes APIs for reading files, connecting to the internet, etc. It parses the LLM output for tool calls, executes the command, and puts the output back into the LLM input. That's all there is to it.
- simonreiff 3mo ago[flagged]
- dominotw 3mo ago> Awesome work! This is really impressive. I gave a GitHub star. { aislop pitch} > Again, great work. i can bet you didnt actually read the op. i hate these comments so much. selfish and rude.
- simonreiff 3mo agoI did indeed read every word. And read the code. Don't be rude.
- dominotw 3mo agoyou read the whole article and the only thought you had about it is "great work" ? your comment exclusively about your aislop project
- simonreiff 3mo agoThe author pointed out that JSON-wrangling and regex issues are a pain in the ass. I found them to be a pain in the ass in my own project, too. He also argued that his project implemented a kernel pattern that acted as a buffer between the LLM and the outside world. I too implemented a buffer pattern in my own project. He built an AI harness, I think it's a nice project, and if it extends MCP compatibility, I do think HIC Mouse would be a nice integration with Ambiance. Yes, I think Ambiance is a great project and I am wishing the author all the best luck and success in the future. Yes, I gave it a GitHub star. Sorry I didn't give fainter praise? By the way, I wrote every line of my project myself; I wrote every word of my research myself; I wrote every word of the copy on my website, including all legal provisions, docs, and blog posts, myself; I write all my comments on HN myself. I'm proud of my work and stand by my project, which ironically is dedicated to reducing AI slop and boosting accuracy so that AI agents can perform surgical, precise code repairs without ever touching or copying any other part of the code base. Let me know what you're doing to reduce slop, other than accusing people falsely of generating content with AI.
- shay_ker 3mo agoHow much do the labs post-train on the harness inputs & outputs? That's a critical piece to understand if a "generic" harness is at all possible
- yencabulator 3mo agohttps://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/ https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools/ says already too much to use foreign tools.
- simonreiff 3mo agoI think tools have to be flexible enough to accommodate a range of expected inputs. Even assuming models are trained to supply a particular schema, they are stochastic and randomly output variants regularly. My experience has been that it's better to try to auto-normalize the range of anticipated outputs and make tools as forgiving as possible rather than force an overly strict schema that might not match training data on the model.
- edg5000 3mo agoI've experimented with completely novel tool call syntaxes combined with ordering and structuring the context differenly, and they seem to have a high tolerance for this stuff. But you always risk losing performance, probably highly unlikely to exceed stock performance across the board.
- skybrian 3mo ago“That can do anything” gives me Zombo.com vibes. I would be more interested if it seemed to do particular things well.
- bcjordan 3mo agoFeel like there's now a lot of modern software that can do particular things well but there is new promise in finding universal abstractions that can ideally run recursively / safely run any software at their leaves. (Unix + tooling is most of the way there ofc) I feel like Docker Compose / K8S / VM / Dagger.io layers are close but can't quite always recurse flexibly and aren't always simple to run with. Networking / devices / auth are often awkward choke points
- abeppu 3mo agoOr if it had any kind of measurement across a range of tasks
- ChicagoDave 3mo agoI think domain specific harnesses are already surpassing generic harnesses. I also think software development is its own domain. My harness is a Claude Code plugin with its own brainstorming, adr, and planning skills with associated review and interview skills. Behavioral testing related to acceptance criteria is built in. Everything in my harness is gated to prevent ratholes. I recently inflated a docker container to execute a set of work with Claude in unsafe mode and immediately saw problems with everything it was doing…and then I realized I had not installed my harness. Running Claude without an engineering harness is like driving a car without brakes or a steering wheel.
- erezsh 3mo agoThank you for framing my laziness as being a daredevil
- edg5000 3mo agoI ask it to propose revisions, which I then review and provide detailed feedback on. It then works, and afterwards I review. I always commit beforehand. This works well for me.
- aszen 3mo agoIs your plugin just skills or does it have custom agents? My own experience is that Claude will use your skills but will ignore your agents or custom search tools.
- ChicagoDave 3mo agoIt adheres to the full scope of skills, agents, and my custom gates. I’d hate Claude without it. https://devarch.ai/ https://devarch.ai/.
- augment_me 3mo agoJust one more harness bro, one more harness I swear one more harness and we will have solved AGI bro
- chuckadams 3mo agoMapping agent concepts to the Unix environment sounds like a great idea in general, but I get off the train at adopting the FHS, which is an ancient relic that has no place in a green-field system. I don't know exactly what shape it should take, but something along the lines of Nix seems more appropriate. Or maybe Plan 9.
- all2 3mo agoOne of the ideas I had (and I'm still tinkering with) is 'agent as a linux user'. All of the perms/groups/access rights are already built in, so it would make sense to leverage these for agents running on the same host. Heck, there's even mail built in for async comms between agents.
- inferhaven 3mo agoLove the Unix philosophy and the buffs mentioned of the Linux FS. Lean, transparent, and auditability-first is exactly the direction harness' should continue in. Something I am convinced of though, there probably isn't a single `best` harness for all tasks. Different workloads will likely perform better with certain combinations of model + harness, especially when we are talking about token budgeting and cost tracking. Ambiance feels like a great base “kernel” to build those variants on top of, rather than the one true harness.
- cws_ai_buddy 3mo ago[flagged]
- Supermancho 3mo agoThis post is chock-full of soft ideas. They make no meaningful steps toward "a harness that can do anything". The suggestion is to replace a small node application with specific tooling, with a vm "to give it more capabilities". This is what Agent sandboxes are, already. Making the sandbox the harness, doesn't achieve a concrete goal.
- chuckadams 3mo agoI find it refreshing to read a blog of someone just noodling on an idea rather than pitching a new product.
- albertgoeswoof 3mo agoIt’s a pitch for their product ambience
- bakies 3mo agoI feel achieved in running pi in containers!
- langs 3mo agothe current bottlenecks for agents are actually bottlenecks of the LLMs, rather than the agents themselves.
- guardiangod 3mo agoI disagree with the idea that file is a good metaphor for LLM. Files have seek and byte streams, which is just an unneeded abstraction for LLM. The LLM doesn't need to seek or jump to the middle of a file, if you store and organize your data properly. Why force the LLM to use files over vector database or key-value stores, just because it's a design principal for UNIX (which is designed for human users, not LLMs.)
- pizlonator 3mo agoI use AI all the time But I don't want the AI to summarize my email for me. I don't want the AI to summarize my calendar for me. I don't want the AI to summarize my Zoom call for me. Thanks
- deanalyzer 3mo agoExactly right. There are many different domains that AI can be helpful in, and I am frustrated by how it's used to process information in a lossy manner.
- tonymet 3mo agoI was hoping this was a climbing harness that could service other trades too
- caspianmagnus 3mo ago[flagged]
- caspianmagnus 3mo ago[flagged]
- redchaosgoddess 3mo ago[flagged]
- wyre 3mo ago>I've been thinking about how to free LLMs from the chat pane for a few years now. LLM's are language models, you can absolutely control them with bash scripts and deterministic code, there are plenty of frameworks that already do that, and a great engineer will use them, but LLMS are at their most powerful when a user can give the agent an input and the model can run its ReAct loop. Wanting to free an LLM from a chat pane is like wanting to free email from the thread model, or closer, removing the chat window to DM a friend or colleague. >what can we learn from the before-fore times, when people used to actually write code? Treat an agent like a human writing code. Give them the best context, give them the best tools. This is why harnesses are overly complicated, because they need to guide the model through the context and tools it has available in a way that is efficient. A good harness is not incompatible with the Unix philosphy, it can do one thing well (interfacing with LLMs and giving them access to filesystems and compute), it will heavily use bash, stringing commands together with the cli tools that it knows (it's context) that it has, and and LLM will naturally handle text streams because that is what it does best. >Everything is a File If you want things to be deterministic why resort to plaintext? Wouldn't we want as much as possible to be typed? A computer can parse json which is what you want if you are trying to make your harness as deterministic as possible. >It watches our FS for changes with cursors on textfiles, Wow. What is your monthly token bill? I don't know how that would use less tokens than a 30 minute heartbeat, which as you mention will already use a lot of tokens. Why not have it notify your agent after a certain amount of files have been changed, or certain files you deem important? It seems like this user works at a 12 week programmer retreat and seems to post their cohort's blog posts about the projects they work on.
- IceHegel 3mo ago[flagged]
- sota_pop 3mo agoIt’s interesting to me how many people articulate things like the “Preiminary Truths” section as if they are novel insights. Effective people managers (of whom I would not specifically consider myself) have known these tenets for as long as history. “Be concise”, “state your intent clearly”, funny how these are touted as novel “strategies” with which to expertly direct AI. I don’t agree that “everything is a file”. Files are arrays of bytes. For an LLM, everything is a vector of tokens/embeddings. An aphorism I recently heard: "All sufficiently advanced technology eventually becomes a web browser". … seems apt especially in the context of the progression from chat-windows to harnesses and onwards to “harnesses that can do anything”.
- TacticalCoder 3mo ago> I don’t agree that “everything is a file”. Files are arrays of bytes. For an LLM, everything is a vector of tokens/embeddings. And yet, as of now, LLMs have a hammer (Bash / command line utilities) and every problem they have looks like a nail. If there are people around you who are non-techies that are using Claude Code or similar, you'll hear them ask "what the heck is cron?" and "why is it talking to me about Bash again?". At some point we may have LLMs working on their own output using a set of tools that'd be the equivalent of Bash (or any other terminal prompt) + command line utilities manipulating not files but tokens/embedded vectors but as of now, it sure looks like everything is a file, especially to LLMs.
- sota_pop 3mo agoThat is fair, I HAVE INDEED seen similar of those anecdotes. However, I still am skeptical of the file analogy. LLMs are good at traversing traditional file systems because file/tree-like structures very likely encompass an overwhelming supermajority of their training set. By contrast, other approaches like graph-based vector/data discovery via sql queries seem equally or more promising (to me), not to mention the ability to run curl/http queries. Either way, it’s still all prompt engineering approaches to discover/manage context. In this way, files seem like a “local minimum” ie a medium that dually optimizes interpretability and accessibility for both LLMs AND humans. Just riffing here, but this could even broadly be considered a discussion of memory-vs-storage (somewhat analogous to fluid vs crystallized memory in humans). In this way, one could imagine the models performing context compaction/backup by periodically dumping their context to files (or some other non-volatile storage) WITHOUT coming back to feature space.. just dump/load the tokens directly. Personally, I am much more interested in even other approaches like VLMs (using visual tokens), architectures like auto encoders and its variations, jepa architectures and other approaches that emphasize operating primarily within the latent space. To be fair, my particular interests have always been more in the computer vision area, and have been amused to watch the attention mechanism rise to prominence even over CNNs (given the contrast |similarity in their mechanics). Then again, I began my journey in the ML field when GANs were still the hotness, but I digress…… for LLMs, it’s tokens all the way down, and the name of the game is how discoverable and accessible can you make them?
- tasoeur 3mo agoLike many others in also building a wrap… sorry I meant a harness for Claude and codex, and though I’m initially going into the route of “VFx node editor, but with prompts” - why not go full meta and actually have your harness bootstrap another harness / flavor of a given tool or combination thereof as a generative output. Current project: https://sxp.studio/apps/subjectivezero https://sxp.studio/apps/subjectivezero
- mandeepj 3mo ago> that can do anything isn't that a tall claim?
- cheschire 3mo agoOh, it’s macOS only. The site didn’t say anything about it, or maybe I’m just blind. I had to look at the repo to see.
- edg5000 3mo agoI spent 6 months frantically working fulltime on a harness, and when it was finished, I just stopped using it and went back to Codex CLI. I've also stopped using all the fancy features: subagents, planning mode. I've also stopped worrying about context, since I just let codex hit compaction. Somehow this doesn't seem to hurt much. Weird.
- dnautics 3mo agoharnesses i suspect are better for noncoding tasks like wrapping claude code to be a personal assistant... but im still procrastinating on building mine... if only i had a personal assistant to tell me how much time i should spend on building a personal assistant wrapper for claude
- dizhn 3mo agoYou adapted to how your tool works rather than try to shape it into what it isn't. So far all the nondeterministic (prompt based) workflows I have tried have been meh in an "I can't tell if it's any better but it sure is more complicated and less portable" way. Current tools. Opencode and whatever cli i can't avoid (like claude code for my first month which I don't thinking I'll renew) usually accessed using Paseo for it's excellen mobile client.
- edg5000 3mo agoYeah, good point. The simpler solution eventually wins. But I do think a GUI makes more sense though, that was the reason I went my own way, but I think you're right that keeping it barebones is the way.
- semcycle 3mo ago[flagged]
- txoria 3mo ago[dead]
- schrodinger 3mo agoTruly enjoyed this article, but hate to admit one thing: I'm so burned by AI slop that once I noticed this seems to have been "assisted" by an LLM, I got irrationally angry because I felt tricked. It hit me at the end, when I read this: > The idea behind Ambiance is simple: the model's priors [...] Everything else here is just in service of that. I've noticed "in service of" take off similar to "load-bearing" with LLMs, and the whole structure just pattern matched to Claude for me. I went back and scanned it over again, and noticed several other tells: * "Think of U/L [Unix / Linux] as a motivating analogy rather than a direct comparison." * "Priors" _and_ "a priori" used in the same article. * "A real kernel […]. The Ambiance Kernel […]. The Kernel […]." — LLMs love this pattern. To be abundantly clear, *I'm not calling this AI SLOP*; it's obvious that a human put a lot of thought into this, gives a shit about the topic, and shared interesting ideas leading to a productive discussion. I really did like it. There's just something empty or hollow about the LLM style; paraphrasing Joni Mitchell "there's something lost and some thing gained" [1] when using AI. Your writing, your code is more consistent, more structured, more planned out — generally better but in a way that loses the character behind human writing. [1]: From "Both Sides Now", a great song about looking something from two perspectives: youthful innocence, and jaded cynicism. Listen to the original 1969 version first, and then the 2000 remake as you can tell she's singing from the respective perspectives. Deeply meaningful song!
- yetanotherjosh 3mo agoI experience this too, but the danger is is making assumptions based on "tells" which might just be how the user wrote the post. It's a situation now where if someone authentically writes something, but happens to rub up against some arbitrary "tell," they flip a bit in the reader and get their work rejected. For example, I used em dashes fairly regularly in my writing before AI started convincing everyone that the mere presence of an em dash is the AI tell, so I actually stopped using them, just to avoid this backlash.
- schrodinger 3mo agoI hear you. Coincidentally, I was actually called out just last night for a message with em dashes: “Are you AI-ing me? Em dash giving you away” It’s a pretty load-bearing (lol) example, yet it was clear in context it wasn’t serious. I’m beginning to notice people getting pretty good at detecting AI content, which I find reassuring. It makes sense: AI-written prose is basically the homogenization of all styles of writing into a single voice, so it’ll stand out against the unique personalities we are used to seeing. It is just taking a minute for the average person to gain literacy — but it seems to be happening rather quickly, again unsurprising since we’re such social monkeys with a lot of our 15 watt brains dedicated to socialization and identity recognition. Use your em-dash proudly!