zambelli

7 ms·

91.
by zambelli 5mo ago
I'd like to think so! ;). It has some brains, but the key insight was to send the model domain-agnostic nudges. I don't need to know what you're trying to do, the LLM already knows, I just need to nudge it back on the structu
92.
by zambelli 5mo ago
In a nutshell, yes. It tries to anyways, but at the end of the day, some models get stuck and you hit a max iterations error that forge will raise, with some context, and the consumer can choose what it wants to do at that point.
93.
by zambelli 5mo ago
It would help ensure that the model executes its tool call correctly. So if you give Pi a task like booking travel... Pi decides to book a flight, hotel, car. It gets the flight in one go, but then sends "here is the payload : [json bl
94.
by zambelli 5mo ago
Does this work? https://github.com/antoinezambelli/forge/tree/main/docs/resu...
95.
by zambelli 5mo ago
Happy to answer questions about the eval methodology, the backend findings, or anything in the repo. I'll be around.
96.
Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks(github.com)
687 points by zambelli 5mo ago|252 comments
97.
by zambelli 5mo ago
Yes, I believe that will be the case. IBM already made the bet with granite models. Personally, I've found that with guardrails, local 8-14B models can match frontier models on agentic tasks. The key is simple tasks with volume. For ve
98.
by zambelli 5mo ago
My harness is only being used with on prem models, so I don't have any checks in place. If the gguf is somehow calling home, I'm not catching it.
99.
by zambelli 5mo ago
I trust the tool in that I don't send anything sensitive in there! Unless I built it, I assume it's going somewhere. We have a policy at work around this where our most sensitive data can only be passed to on prem models. That bei
100.
by zambelli 5mo ago
I use Opus 4.7 for personal stuff (basically for everything), but have been considering gpt-5.5 given all I hear about it. At work I use 4.6 because we don't have 4.7 yet...zzz... I also do a LOT of personal/portfolio work with se
101.
by zambelli 5mo ago
I personally don't. I do maintain a repo/obsidian vault of useful skills. They're patterns I've "debugged" (ie, how to setup a systemd unattended batch job specifically on this linux box). When I start a projec
102.
by zambelli 5mo ago
Are you sure these are talent shortages and not H1B postings? Are you filtering for changing JDs or taking something like the URL being up as canonical?
103.
by zambelli 5mo ago
I'm working on behavioral fine-tuning of small LLMs. Ie, not fine-tuning or distilling knowledge, but operating practices (TODO lists, scratchpad reflex, etc).
104.
by zambelli 5mo ago
Only thing I would add is the ability to point to new/uncataloged benchmarks. If I have a favorite benchmark that best matches my use-case, the ability to point to it and have it fuzzy match model names or what have you would be a neat
105.
by zambelli 6mo ago
I'll echo what's been said. I use Opus for long running coding tasks - mainly for its stability with long context. I can knock out huge chunks of a project in a single conversation without any attention degradation. Sonnet's
106.
by zambelli 6mo ago
I only use parallel agents when I have simple tasks to get done on all projects. Cleanup, running evals, etc. That whole day becomes parallel agents on easy tasks. Otherwise I do tight feedback loops on single agent (with subagents). When r
107.
by zambelli 6mo ago
I've been building a structured agent swarm to autonomously build apps using small local models. Big models can navigate large scope well enough, but smaller ones need more scoping. I feel like it's an under-developed dimension of
108.
by zambelli 6mo ago
I've recently been tasked with hiring new college grads (less than 2 years of experience) AI Engineers. Some things that we've been doing: 1) Do they understand the ecosystem of algorithms and models? How do those coexist? Statist
109.
by zambelli 6mo ago
Can you expand on your approach of defining things around state transitions? Are you thinking of it as the state of a task (built, validated, integrated, etc)? Or something else entirely? I'm not sure I followed that part and it seemed
110.
by zambelli 6mo ago
Anyone expecting a higher tier subscription to be announced since this current reduction? Cynicism aside - I do wonder what the future will hold given that current token burn rates aren't sustainable without VC cash. Anthropic even pus
111.
by zambelli 6mo ago
I'm glad to see that it stands its ground more than other models - which is a genuinely useful trait for an assistant. Both on technical and emotional topics.
112.
by zambelli 6mo ago
Are there any natrual ways of swapping from clock time to agent "active time"? For some agents that are running intermittently I might want to keep those memories longer (in clock time).