zambelli

9 ms·

31.
by zambelli 5mo ago
Oh neat! I'm a little slammed the next couple of weeks with an existing engagement + CAIS, but happy to connect and see if timelines work out?
32.
by zambelli 5mo ago
Forge doesn't modify the prompt, it just injects information into the conversation as if it was a conversation turn. Over many turns - it can degrade the model (a concept I'm calling "effective attention"). But that re
33.
by zambelli 5mo ago
Within limits, yes. Forge has escalating nudges that will tell the model effectively "stop responding with text, you MUST call a tool" vibes. If the model is emitting something like "ok, let me call the tool: [valid json tool
34.
by zambelli 5mo ago
I think there's certainly overlap there - and I love to see small local models being leveraged! I do think there's some differences though. The biggest one being that forge isn't a coding harness, it's a guardrail primit
35.
by zambelli 5mo ago
I think you have me confused with someone else. I haven't worked on any AI watermark removal project. What project are you referring to specifically?
36.
by zambelli 5mo ago
Fun! I've been looking at autonomous engineering as well - completely agree that tools and guardrails are the key. ToolResolutionError is really inspired by HTTP 4xx vs 5xx codes. I don't even have a super clean abstraction I&#x27
37.
by zambelli 5mo ago
Yes and no. Harnesses do have retry mechanisms. In opencode in particular, I think they return the error as-is to the model in the next turn. But that's slightly different. Harness retries come mostly in two flavors: 1) provider-layer:
38.
by zambelli 5mo ago
Basically, yeah...this uplifts everything I've tested, but it's the small models that benefit most. A perfect model would get no benefit. Mostly, I'm embarrassed I've done this whole public reveal without any use of Alan
39.
by zambelli 5mo ago
I think I'm aligned with the idea that some parts of some workflows are mandatory - auth, read before edit, etc. But otherwise, forge really doesn't own or opine much of the workflow. Step enforcement exists if you want it, so
40.
by zambelli 5mo ago
No, I haven't - hadn't heard of it. I'll try to squeeze in a quick read in the coming weeks!
41.
by zambelli 5mo ago
Oh, interesting idea. Formalizing an abstraction layer for testing all the integration types out there in the AI ether, essentially? MCP, skills, etc. I think this sits a level higher than Forge - maybe testing the workflow proper and integ
42.
by zambelli 5mo ago
Yeah, throughput on small models can get really fun :). As for MTP, should work fine since forge just sits between model and consumer. As long as MTP didn't change the model endpoint contract (ie, you call llama.cpp the same way you wo
43.
by zambelli 5mo ago
I'm sorry you feel that way about my posts - hopefully you still find the work valuable. Still human here btw, and still 100% honest.
44.
by zambelli 5mo ago
I believe there's a comment below mentioning "qwen" but not a specific version number - if you're looking for 3rd party validation. I've personally tried qwen3.6-35b-a3b, qwen3.5-35b-a3b, and qwen3.5-27b with forge
45.
by zambelli 5mo ago
Yep! The big frontier models are already quite good at doing that, and they have decent harnesses. That's why Opus on Claude Code does what it does. Small models aren't there yet and they would veer off course, this just nudges th
46.
by zambelli 5mo ago
100%, a better tool would work or even remove the problem overall. The isssue/use-case is more around, say, a database table or legacy systems where your tool is just hitting a legacy API that may or may not be good. A surface you don&
47.
by zambelli 5mo ago
That was fast! It's great to hear it's working well :) Did you notice any particular guardrails firing? Always curious about things I haven't tested on - especially if it has a different shape.
48.
by zambelli 5mo ago
lol - my bad! but thanks!
49.
by zambelli 5mo ago
Very cool! Hopefully you'll share it someday!
50.
by zambelli 5mo ago
Thank you! I completely agree - especially for always-on systems like agents crawling databases or doing audits and the like. The sheer volume of calls will be enormous and being able to run it on simple hardware with a small model that fit
51.
by zambelli 5mo ago
I haven't specifically tested this with Hermes, but I would expect so. Hermes is orchestrating things - it decides it needs to...whatever you want, book a trip for you. Forge will help make sure that the API calls to hotel booking site
52.
by zambelli 5mo ago
Yeah I'm sorry about that - I thought that link would work. Here is the fixed one (dashboard inside): https://github.com/antoinezambelli/forge/tree/main/docs/resu...
53.
by zambelli 5mo ago
Good call! The latest forge version has per-model-parameter configs sourced from official sources (can be overridden), that's what I'll use for evals and each eval set will be paired with a commit hash. But I'll make sure to
54.
by zambelli 5mo ago
I'm sorry to hear that! I'll take a fresh look at docs in my upcoming release. In a nutshell, it applies guardrails around LLM calls to make them more reliable - specifically small models but works on all: "on multi-step agen
55.
by zambelli 5mo ago
I genuinely don't, sadly. I'm a mathematician originally, evolved organically into ML then AI - but I never really was a SWE. I feel like there's some backend decoding or chat template thing going on at a much lower level tha
56.
by zambelli 5mo ago
Do you mean catching errors as tokens stream back versus waiting for the full message? If so, then no I hadn't looked into that. This was mostly geared towards local models so token cost isn't really a big deal, though latency m
57.
by zambelli 5mo ago
I went through a similar (in philosophy) exercise with my small-model agentic coding harness - built on forge. A few things I noticed related to your points: - on conversation rewind, I implemented a similar tool call collapse on the main a
58.
by zambelli 5mo ago
That's a fair point, and frankly something that might not age well in my docs one day. I genuinely don't know what the industry will standardize on when it comes to the use of the term "guardrails". I've seen the se
59.
by zambelli 5mo ago
Oh interesting - I hadn't come across that! I'd assume they could be combined. A coding harness would own the agentic workflow by nature, forge guardrails would help tool calling. I haven't given it a thorough read yet but I
60.
by zambelli 5mo ago
I would expect so! I'm currently running Gemma 4 E4B evals and it's behaving the same. Better with guardrails. There might be a floor where any error nudge confuses the model more than helps, but I haven't found it across man

More ›