4 ms·

I wouldn't be surprised if agents start getting managed by a distributed agentic system - think about it. Right now you get codex/claude/etc... and it's system

I wouldn't be surprised if agents start getting managed by a distributed agentic system - think about it. Right now you get codex/claude/etc... and it's system prompt and various other internally managed prompts are locked down to the version you downloaded. What if a distributed system ran experimental prompts and monitored the success rate (what code makes it into a commit) and provides feedback to the agent manager. That could help automatically fine tune it's own prompts.

datadrivenangel 1y ago
This is what Anthropic does for their "High Compute" SWE benchmarking: " For our "high compute" numbers we adopt additional complexity and parallel test-time compute as follows: - We sample multiple parallel attempts. - We discard patches that break the visible regression tests in the repository, similar to the rejection sampling approach adopted by Agentless (Xia et al. 2024); note no hidden test information is used. - We then use an internal scoring model to select the best candidate from the remaining attempts. - This results in a score of 82.0% for Sonnet 4.5. " But running in parallel means you use more compute, meaning that the cost is higher. Good results are worth paying for, but if a super recursive approach costs more and takes longer than a human...