3 ms·

I think you are confusing 2 things here. What the labs mean when they announce x hours sessions is on "one session" (i.e. the agent manages its own context via

I think you are confusing 2 things here. What the labs mean when they announce x hours sessions is on "one session" (i.e. the agent manages its own context via trimming and memory files, etc). What the project I linked did was "run in a bash loop", that basically resets the context every time the agent "finishes". That would mean that every few hours the agent starts fresh, does the inspect repo thing, does the plan for that session, and so on. That would explain why it took it ~3 months to do what a human + ai could probably do in a few weeks. That's why it doesn't sound too ludicrous for me. If you look at the repo there are a lot of things that are not strictly needed for the initial prompt (make a programming language like go but with genz stuff, nocap). Oh, and if you look at their discord + repo, lots of things don't actually work. Some examples do, some segfault. That's exactly what you'd expect from "running an agent in a loop". I still think it's impressive nonetheless. The fact that you are so incredulous (and I get why that is, scepticism is warranted in this space) is actually funny. We are on the right track.

sarchertech 1y ago
There’s absolutely no difference from what he says he did and what Claude code can do behind the scenes. If Anthropic thought they could produce anything remotely useful by wiping the context and reprompting every few hours, they would be doing it. And they’d be saying “look at this we implemented hard context reset and we can now run our agent for 30 days and produce an entire language implementation!” In 3 months or 300 years of operating like this a current agent being freshly reprompted every few hours would never produce anything that even remotely looked like a language implementation. As soon as its context was poisoned with slightly off topic todo comments it would spin out into writing a game of life implementation or whatever. You’d have millions of lines of nonsense code with nothing useful after 3 months of that. The only way I see anything like this doing anything approaching “useful” is if the outer loop wipes the repo on every reset as well, and collects the results somewhere the agent can’t access. Then you essentially have 100 chances to one shot the thing. But at that point you just have a needlessly expensive and slow agent.

sarchertech 1y ago

There’s absolutely no difference from what he says he did and what Claude code can do behind the scenes. If Anthropic thought they could produce anything remotely useful by wiping the context and reprompting every few hours, they would be doing it. And they’d be saying “look at this we implemented hard context reset and we can now run our agent for 30 days and produce an entire language implementation!” In 3 months or 300 years of operating like this a current agent being freshly reprompted every few hours would never produce anything that even remotely looked like a language implementation. As soon as its context was poisoned with slightly off topic todo comments it would spin out into writing a game of life implementation or whatever. You’d have millions of lines of nonsense code with nothing useful after 3 months of that. The only way I see anything like this doing anything approaching “useful” is if the outer loop wipes the repo on every reset as well, and collects the results somewhere the agent can’t access. Then you essentially have 100 chances to one shot the thing. But at that point you just have a needlessly expensive and slow agent.