4 ms·

We do something similar with Magic Loops[0], but within the context of generating a single "loop" (automation). We've found that LLMs are pretty bad at prompti

We do something similar with Magic Loops[0], but within the context of generating a single "loop" (automation). We've found that LLMs are pretty bad at prompting other LLMs, unless the problem at hand is very limited in scope. It's too easy to get incorrect/expensive behavior otherwise (e.g. starts building a framework against an imaginary API, instead of using an existing tool). Our approach looks more like a state machine under the hood, mostly code with LLM-based "magic" sprinkled throughout. The tool can both edit code and LLM "blocks" as it sees fit, allowing it to change it's functionality and prompting dynamically. Interestingly, we first set the validate->fix threshold to N=3, but the "agent" often gets stuck in a pattern of retries based on low quality user input. [0] Feel free to give our tool a try, it's very much in an alpha state: https://magicloops.dev/ https://magicloops.dev/

3cats-in-a-coat 3y ago
That's what keeps LLMs from being self-sufficient it seems. Life has this peculiar ability to always keep basic necessities at the center, and always default to them, so the loop can keep going: 1. Try your hardest not to die. 2. Try your hardest to reproduce. 3. Try to thrive maybe also while at it, but DON'T FORGET 1 AND 2! This is why humans are this weird combination of instinct and reasoning that we find hard to control. The limbic system is very basic, but very strong and very stable. The primary loop. Everything else allows us to go farther, but when we fail we go back to relying on the limbic system. If we allowed reason to take over instinct, we'd likely end ourselves. Not that we can't end ourselves by also relying on instinct but essentially the strategy there is: sacrifice the intelligent shell and let the basic loop continue so we can rebuild the intelligent shell in a new, more stable and suited for the environment form.
hmottestad 3y ago
> starts building a framework against an imaginary API, instead of using an existing tool If you swap “API” for “business need” you’ve got a basic software developer. Most of the time those imaginary business needs are a waste of time, but every once in a while the solution is pure genius to a problem no one realised they had. Mostly the first though.
regularfry 3y ago
One interesting detail in the Code As Policies paper from last year was that they generate code from the given API, then recursively get the LLM to implement any functions from what it generates that don't already exist. I thought that was quite neat. I've started seeing those sort of hallucinated API calls as a signal for something that ought to exist; if it's predictable enough for an LLM to think it might, then maybe it should, to make it easier for humans too?

3cats-in-a-coat 3y ago

That's what keeps LLMs from being self-sufficient it seems. Life has this peculiar ability to always keep basic necessities at the center, and always default to them, so the loop can keep going: 1. Try your hardest not to die. 2. Try your hardest to reproduce. 3. Try to thrive maybe also while at it, but DON'T FORGET 1 AND 2! This is why humans are this weird combination of instinct and reasoning that we find hard to control. The limbic system is very basic, but very strong and very stable. The primary loop. Everything else allows us to go farther, but when we fail we go back to relying on the limbic system. If we allowed reason to take over instinct, we'd likely end ourselves. Not that we can't end ourselves by also relying on instinct but essentially the strategy there is: sacrifice the intelligent shell and let the basic loop continue so we can rebuild the intelligent shell in a new, more stable and suited for the environment form.

hmottestad 3y ago

> starts building a framework against an imaginary API, instead of using an existing tool If you swap “API” for “business need” you’ve got a basic software developer. Most of the time those imaginary business needs are a waste of time, but every once in a while the solution is pure genius to a problem no one realised they had. Mostly the first though.

regularfry 3y ago

One interesting detail in the Code As Policies paper from last year was that they generate code from the given API, then recursively get the LLM to implement any functions from what it generates that don't already exist. I thought that was quite neat. I've started seeing those sort of hallucinated API calls as a signal for something that ought to exist; if it's predictable enough for an LLM to think it might, then maybe it should, to make it easier for humans too?