3 ms·

> An external loop really isn't any different to "think step by step" - you're giving the model a chance to see it's own prior output and carry on from that. I

by vidarh 3y ago

> An external loop really isn't any different to "think step by step" - you're giving the model a chance to see it's own prior output and carry on from that. It is very different, because you control the context fed in at the start of each cycle. Put another way, it's fairly easy to get most LLMs to follow instructions one step. With a loop, that makes it fairly easy to get them to follow however many steps you like. But getting them to follow multiple steps by asking them to "think step by step" tends to fall apart after quite few steps because suddenly your instructions are a small, distant part of the context. Test it. The difference is dramatic.

HarHarVeryFunny 3y ago
Interesting - it also suggests a "think step by step" approach where you tell the model to itself repeat (echo) the instructions before each step.
- vidarh 3y ago
  I think that'd be a worthwhile experiment, but I suspect it might not always be enough (but absolutely worth testing). You can "coerce" the model more strongly when you explicitly control everything, though. Adding a loop also allows you to put it in a "straitjacket" in many ways - e.g. if a local model you can select output tokens to force the response to match a given grammar to give it less room to go off the rails (see llama.cpp/llamafile that supports this), you can repeat and adjust temperature until responses match requirements or manipulating the answer so far and have it complete it. Or combine with an evaluation step to force a "does this answer comply with restrictions X, Y, Z: ..." stage, and so on. Of course these are not suitable to get you AGI, but when you have tasks where testing compliance or ranking quality of output are "cheap"/"easy" but coming up with answers is hard, they provide ways to get LLMs to work even in cases where the simple approach of individual, complex prompts are too unreliable. (the tradeoff of course being that if you have to spend too much time tuning these kinds of tricks to a specific problem you need to weigh up if they're worth it vs. solving the problem without an LLM) Of course they are workarounds, and I'm sure there will be architectural advances that will hopefully make these relatively crude methods less important, but in the meantime, they do help.

by vidarh 3y ago

HarHarVeryFunny 3y ago

Interesting - it also suggests a "think step by step" approach where you tell the model to itself repeat (echo) the instructions before each step.

vidarh 3y ago
I think that'd be a worthwhile experiment, but I suspect it might not always be enough (but absolutely worth testing). You can "coerce" the model more strongly when you explicitly control everything, though. Adding a loop also allows you to put it in a "straitjacket" in many ways - e.g. if a local model you can select output tokens to force the response to match a given grammar to give it less room to go off the rails (see llama.cpp/llamafile that supports this), you can repeat and adjust temperature until responses match requirements or manipulating the answer so far and have it complete it. Or combine with an evaluation step to force a "does this answer comply with restrictions X, Y, Z: ..." stage, and so on. Of course these are not suitable to get you AGI, but when you have tasks where testing compliance or ranking quality of output are "cheap"/"easy" but coming up with answers is hard, they provide ways to get LLMs to work even in cases where the simple approach of individual, complex prompts are too unreliable. (the tradeoff of course being that if you have to spend too much time tuning these kinds of tricks to a specific problem you need to weigh up if they're worth it vs. solving the problem without an LLM) Of course they are workarounds, and I'm sure there will be architectural advances that will hopefully make these relatively crude methods less important, but in the meantime, they do help.