4 ms·
> What's the actual goal here? I tried to expand on my goals and paths I want to explore in a comment below [1], but basically I wonder if we can use this sort
by andyk 4y ago
> What's the actual goal here?
I tried to expand on my goals and paths I want to explore in a comment below [1], but basically I wonder if we can use this sort of technique as a more powerful version of CoT where prompts can break down a task into sub-tasks (as CoT does) and then recursively do that for each sub-task, until we hit a base-case on all of the sub-sub-...-sub-tasks and (when rolled back up?) the problem is solved.
> You may also be able to get better results by just including the definition of fibonacci in the outer prompt
Yeah, I played with including the mathematical definition of Fibonacci, for example in [2]:
<quote>
You are a recursive function ... the paragraph you generate will be an exact copy of this one ... but with updated variables as follows: FIB_INDEX = FIB_INDEX+1; CURR_MINUS_TWO = CURR_MINUS_ONE; CURR_MINUS_ONE = CURR_VALUE; CURR_VAL = CURR_MINUS_TWO + CURR_MINUS_ONE. Otherwise, ...
</quote>
[1] https://news.ycombinator.com/item?id=35240093 https://news.ycombinator.com/item?id=35240093
[2] https://raw.githubusercontent.com/andyk/recursive_llm/main/prompt_fibonnaci_include_math.txt https://raw.githubusercontent.com/andyk/recursive_llm/main/p...
- ShamelessC 4y agoSeems like your method is going to be under-represented in the training data and hence prone to error accumulating. Chain of thought works (better, at least) specifically because the model has seen examples of CoT in its data
- lgas 4y agoIf the goal is just to have the model break down each task into sub tasks until they are small enough to perform, why not implement the recursion in the code that calls the models where it's a solved problem? Even if you got this working really well, it's going to be somewhat probabilistic whereas implementing it in code is, well, deterministic.