3 ms·

What's the actual goal here? If you got it working really well, what is it that would you be able to do with it better than using some other approach? As to g

by lgas 4y ago

What's the actual goal here? If you got it working really well, what is it that would you be able to do with it better than using some other approach? As to getting the math/logic working better in the prompt, it seems like the obvious thing would be asking it to explain its work (CoT) before reproducing the new prompt. You may also be able to get better results by just including the definition of fibonacci in the outer prompt, but since it's not clear to me what your actual goal here is I'm not sure if either of those suggestions make sense. And since ChatGPT is down I can't test anything. :(

andyk 4y ago
> What's the actual goal here? I tried to expand on my goals and paths I want to explore in a comment below [1], but basically I wonder if we can use this sort of technique as a more powerful version of CoT where prompts can break down a task into sub-tasks (as CoT does) and then recursively do that for each sub-task, until we hit a base-case on all of the sub-sub-...-sub-tasks and (when rolled back up?) the problem is solved. > You may also be able to get better results by just including the definition of fibonacci in the outer prompt Yeah, I played with including the mathematical definition of Fibonacci, for example in [2]: <quote> You are a recursive function ... the paragraph you generate will be an exact copy of this one ... but with updated variables as follows: FIB_INDEX = FIB_INDEX+1; CURR_MINUS_TWO = CURR_MINUS_ONE; CURR_MINUS_ONE = CURR_VALUE; CURR_VAL = CURR_MINUS_TWO + CURR_MINUS_ONE. Otherwise, ... </quote> [1] https://news.ycombinator.com/item?id=35240093 https://news.ycombinator.com/item?id=35240093 [2] https://raw.githubusercontent.com/andyk/recursive_llm/main/prompt_fibonnaci_include_math.txt https://raw.githubusercontent.com/andyk/recursive_llm/main/p...
- ShamelessC 4y ago
  Seems like your method is going to be under-represented in the training data and hence prone to error accumulating. Chain of thought works (better, at least) specifically because the model has seen examples of CoT in its data
- lgas 4y ago
  If the goal is just to have the model break down each task into sub tasks until they are small enough to perform, why not implement the recursion in the code that calls the models where it's a solved problem? Even if you got this working really well, it's going to be somewhat probabilistic whereas implementing it in code is, well, deterministic.

by lgas 4y ago

andyk 4y ago

> What's the actual goal here? I tried to expand on my goals and paths I want to explore in a comment below [1], but basically I wonder if we can use this sort of technique as a more powerful version of CoT where prompts can break down a task into sub-tasks (as CoT does) and then recursively do that for each sub-task, until we hit a base-case on all of the sub-sub-...-sub-tasks and (when rolled back up?) the problem is solved. > You may also be able to get better results by just including the definition of fibonacci in the outer prompt Yeah, I played with including the mathematical definition of Fibonacci, for example in [2]: <quote> You are a recursive function ... the paragraph you generate will be an exact copy of this one ... but with updated variables as follows: FIB_INDEX = FIB_INDEX+1; CURR_MINUS_TWO = CURR_MINUS_ONE; CURR_MINUS_ONE = CURR_VALUE; CURR_VAL = CURR_MINUS_TWO + CURR_MINUS_ONE. Otherwise, ... </quote> [1] https://news.ycombinator.com/item?id=35240093 https://news.ycombinator.com/item?id=35240093 [2] https://raw.githubusercontent.com/andyk/recursive_llm/main/prompt_fibonnaci_include_math.txt https://raw.githubusercontent.com/andyk/recursive_llm/main/p...

ShamelessC 4y ago
Seems like your method is going to be under-represented in the training data and hence prone to error accumulating. Chain of thought works (better, at least) specifically because the model has seen examples of CoT in its data
lgas 4y ago
If the goal is just to have the model break down each task into sub tasks until they are small enough to perform, why not implement the recursion in the code that calls the models where it's a solved problem? Even if you got this working really well, it's going to be somewhat probabilistic whereas implementing it in code is, well, deterministic.