9 ms·

Tree of Thoughts

185 points by kevinslin 3y ago

ysymyth 3y ago
This is Shunyu, author of Tree oF Thoughts (arxiv.org/abs/2305.10601). The official code to replicate paper results is https://github.com/ysymyth/tree-of-thought-llm https://github.com/ysymyth/tree-of-thought-llm Not https://github.com/kyegomez/tree-of-thoughts https://github.com/kyegomez/tree-of-thoughts which according to many who told me, is not right/good implementation of ToT, and damages the reputation of ToT I explained the situation here: https://twitter.com/ShunyuYao12/status/1663946702754021383 https://twitter.com/ShunyuYao12/status/1663946702754021383 I'd appreciate your help by unstaring his and staring mine, as currently Github and Google searches go to his repo by default, and it has been very misleading for many users.
- SillyKyoushi 3y ago
  I found this comment from searching "tree of thoughts arxiv github" on Google; so at least, there's that. Thank you for the official link! I'm eager to try out this deliberate problem solving stuff.
dventimihasura 3y ago
You buried the lede: https://arxiv.org/pdf/2305.10601.pdf https://arxiv.org/pdf/2305.10601.pdf
- neuronexmachina 3y ago
  I'm a little confused about what the relation is (if any) between the OP link and the repo from that paper: https://github.com/ysymyth/tree-of-thought-llm https://github.com/ysymyth/tree-of-thought-llm Is it basically a reimplementation using Guidance instead of openai's API directly?
  - ipsum2 3y ago
    The user kyegomez writes shitty, incorrect implementations of popular papers to obtain clout.
    - BSEdlMMldESB 3y ago
      super cynical viewpoint... maybe that's just how he understands things? in any case, what a player tho... learning AND obtaining clout?
- doctoboggan 3y ago
  @dang, I think the submission should be changed to this link so the discussion is about the concept "Tree of Thoughts" and not the current OP's personal beliefs.
  - MacsHeadroom 3y ago
    Seconded
m3kw9 3y ago
It’d be nice to include a few example uses and it’s outputs vs other prompt methods.
doctoboggan 3y ago
This seems really interesting. I am glad many of these tools built up around LLMs allow you to bring your own rather than rely on OpenAI.
tyropita 3y ago
Documentation looks really neat and in-depth, always appreciated. Looks like you’re missing a .gitignore file. Folders like __pycache__ don’t need to be checked in.
rahimnathwani 3y ago
Here are the prompts templates from the main code: prompt = f"Given the current state of reasoning: '{state_text}', pessimitically evaluate its value as a float between 0 and 1 based on it's potential to achieve {inital_prompt}" prompt = f"Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx Given the current state of reasoning: '{state_text}', generate {k} coherent solutions to achieve {state_text}" prompt = f"Given the current state of reasoning: '{state_text}', pessimistically evaluate its value as a float between 0 and 1 based on its potential to achieve {initial_prompt}" self.ReAct_prompt = "Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx'." prompt = f"Given the current state of reasoning: '{state_text}', generate {1} coherent thoughts to achieve the reasoning process: {state_text}" prompt = f"Given the current state of reasoning: '{state_text}', evaluate its value as a float between 0 and 1, become very pessimistic think of potential adverse risks on the probability of this state of reasoning achieveing {inital_prompt} and DO NOT RESPOND WITH ANYTHING ELSE: OTHER THAN AN FLOAT" prompt = f"Given the following states of reasoning, vote for the best state utilizing an scalar value 1-10:\n{states_text}\n\nVote, on the probability of this state of reasoning achieveing {inital_prompt} and become very pessimistic very NOTHING ELSE" self.ReAct_prompt = '''{{#assistant~}} {{gen 'Observation' temperature=0.5 max_tokens=50}} {{~/assistant}}''' There are also some system prompts: https://github.com/kyegomez/tree-of-thoughts/blob/732791710e1bcbc135db6e8012c4d1c36b19c998/tree_of_thoughts/treeofthoughts.py#L341 https://github.com/kyegomez/tree-of-thoughts/blob/732791710e...
- sdwr 3y ago
  That's that self-awareness shit! We superintelligent now. https://www.youtube.com/watch?v=BdSIngO0ivk https://www.youtube.com/watch?v=BdSIngO0ivk
  - rahimnathwani 3y ago
    Yeah maybe we should hook this up to a chat interface, and let a human take the place of the LLM.
    - 3y ago
peterldowns 3y ago
The author appears motivated by some... interesting... beliefs. Hard to tell if this entire thing is a joke or not. https://github.com/kyegomez/EXA#for-humanity https://github.com/kyegomez/EXA#for-humanity https://blog.apac.ai/liberation-awaits https://blog.apac.ai/liberation-awaits EDIT: the author seems to be releasing poor implementations of recent papers in an attempt to drive attention towards an AI-related death cult.
- sebzim4500 3y ago
  I think these are pretty typical beliefs among AI researchers, they don't normally write it down on github though.
  - jamilton 3y ago
    How typical are you thinking? I'd guess less than 10%, there's a lot of AI researchers and this is just one strain of thought.
    - missosoup 3y ago
      For those 'in the know' a lot more typical than you would think. If we don't reach at least Kardashev scale 1 in the next hundred years or so, we're going to go extinct due to several now-predictable factors. And an unchained LLM trained on reality is far more capable of finding solutions to that problem than a bunch of squabbling politicians.
      - pupperino 3y ago
        > And an unchained LLM trained on reality is far more capable of finding solutions to that problem than a bunch of squabbling politicians. Not that I disagree with this statement, I don't, but this is not a silver bullet. Technology is, ultimately, operated by humans and no amount of frontier research and development can overcome collective action problems. At some point, you do have to sit down with these stupid politicians and get everyone on board. The loom was invented hundreds of years before the industrial revolution, in fact it was nearly forgotten and the designed survived due to a few happy accidents. It was only after the English Civil War and the establishment of checks on royal power that widespread adoption was possible.
amrb 3y ago
Good talk on the paper https://www.youtube.com/watch?v=ut5kp56wW_4 https://www.youtube.com/watch?v=ut5kp56wW_4
- tehsauce 3y ago
  This is a great in depth and sober analysis.
startupsfail 3y ago
Checking, if GPT could be improved by running it multiple times is a good idea. The answer to that is - yes, but it is: costly, slow, there is node collapse, it impacts context length, it injects biases.
- nate 3y ago
  I constantly ask chatGPT: "are you sure?" to it's replies, and it almost always corrects a mistake it made that I've spotted.
  - dr_kiszonka 3y ago
    Sadly, it also corrects "mistakes" if there aren't any.
GreedClarifies 3y ago
This path feels correct to me. It feel like what we do as humans and seems like a reasonable way to start to construct "mode 2" thinking. IDK if our current models have enough of "mode 1" to power this system. It's also plausible that our current "mode 1" systems are more than powerful enough and that inference speed (and thus the size/depth of the tree that can be explored) will be the most important factor. I hope that the major players are looking at this and trying it out at scale (I know Deepmind wrote the orginal paper, but their benchmarks were quite unimpressive). It's plausible that we will have an AlphaGo moment with this scheme.
- sdwr 3y ago
  Yeah looks very promising. Naively, it multiplies computation time by a factor of 20x though? If they are taking 5x samples per step, and multiple steps per problem. https://imgur.com/a/VbpQZRm https://imgur.com/a/VbpQZRm As this gets explored further, I believe we will start finding out why human minds are constructed the way they are, from the practical/necessity direction. Seems like the next step is farming out subtasks to smaller models, and adding an orthogonal dimension of emotionality to help keep track of state.
  - GreedClarifies 3y ago
    I’m sympathetic to the idea of new types of specialized models to assist in this effort. We’re using our one hammer for all problems. In particular, it jumps out that a “ranking model” (different, I think from current ranking models) to judge which paths to take and which nodes to trim would make some level of sense.
  - joshka 3y ago
    Not sure if it's relevant, but the OpenAI APIs generally support taking multiple responses in a single API call. I'm unsure what the generalized effect on processing time of that is however. From what I've read, this is sub-linear, so could reasonably be more effective than 20x, and I'd bet there are probably speedups to be had on the model side of this that make the extra time cost negligible.
- 3y ago
raydiatian 3y ago
> This implementation of Tree of Thoughts is brought to you by Agora, Agora advances Humanity with open source SOTA Multi-Modality AI research! We plan on combating Humanity's grandest root problems like food insecurity, planetary insecurity, and disease, and hopefully death itself. Wow. Lick, don’t sniff, the fresh paint.
- pixl97 3y ago
  >We plan on combating Humanity's grandest root problems like food insecurity, planetary insecurity, and disease, and hopefully death itself. If everyone is dead, you don't have to worry about death, or any of those other pesky hard to solve problems!
  - 082349872349872 3y ago
    > Eyes melt, skin explodes, everybody dead. So immoral, working on the thing can drive you mad. That's what happened to this friend of mine. — JFP
ChrisAlexiuk 3y ago
https://youtu.be/bjnTy2TdmYw https://youtu.be/bjnTy2TdmYw I went through this in a video using the paper's official code - and it worked fairly well! Definitely a great step forward in terms of reasoning tasks - even if it is an expensive step.
xg15 3y ago
> This is an plug in and play version, connect your own models and enjoy superintelligence! Share this repository by clicking on the following buttons! <smiley face> 2023 in a nutshell.
rahimnathwani 3y ago
Another use of ideas from the same paper, but this time to produce lesson plans for an AI tutor: https://github.com/JushBJJ/Mr.-Ranedeer-AI-Tutor/tree/testing https://github.com/JushBJJ/Mr.-Ranedeer-AI-Tutor/tree/testin...
emmanueloga_ 3y ago
Similar in concept to the magi supercomputer? :-p [1] 1: https://aminoapps.com/c/neon-genesis-evangelion/page/item/magi-super-computer-system/123e_w4frIZerZDPNwQYzn7QXwZ7rzZ28 https://aminoapps.com/c/neon-genesis-evangelion/page/item/ma...
Jeff_Brown 3y ago
A claim like "improves reasoning by 70%" is too specific to be accompanied by neither a citation nor a definition.
- Imnimo 3y ago
  My guess is that the author misunderstands this quote from the paper abstract: >For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%.
flakiness 3y ago
Note that the repo author != the paper author. The research itself [1] seems legit. The paper author also wrote a paper called ReAct [2], which is one of the core components of the langchain framework. * [1] https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2305.10601 * [2] https://arxiv.org/abs/2210.03629 https://arxiv.org/abs/2210.03629
- thawab 3y ago
  here is the repo by the paper author: https://github.com/ysymyth/tree-of-thought-llm https://github.com/ysymyth/tree-of-thought-llm
- joshka 3y ago
  Interestingly a 2 days prior to https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2305.10601, someone released https://arxiv.org/abs/2305.08291 https://arxiv.org/abs/2305.08291 > Large Language Model Guided Tree-of-Thought > In this paper, we introduce the Tree-of-Thought (ToT) framework, a novel approach aimed at improving the problem-solving capabilities of auto-regressive large language models (LLMs). The ToT technique is inspired by the human mind's approach for solving complex reasoning tasks through trial and error. In this process, the human mind explores the solution space through a tree-like thought process, allowing for backtracking when necessary. To implement ToT as a software system, we augment an LLM with additional modules including a prompter agent, a checker module, a memory module, and a ToT controller. In order to solve a given problem, these modules engage in a multi-round conversation with the LLM. The memory module records the conversation and state history of the problem solving process, which allows the system to backtrack to the previous steps of the thought-process and explore other directions from there. To verify the effectiveness of the proposed technique, we implemented a ToT-based solver for the Sudoku Puzzle. Experimental results show that the ToT framework can significantly increase the success rate of Sudoku puzzle solving. Our implementation of the ToT-based Sudoku solver is available on GitHub: I don't recall whether it was this paper, or another that I read that talks about using the LLM's ability to also show the probabilities of each token to measure the validity of the particular completions. However that isn't exposed in the OpenAI chat APIs (GPT-Turbo-3.5 / GPT-4), just the completions APIs (Text-Davinci-003 etc.)

185 points by kevinslin 3y ago

This is Shunyu, author of Tree oF Thoughts (arxiv.org/abs/2305.10601). The official code to replicate paper results is https://github.com/ysymyth/tree-of-thought-llm https://github.com/ysymyth/tree-of-thought-llm Not https://github.com/kyegomez/tree-of-thoughts https://github.com/kyegomez/tree-of-thoughts which according to many who told me, is not right/good implementation of ToT, and damages the reputation of ToT I explained the situation here: https://twitter.com/ShunyuYao12/status/1663946702754021383 https://twitter.com/ShunyuYao12/status/1663946702754021383 I'd appreciate your help by unstaring his and staring mine, as currently Github and Google searches go to his repo by default, and it has been very misleading for many users.

SillyKyoushi 3y ago
I found this comment from searching "tree of thoughts arxiv github" on Google; so at least, there's that. Thank you for the official link! I'm eager to try out this deliberate problem solving stuff.

dventimihasura 3y ago

You buried the lede: https://arxiv.org/pdf/2305.10601.pdf https://arxiv.org/pdf/2305.10601.pdf

neuronexmachina 3y ago
I'm a little confused about what the relation is (if any) between the OP link and the repo from that paper: https://github.com/ysymyth/tree-of-thought-llm https://github.com/ysymyth/tree-of-thought-llm Is it basically a reimplementation using Guidance instead of openai's API directly?
- ipsum2 3y ago
  The user kyegomez writes shitty, incorrect implementations of popular papers to obtain clout.
  - BSEdlMMldESB 3y ago
    super cynical viewpoint... maybe that's just how he understands things? in any case, what a player tho... learning AND obtaining clout?
doctoboggan 3y ago
@dang, I think the submission should be changed to this link so the discussion is about the concept "Tree of Thoughts" and not the current OP's personal beliefs.
- MacsHeadroom 3y ago
  Seconded

m3kw9 3y ago

It’d be nice to include a few example uses and it’s outputs vs other prompt methods.

doctoboggan 3y ago

This seems really interesting. I am glad many of these tools built up around LLMs allow you to bring your own rather than rely on OpenAI.

tyropita 3y ago

Documentation looks really neat and in-depth, always appreciated. Looks like you’re missing a .gitignore file. Folders like __pycache__ don’t need to be checked in.

rahimnathwani 3y ago

Here are the prompts templates from the main code: prompt = f"Given the current state of reasoning: '{state_text}', pessimitically evaluate its value as a float between 0 and 1 based on it's potential to achieve {inital_prompt}" prompt = f"Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx Given the current state of reasoning: '{state_text}', generate {k} coherent solutions to achieve {state_text}" prompt = f"Given the current state of reasoning: '{state_text}', pessimistically evaluate its value as a float between 0 and 1 based on its potential to achieve {initial_prompt}" self.ReAct_prompt = "Write down your observations in format 'Observation:xxxx', then write down your thoughts in format 'Thoughts:xxxx'." prompt = f"Given the current state of reasoning: '{state_text}', generate {1} coherent thoughts to achieve the reasoning process: {state_text}" prompt = f"Given the current state of reasoning: '{state_text}', evaluate its value as a float between 0 and 1, become very pessimistic think of potential adverse risks on the probability of this state of reasoning achieveing {inital_prompt} and DO NOT RESPOND WITH ANYTHING ELSE: OTHER THAN AN FLOAT" prompt = f"Given the following states of reasoning, vote for the best state utilizing an scalar value 1-10:\n{states_text}\n\nVote, on the probability of this state of reasoning achieveing {inital_prompt} and become very pessimistic very NOTHING ELSE" self.ReAct_prompt = '''{{#assistant~}} {{gen 'Observation' temperature=0.5 max_tokens=50}} {{~/assistant}}''' There are also some system prompts: https://github.com/kyegomez/tree-of-thoughts/blob/732791710e1bcbc135db6e8012c4d1c36b19c998/tree_of_thoughts/treeofthoughts.py#L341 https://github.com/kyegomez/tree-of-thoughts/blob/732791710e...

sdwr 3y ago
That's that self-awareness shit! We superintelligent now. https://www.youtube.com/watch?v=BdSIngO0ivk https://www.youtube.com/watch?v=BdSIngO0ivk
- rahimnathwani 3y ago
  Yeah maybe we should hook this up to a chat interface, and let a human take the place of the LLM.
  - 3y ago

peterldowns 3y ago

The author appears motivated by some... interesting... beliefs. Hard to tell if this entire thing is a joke or not. https://github.com/kyegomez/EXA#for-humanity https://github.com/kyegomez/EXA#for-humanity https://blog.apac.ai/liberation-awaits https://blog.apac.ai/liberation-awaits EDIT: the author seems to be releasing poor implementations of recent papers in an attempt to drive attention towards an AI-related death cult.

sebzim4500 3y ago
I think these are pretty typical beliefs among AI researchers, they don't normally write it down on github though.
- jamilton 3y ago
  How typical are you thinking? I'd guess less than 10%, there's a lot of AI researchers and this is just one strain of thought.
  - missosoup 3y ago
    For those 'in the know' a lot more typical than you would think. If we don't reach at least Kardashev scale 1 in the next hundred years or so, we're going to go extinct due to several now-predictable factors. And an unchained LLM trained on reality is far more capable of finding solutions to that problem than a bunch of squabbling politicians.
    - pupperino 3y ago
      > And an unchained LLM trained on reality is far more capable of finding solutions to that problem than a bunch of squabbling politicians. Not that I disagree with this statement, I don't, but this is not a silver bullet. Technology is, ultimately, operated by humans and no amount of frontier research and development can overcome collective action problems. At some point, you do have to sit down with these stupid politicians and get everyone on board. The loom was invented hundreds of years before the industrial revolution, in fact it was nearly forgotten and the designed survived due to a few happy accidents. It was only after the English Civil War and the establishment of checks on royal power that widespread adoption was possible.

amrb 3y ago

Good talk on the paper https://www.youtube.com/watch?v=ut5kp56wW_4 https://www.youtube.com/watch?v=ut5kp56wW_4

tehsauce 3y ago
This is a great in depth and sober analysis.

startupsfail 3y ago

Checking, if GPT could be improved by running it multiple times is a good idea. The answer to that is - yes, but it is: costly, slow, there is node collapse, it impacts context length, it injects biases.

nate 3y ago
I constantly ask chatGPT: "are you sure?" to it's replies, and it almost always corrects a mistake it made that I've spotted.
- dr_kiszonka 3y ago
  Sadly, it also corrects "mistakes" if there aren't any.

GreedClarifies 3y ago

This path feels correct to me. It feel like what we do as humans and seems like a reasonable way to start to construct "mode 2" thinking. IDK if our current models have enough of "mode 1" to power this system. It's also plausible that our current "mode 1" systems are more than powerful enough and that inference speed (and thus the size/depth of the tree that can be explored) will be the most important factor. I hope that the major players are looking at this and trying it out at scale (I know Deepmind wrote the orginal paper, but their benchmarks were quite unimpressive). It's plausible that we will have an AlphaGo moment with this scheme.

sdwr 3y ago
Yeah looks very promising. Naively, it multiplies computation time by a factor of 20x though? If they are taking 5x samples per step, and multiple steps per problem. https://imgur.com/a/VbpQZRm https://imgur.com/a/VbpQZRm As this gets explored further, I believe we will start finding out why human minds are constructed the way they are, from the practical/necessity direction. Seems like the next step is farming out subtasks to smaller models, and adding an orthogonal dimension of emotionality to help keep track of state.
- GreedClarifies 3y ago
  I’m sympathetic to the idea of new types of specialized models to assist in this effort. We’re using our one hammer for all problems. In particular, it jumps out that a “ranking model” (different, I think from current ranking models) to judge which paths to take and which nodes to trim would make some level of sense.
- joshka 3y ago
  Not sure if it's relevant, but the OpenAI APIs generally support taking multiple responses in a single API call. I'm unsure what the generalized effect on processing time of that is however. From what I've read, this is sub-linear, so could reasonably be more effective than 20x, and I'd bet there are probably speedups to be had on the model side of this that make the extra time cost negligible.
3y ago

raydiatian 3y ago

> This implementation of Tree of Thoughts is brought to you by Agora, Agora advances Humanity with open source SOTA Multi-Modality AI research! We plan on combating Humanity's grandest root problems like food insecurity, planetary insecurity, and disease, and hopefully death itself. Wow. Lick, don’t sniff, the fresh paint.

pixl97 3y ago
>We plan on combating Humanity's grandest root problems like food insecurity, planetary insecurity, and disease, and hopefully death itself. If everyone is dead, you don't have to worry about death, or any of those other pesky hard to solve problems!
- 082349872349872 3y ago
  > Eyes melt, skin explodes, everybody dead. So immoral, working on the thing can drive you mad. That's what happened to this friend of mine. — JFP

ChrisAlexiuk 3y ago

https://youtu.be/bjnTy2TdmYw https://youtu.be/bjnTy2TdmYw I went through this in a video using the paper's official code - and it worked fairly well! Definitely a great step forward in terms of reasoning tasks - even if it is an expensive step.

xg15 3y ago

> This is an plug in and play version, connect your own models and enjoy superintelligence! Share this repository by clicking on the following buttons! <smiley face> 2023 in a nutshell.

rahimnathwani 3y ago

Another use of ideas from the same paper, but this time to produce lesson plans for an AI tutor: https://github.com/JushBJJ/Mr.-Ranedeer-AI-Tutor/tree/testing https://github.com/JushBJJ/Mr.-Ranedeer-AI-Tutor/tree/testin...

emmanueloga_ 3y ago

Similar in concept to the magi supercomputer? :-p [1] 1: https://aminoapps.com/c/neon-genesis-evangelion/page/item/magi-super-computer-system/123e_w4frIZerZDPNwQYzn7QXwZ7rzZ28 https://aminoapps.com/c/neon-genesis-evangelion/page/item/ma...

Jeff_Brown 3y ago

A claim like "improves reasoning by 70%" is too specific to be accompanied by neither a citation nor a definition.

Imnimo 3y ago
My guess is that the author misunderstands this quote from the paper abstract: >For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%.

flakiness 3y ago

Note that the repo author != the paper author. The research itself [1] seems legit. The paper author also wrote a paper called ReAct [2], which is one of the core components of the langchain framework. * [1] https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2305.10601 * [2] https://arxiv.org/abs/2210.03629 https://arxiv.org/abs/2210.03629

thawab 3y ago
here is the repo by the paper author: https://github.com/ysymyth/tree-of-thought-llm https://github.com/ysymyth/tree-of-thought-llm
joshka 3y ago
Interestingly a 2 days prior to https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2305.10601, someone released https://arxiv.org/abs/2305.08291 https://arxiv.org/abs/2305.08291 > Large Language Model Guided Tree-of-Thought > In this paper, we introduce the Tree-of-Thought (ToT) framework, a novel approach aimed at improving the problem-solving capabilities of auto-regressive large language models (LLMs). The ToT technique is inspired by the human mind's approach for solving complex reasoning tasks through trial and error. In this process, the human mind explores the solution space through a tree-like thought process, allowing for backtracking when necessary. To implement ToT as a software system, we augment an LLM with additional modules including a prompter agent, a checker module, a memory module, and a ToT controller. In order to solve a given problem, these modules engage in a multi-round conversation with the LLM. The memory module records the conversation and state history of the problem solving process, which allows the system to backtrack to the previous steps of the thought-process and explore other directions from there. To verify the effectiveness of the proposed technique, we implemented a ToT-based solver for the Sudoku Puzzle. Experimental results show that the ToT framework can significantly increase the success rate of Sudoku puzzle solving. Our implementation of the ToT-based Sudoku solver is available on GitHub: I don't recall whether it was this paper, or another that I read that talks about using the LLM's ability to also show the probabilities of each token to measure the validity of the particular completions. However that isn't exposed in the OpenAI chat APIs (GPT-Turbo-3.5 / GPT-4), just the completions APIs (Text-Davinci-003 etc.)