3 ms·

> Chain-of-Thought simply let's the model see it's own output as an input and therefore build upon that. It lets them break a complex problem down into a series

> Chain-of-Thought simply let's the model see it's own output as an input and therefore build upon that. It lets them break a complex problem down into a series of simpler steps which they can see (output becomes input) and build upon. Sure, but why does that make the model more effective? Are you sure it's "breaking the problem down into simpler steps", or is it just appearing to do so? How does this breakdown happen in the model, exactly? If we can better understand the mechanics involved, then maybe this process can be built into a new model that can achieve the same results more efficiently instead of as a recursive process that runs the model more than once.

HarHarVeryFunny 3y ago
You can think of an LLM as a production line - feed a series of tokens in, and they get embedded and then processed through the system one step at a time though however many transformer layers the model has (undisclosed for most recent models, but GPT-3 has 96). Those fixed 96 (or whatever) steps of processing limit the complexity of what the model can do, so it will fail if the task is too complicated unless it it breaks it down into simpler steps that each can be done well with that depth (96 steps) of processing. It's not just appearing to do so - with chain-of-thought prompting you are literally telling it to "think step by step" as part of the prompt, so this is what it outputs. You could also tell it to generate a step by step plan, then elaborate on each of those steps. I don't think we can say exactly how it is deciding to break a task into steps, anymore than we can in general say exactly how these LLMs are working, but intuitively it's similar to how we think and talk (which is what the LLM is trained on) - a good speaker/writer will introduce a complex topic as a top-down decomposition.
phailhaus 3y ago
> Sure, but why does that make the model more effective? Because the model can't "think". There is no "reasoning" that goes into generating an answer. So for complex problems that require multiple steps of reasoning, the model needs to persist those intermediate steps in order to be able to build up to a solution.
Bjartr 3y ago
> Sure, but why does that make the model more effective? If you look something up (on the internet, in a book, asking people directly) and receive two answers, one which describes the steps used to arrive at the answer, and one which doesn't, on average which one is more likely to be the correct answer? As a prediction machine, an LLM is more constrained by what is likely to appear after a chain of reasoning.
- naasking 3y ago
  You're just repeating the previous explanation with different words, so that's not really satisfactory. A mechanistic demonstration of how step by step reasoning tends to constrain the space of solutions to ones that are more likely to be correct would be an actual explanation, until then this is a just-so story.

HarHarVeryFunny 3y ago

You can think of an LLM as a production line - feed a series of tokens in, and they get embedded and then processed through the system one step at a time though however many transformer layers the model has (undisclosed for most recent models, but GPT-3 has 96). Those fixed 96 (or whatever) steps of processing limit the complexity of what the model can do, so it will fail if the task is too complicated unless it it breaks it down into simpler steps that each can be done well with that depth (96 steps) of processing. It's not just appearing to do so - with chain-of-thought prompting you are literally telling it to "think step by step" as part of the prompt, so this is what it outputs. You could also tell it to generate a step by step plan, then elaborate on each of those steps. I don't think we can say exactly how it is deciding to break a task into steps, anymore than we can in general say exactly how these LLMs are working, but intuitively it's similar to how we think and talk (which is what the LLM is trained on) - a good speaker/writer will introduce a complex topic as a top-down decomposition.

phailhaus 3y ago

> Sure, but why does that make the model more effective? Because the model can't "think". There is no "reasoning" that goes into generating an answer. So for complex problems that require multiple steps of reasoning, the model needs to persist those intermediate steps in order to be able to build up to a solution.

Bjartr 3y ago

> Sure, but why does that make the model more effective? If you look something up (on the internet, in a book, asking people directly) and receive two answers, one which describes the steps used to arrive at the answer, and one which doesn't, on average which one is more likely to be the correct answer? As a prediction machine, an LLM is more constrained by what is likely to appear after a chain of reasoning.

naasking 3y ago
You're just repeating the previous explanation with different words, so that's not really satisfactory. A mechanistic demonstration of how step by step reasoning tends to constrain the space of solutions to ones that are more likely to be correct would be an actual explanation, until then this is a just-so story.