3 ms·
This is true but you have to also consider the autoregressive component. In your example, it's 50 steps per iteration of the model, where the model is executed
by radarsat1 3y ago
This is true but you have to also consider the autoregressive component. In your example, it's 50 steps per iteration of the model, where the model is executed once for each token in the output.
So practically speaking it's a bit more complicated to calculate how much the model can "think". Of course once a token is output it is committed to that (in the most basic scenario), but that doesn't mean it is not still "thinking" as it produces subsequent tokens.
> perhaps a notepad
Exactly, the context and previously output tokens can be considered such a notepad since they are input for the next steps of the model.
- Closi 3y agoAgreed - also prompt engineering encourages LLM's to do this too (i.e. asking the LLM to explain the steps it will take to solve an answer, prior to answering - e.g. Zero-Shot CoT 'Let's think step by step')
- carlmr 3y agoTeachers asking students to show their work are human prompt engineers.
- anon291 3y agoSo part of my general issue with this kind of thinking is that, if we take this as the main means of creating complexity, then shorter prompts are worse for reasoning than longer ones, because longer ones automatically give the model more 'space', to think. Now, I realize that the research community knows this, but I like papers like this that explicitly seek ways to enable the model to 'breathe' a bit,.
- danielmarkbruce 3y agoThis doesn't make sense. The responses can be long.