3 ms·

I feel there's an important difference : we humans can, to some extent, analyse what we just said (or were about to say) (or what we just typed and didn't press

I feel there's an important difference : we humans can, to some extent, analyse what we just said (or were about to say) (or what we just typed and didn't press send yet!) and conceptualise it as a whole to evaluate its veracity. Such an ability would really elevate current LLMs to a higher level of “intelligence” imo. It is, however, interesting that the “basic” word-spewing ability we now share with LLMs seems vital to the intelligence process. I have no idea how that reckoning phase would be implemented.

mavhc 3y ago
ask it to consider what it just wrote
- jebarker 3y ago
  Right. I think we'll end up seeing models that iterate on larger parts of autoregressive generations before presenting them to the user. It's just more compute, i.e. the bitter lesson.
TeMPOraL 3y ago
My take for almost a year now is: LLMs are like your inner voice. The part where we can "analyse what we just said (or were about to say) (or what we just typed and didn't press send yet!) and conceptualise it as a whole to evaluate its veracity" involves going back-and-forth in yourself. A loop. LLMs are the part inside the loop, and so if you want to get better results, you have to feed the output back to LLM. This is, arguably, part of what the "conversation mode" does, by always including the growing message log in each request - which is why e.g. GPT-4 is good at correcting a mistake once you say, without pointing it out, that there exist a mistake.
- Jensson 3y ago
  > which is why e.g. GPT-4 is good at correcting a mistake once you say, without pointing it out, that there exist a mistake. It is also good at adding a mistake, so this is completely unrelated. It just rolls to get another random. What you should add is that ChatGPT does better when it writes out every step, so what you said previously is still true just that your example doesn't properly support it.
  - TeMPOraL 3y ago
    The "chain of thought" thing where you tell it to "think step by step" is a slightly different thing. That's a single forward pass, where it can only reason based on what it output earlier. But once you reply to it, and the interface feeds the whole conversation back to the LLM, it now can look at the entirety of its previous answer.
    - Jensson 3y ago
      When it does it step by step it can look at every step when it writes down its final answer after the steps. There is no extra magic that happens just because you write something in between. Edit: To expand on that, a humans inner thought would have caught the mistake at that point and corrected it by writing more instead of ending the text. The LLM basically never does that unless you tell it to fix the answer.
      - Martinussen 3y ago
        Every mistake you don't catch as being wrong in your inner thoughts simply become a part of your subjective experience/existence until challenged again later, if not forever.
cout 3y ago
What would happen if an existing LLM were trained with the ability to emit a backspace token? (Would that even be possible?)
- jgtrosh 3y ago
  No idea, but this makes me wonder how we'd even go about getting training data. And it reminds me one time I built a poc editor[0] which saves deleted content so you can get a similar view to manuscripts in which the author strikes out text. Maybe that would be an interesting starting point… [0]: https://github.com/trosh/nohide https://github.com/trosh/nohide

mavhc 3y ago

ask it to consider what it just wrote

jebarker 3y ago
Right. I think we'll end up seeing models that iterate on larger parts of autoregressive generations before presenting them to the user. It's just more compute, i.e. the bitter lesson.

TeMPOraL 3y ago

My take for almost a year now is: LLMs are like your inner voice. The part where we can "analyse what we just said (or were about to say) (or what we just typed and didn't press send yet!) and conceptualise it as a whole to evaluate its veracity" involves going back-and-forth in yourself. A loop. LLMs are the part inside the loop, and so if you want to get better results, you have to feed the output back to LLM. This is, arguably, part of what the "conversation mode" does, by always including the growing message log in each request - which is why e.g. GPT-4 is good at correcting a mistake once you say, without pointing it out, that there exist a mistake.

Jensson 3y ago
> which is why e.g. GPT-4 is good at correcting a mistake once you say, without pointing it out, that there exist a mistake. It is also good at adding a mistake, so this is completely unrelated. It just rolls to get another random. What you should add is that ChatGPT does better when it writes out every step, so what you said previously is still true just that your example doesn't properly support it.
- TeMPOraL 3y ago
  The "chain of thought" thing where you tell it to "think step by step" is a slightly different thing. That's a single forward pass, where it can only reason based on what it output earlier. But once you reply to it, and the interface feeds the whole conversation back to the LLM, it now can look at the entirety of its previous answer.
  - Jensson 3y ago
    When it does it step by step it can look at every step when it writes down its final answer after the steps. There is no extra magic that happens just because you write something in between. Edit: To expand on that, a humans inner thought would have caught the mistake at that point and corrected it by writing more instead of ending the text. The LLM basically never does that unless you tell it to fix the answer.
    - Martinussen 3y ago
      Every mistake you don't catch as being wrong in your inner thoughts simply become a part of your subjective experience/existence until challenged again later, if not forever.

cout 3y ago

What would happen if an existing LLM were trained with the ability to emit a backspace token? (Would that even be possible?)

jgtrosh 3y ago
No idea, but this makes me wonder how we'd even go about getting training data. And it reminds me one time I built a poc editor[0] which saves deleted content so you can get a similar view to manuscripts in which the author strikes out text. Maybe that would be an interesting starting point… [0]: https://github.com/trosh/nohide https://github.com/trosh/nohide