4 ms·

The part that was interesting for me was: > So if you positively encourage it to “think out loud”, it stands a much better chance of being able to use its dedu

The part that was interesting for me was: > So if you positively encourage it to “think out loud”, it stands a much better chance of being able to use its deduction and reasoning capabilities (which are quite significant).

mirekrusin 3y ago
It's well known prompt engineering technique to ask LLMs to articulate each step. One way to think about it is that LLM's output is pure function on input. You need to chain them to arrive at final answer. Each step can't escape its comprehension capability. One step's output appends to next step's input so you can ask it to do it itself.
- seydor 3y ago
  i wonder if this is a result of the GPT part (i.e. the distribution of 'next tokens' is such that it tends to be logically coherent), or the RLHF part (i.e. it tends to steer the GPT to logical coherence)
  - nullc 3y ago
    The just internet trained models don't think out loud, so I assume it was RLHFed conditioned to that behavior since it makes it able to solve much more complicated questions, though it's sometimes irritating to read.
  - mirekrusin 3y ago
    Both and none at the same time. Both contribute to overall performance. None are directly related to solving it in general. You can't solve it in general because you can't have answer to arbitrarily complex question with a single pass on limited data. However I do think that current methods are extremely inefficient. There is a lot of nonsense, noise and duplication in training data. It can be compressed to be orders of magnitude smaller. Imagine having dense, close to optimal way to kick off learning with logic, reasoning, things like set theory, category theory etc. so it is later reused when learning ie. programming language - where learning programming language is also efficient, based on cleaned up specification, standard library learned from code of standard library itself, possibly access to compiler/interpreter for feedback. I mean quality of data slider pushed to the limit. But this single pass, void of inner dialogue approach will always require articulating steps, no matter how good it is quality wise.
jstx1 3y ago
> We realized (...) that we can dramatically improve its ability in math and tutoring if we allow the AI to think before it speaks. This is from Sal Khan's TED talk on how they're integrating ChatGPT into Khan Academy - https://youtu.be/hJP5GqnTrNo?t=745 https://youtu.be/hJP5GqnTrNo?t=745

mirekrusin 3y ago

It's well known prompt engineering technique to ask LLMs to articulate each step. One way to think about it is that LLM's output is pure function on input. You need to chain them to arrive at final answer. Each step can't escape its comprehension capability. One step's output appends to next step's input so you can ask it to do it itself.

seydor 3y ago
i wonder if this is a result of the GPT part (i.e. the distribution of 'next tokens' is such that it tends to be logically coherent), or the RLHF part (i.e. it tends to steer the GPT to logical coherence)
- nullc 3y ago
  The just internet trained models don't think out loud, so I assume it was RLHFed conditioned to that behavior since it makes it able to solve much more complicated questions, though it's sometimes irritating to read.
- mirekrusin 3y ago
  Both and none at the same time. Both contribute to overall performance. None are directly related to solving it in general. You can't solve it in general because you can't have answer to arbitrarily complex question with a single pass on limited data. However I do think that current methods are extremely inefficient. There is a lot of nonsense, noise and duplication in training data. It can be compressed to be orders of magnitude smaller. Imagine having dense, close to optimal way to kick off learning with logic, reasoning, things like set theory, category theory etc. so it is later reused when learning ie. programming language - where learning programming language is also efficient, based on cleaned up specification, standard library learned from code of standard library itself, possibly access to compiler/interpreter for feedback. I mean quality of data slider pushed to the limit. But this single pass, void of inner dialogue approach will always require articulating steps, no matter how good it is quality wise.

jstx1 3y ago

> We realized (...) that we can dramatically improve its ability in math and tutoring if we allow the AI to think before it speaks. This is from Sal Khan's TED talk on how they're integrating ChatGPT into Khan Academy - https://youtu.be/hJP5GqnTrNo?t=745 https://youtu.be/hJP5GqnTrNo?t=745