3 ms·

from your reference video (thanks): > 7:36 The exam is structured with multiple choice questions at the start and then some freeform short answers towards the

from your reference video (thanks): > 7:36 The exam is structured with multiple choice questions at the start and then some freeform short answers towards the end. Imagine if you could look up all material in the last 10 years published about astrophysics (or insert any major topic here) near instantly, the question's details are also presented for ingestion, then make prediction what is and is not based on weighing the frequency and potentially the source of information, and then respond in relatively coherent sentences. There is no astrophysics in this.

tiborsaas 4y ago
ChatGPT adds explanations why it picked a certain answer. But you are right in a sense, this checks for existing knowledge, It probably couldn't come up with a solution to an open problem.
- FeepingCreature 4y ago
  Minor correction: ChatGPT does not add explanations why it picked a certain answer; ChatGPT adds explanations that it thinks fit with the answer it has already given. The what predates the why. That's why, if you want ChatGPT to actually think about a problem, you have to get it to give the explanation first, gradually, and only then output the answer. That way, GPT can use the rationale it has given in the prediction of the answer that fits best. If it has to give the answer first, it's already committed. If a human has to answer a question, they follow a process like this: - parse the question - compute a rationale, and evaluate it to gain an answer - output the answer first, because it's the most useful - output the cached rationale. But GPT trains left to right, and it cannot see the first two steps as they're not in the text. From its perspective, most humans produce answers and explanations simultaneously, instantaneously and intuitively. This makes it almost impossible for GPT to learn how to think. It has to rely on the very few samples where humans give the rationale first, and then the answer. And because this is rare, you have to really push GPT to actually use its scant cognitive abilities. ("Let's think about it step by step.")

tiborsaas 4y ago

ChatGPT adds explanations why it picked a certain answer. But you are right in a sense, this checks for existing knowledge, It probably couldn't come up with a solution to an open problem.

FeepingCreature 4y ago
Minor correction: ChatGPT does not add explanations why it picked a certain answer; ChatGPT adds explanations that it thinks fit with the answer it has already given. The what predates the why. That's why, if you want ChatGPT to actually think about a problem, you have to get it to give the explanation first, gradually, and only then output the answer. That way, GPT can use the rationale it has given in the prediction of the answer that fits best. If it has to give the answer first, it's already committed. If a human has to answer a question, they follow a process like this: - parse the question - compute a rationale, and evaluate it to gain an answer - output the answer first, because it's the most useful - output the cached rationale. But GPT trains left to right, and it cannot see the first two steps as they're not in the text. From its perspective, most humans produce answers and explanations simultaneously, instantaneously and intuitively. This makes it almost impossible for GPT to learn how to think. It has to rely on the very few samples where humans give the rationale first, and then the answer. And because this is rare, you have to really push GPT to actually use its scant cognitive abilities. ("Let's think about it step by step.")