4 ms·
You're assuming that openAI isn't just gonna add the new questions to the training data.
by mupuff1234 2y ago
You're assuming that openAI isn't just gonna add the new questions to the training data.
- Lerc 2y agoTheir methodology shows they can create an infinite variety of problems. This is the same thing as synthetic training data. It doesn't matter if models are trained on the output of the generated data or not. If the model ends up being able to solve newly generated variations, you'd have to admit that it understands the underlying problems.
- mupuff1234 2y agoI think what it shows that it has minimal "understanding" of the problem - otherwise such small variations wouldn't pose a challenge. Training it to handle these specific small variations doesn't change that. It's good in automation, not understanding.
- Lerc 2y agoIf it were a complete failure on variations I would be inclined to agree. Instead it was a 30% drop in performance. I would characterise that as limited understanding.
- sirolimus 2y agoFully agree with this
- deleted 2y ago[deleted]
- cgriswald 2y agoMy guess is that what’s understood isn’t various parts of solving the problem but various aspects of the expected response. I see this more akin to a human faking their way through a conversation.
- CamperBob2 2y agoI see this more akin to a human faking their way through a conversation. That works in English class. Try it in a math class and you'll get a much lower grade than ChatGPT will.
- sirolimus 2y agoExactly. The naivity is just sky-high