2 ms·
I think the part of overconfidence is actually important here. Looking at the paper Figure 2 shows the number of questions. For all sessions with GPT students a
by godelski 2y ago
I think the part of overconfidence is actually important here. Looking at the paper Figure 2 shows the number of questions. For all sessions with GPT students averaged 2 messages per problem. Interestingly 20% isn't restating the problem or asking for the answer. Looks like 50-60% of tutor questions are asking for answer or restating problem and they ask more questions.
In my experience, usually where GPT and other LLMs get things wrong is in the steps. I see people are frequently over confident about the results and I've long thought this was a big part of it that the answer may be right but steps to get there are wrong. I see this a lot with river crossing problems. People will often show a prompt that solves it (half the time there's information leakage) and a good portion of those will have the logic errors. But people just look at the answer (btw, this is the reason I claim LLMs don't reason. It's not about getting the answer wrong, it's about the logical steps. That's the real evaluation!)
Personally I think this is good evidence of over fitting (memorization).
I also think this is why you should be careful when using it to code. It's the details that matter a lot. Since LLMs are aggregators I'd like to remind everyone that the average coder and average code is terrible. Hell, even good programmers often suck. Code is hard! Whenever I try using LLMs to code I find that I write lines faster but I end up spending more time debugging and prompt engineering than if I just read the docs and did it myself (this also has the added benefit of the struggle making me remember more). So I'll use it to write things I don't really care about but otherwise I'm not getting how people are finding it so helpful (yes, I read my comment). But I suspect many people are similar to the students in the study.