3 ms·
You did write that, but you first wrote that GPT-4's performance says more about the task than about its capabilities. Which is the exact same pattern that has
by p-e-w 3y ago
You did write that, but you first wrote that GPT-4's performance says more about the task than about its capabilities.
Which is the exact same pattern that has been used to "deal with" advances in AI since forever.
And Caplan is a university professor, not a random person. He's not an amateur at giving tests. It's literally (part of) his job. So what happened here is not at all analogous to your chess examples.
- palata 3y agoI would say that the OP is entitled to share their opinion. Which I tend to agree with. Also I would question if the job of a university professor is really to give tests, or rather to teach and conduct research. Tests are just an annoyance (that is deemed necessary). Because one passes the university tests does not mean they necessarily understand the topic very deeply, I think it's pretty obvious. That's actually an interesting point: university students can totally pass exams by "mimicking": not understanding the topic, but solving the exam problems in a way similar to the exercises they learned beforehand. I argue that the test is not perfect at discriminating those who understand from those who mimick impressively well.
- ruszki 3y ago> the test is not perfect at discriminating those who understand from those who mimick impressively well. They discriminate all the time, just in the wrong direction. There are a lot of exams which are written in a way which counts on this mimicking, but in a way which makes tests way more difficult even if you understand the topic. It’s the same problem with remote learning: it’s way easier to cheat on them. In the past 3 years, (except the first semester when COVID broke out) remote exams were way more difficult than before to mitigate this. I had tests which were clearly made this cheating in mind. It was borderline impossible without cheating, they counted on it. At that point, I have no idea why they still tried to enforce any such rules. And still, I know many teachers who clearly don’t understand their topic completely, they just “mimic” it.
- posterboy 3y agoShifting the goal post is a realistic measure on a slippery sloap. Chess isn't that different, inasmuch as memorizing openings and end games is part of the training.
- simonh 3y ago>And Caplan is a university professor, not a random person. He wasn't compared to random people, he was compared to other professors also setting tests, and how GPT-4 fared on their tests. You're blatantly misrepresenting the point being made.
- p-e-w 3y agoI was referring to the chess analogy from the comment I responded to, not to the comparison from the top level comment.