4 ms·

First point is that LLM not necessary "solve", but potentially "memorize", while students can still be able to solve questions they didn't see. Second point is

First point is that LLM not necessary "solve", but potentially "memorize", while students can still be able to solve questions they didn't see. Second point is proposal how benchmarks can be improved.

rafaelero 4y ago
You can't "memorize" from previous tests, otherwise that's exactly what students would do.
- riku_iki 4y ago
  LLM is much more capable in memorizing than human. Human (by my hypothesis) still better in logical reasoning.
  - visarga 4y ago
    You are mistaken, memorisation is trivial, what is hard is to recognise which of the memories to reuse later, and how to adapt it.
    - riku_iki 4y ago
      I would love to learn why do you think so.