5 ms·

There is a leap in this logic- going from the fact that SAT is solvable by a LLM to the conclusion that the SAT is useless. Questions about addition can be solv

There is a leap in this logic- going from the fact that SAT is solvable by a LLM to the conclusion that the SAT is useless. Questions about addition can be solved using calculators, does that mean that teaching addition is useless? Also there is an underlying assumption that memorization is a useless skill, which I'd argue against, as in today's world most innovation is done by identifying patterns between disciplines, and memorization is an important part of recognising patterns

ekianjo 4y ago
not so much memorization as decomposition of numerous examples into logic based knowledge. As a simple example, knowing volumes of words is not as useful as knowing etymology since etymology brings you a deeper structure.
- shreyshnaccount 4y ago
  I'd say that's subjective, depending on context and individuals. Sure etymology might be amazing to know for a lit major or author, but for someone who wants to just be able to write clearly, knowing a lot of words is useful.
hgsgm 4y ago
Well it shows that you can do well on SAT with basic word association without understanding and explaining the ideas of the passage. The "Joe Bloggs" theory has known this for ages.
- shreyshnaccount 4y ago
  Yeah but that also says nothing about how useful or useless that is
sublinear 4y ago
You're correct that memorization and pattern recognition are useful, but they're not supposed to be the majority of the test! A good test should also include reasoning skills and comprehension. Having reasoning skills means not all necessary information can be derived from the test itself, the necessary information isn't always well defined, yet a correct answer is produced. Demonstrating comprehension requires the test taker to write out a model of what they think they know in their own words. Undoubtedly the answer will be at least slightly wrong which is why subjective expertise is required to evaluate the level of comprehension. To me it seems the quality of the tests has deteriorated due to cost cutting, not because AI has become so much more sophisticated. Subjectivity is usually dismissed these days as a roll of the dice, but it's vital to fitting the real world full of humans living in it. Humans must be the gatekeepers, not to fight off AI, but by the very definition of what it means to be convincing. Answers to real world questions solve many more constraints than just the concrete subject matter. The whole point to updating a test on a well understood topic should not merely be to prevent "replay attack" cheating, but to conform to the new subjectivity on the topic.

ekianjo 4y ago

not so much memorization as decomposition of numerous examples into logic based knowledge. As a simple example, knowing volumes of words is not as useful as knowing etymology since etymology brings you a deeper structure.

shreyshnaccount 4y ago
I'd say that's subjective, depending on context and individuals. Sure etymology might be amazing to know for a lit major or author, but for someone who wants to just be able to write clearly, knowing a lot of words is useful.

hgsgm 4y ago

Well it shows that you can do well on SAT with basic word association without understanding and explaining the ideas of the passage. The "Joe Bloggs" theory has known this for ages.

shreyshnaccount 4y ago
Yeah but that also says nothing about how useful or useless that is

sublinear 4y ago

You're correct that memorization and pattern recognition are useful, but they're not supposed to be the majority of the test! A good test should also include reasoning skills and comprehension. Having reasoning skills means not all necessary information can be derived from the test itself, the necessary information isn't always well defined, yet a correct answer is produced. Demonstrating comprehension requires the test taker to write out a model of what they think they know in their own words. Undoubtedly the answer will be at least slightly wrong which is why subjective expertise is required to evaluate the level of comprehension. To me it seems the quality of the tests has deteriorated due to cost cutting, not because AI has become so much more sophisticated. Subjectivity is usually dismissed these days as a roll of the dice, but it's vital to fitting the real world full of humans living in it. Humans must be the gatekeepers, not to fight off AI, but by the very definition of what it means to be convincing. Answers to real world questions solve many more constraints than just the concrete subject matter. The whole point to updating a test on a well understood topic should not merely be to prevent "replay attack" cheating, but to conform to the new subjectivity on the topic.