You're correct that memorization and pattern recognition are useful, but they're not supposed to be the majority of the test! A good test should also include r

You're correct that memorization and pattern recognition are useful, but they're not supposed to be the majority of the test! A good test should also include reasoning skills and comprehension. Having reasoning skills means not all necessary information can be derived from the test itself, the necessary information isn't always well defined, yet a correct answer is produced. Demonstrating comprehension requires the test taker to write out a model of what they think they know in their own words. Undoubtedly the answer will be at least slightly wrong which is why subjective expertise is required to evaluate the level of comprehension. To me it seems the quality of the tests has deteriorated due to cost cutting, not because AI has become so much more sophisticated. Subjectivity is usually dismissed these days as a roll of the dice, but it's vital to fitting the real world full of humans living in it. Humans must be the gatekeepers, not to fight off AI, but by the very definition of what it means to be convincing. Answers to real world questions solve many more constraints than just the concrete subject matter. The whole point to updating a test on a well understood topic should not merely be to prevent "replay attack" cheating, but to conform to the new subjectivity on the topic.