I think you're right in that most test aren't useful for what they are trying to assess. And that you can't validate anything that you can't trust the integrity

I think you're right in that most test aren't useful for what they are trying to assess. And that you can't validate anything that you can't trust the integrity of. But I think the takeaway is that "critical application" is what constitutes a valuable test. In math, that just means "can you use the procedures of mathematics correctly, resulting in the correct answer?" But in, say, English, memorization has very little value. A valuable test would be in testing whether or not a student can match events in a dramatic story to, say, modern real world events and critically discuss the similarities and differences. And, unfortunately, writing that out clinically and expecting a student to both understand it and be able to do it on command while under pressure of failure is not exactly conducive to positive results. So English tests really shouldn't look anything like Math tests, once you get past the point of memorizing definitions. The same is true of all disciplines. Tests should be generated (and often updated) based on what is currently useful to the discipline in a real and practical sense, at a fundamental level, and then on a more philosophical or exploratory sense once you get into higher learning.