I think this is unfair. As a crude comparison, it would be like a mathematician being asked to recite his times tables. Something he could probably do, given a

I think this is unfair. As a crude comparison, it would be like a mathematician being asked to recite his times tables. Something he could probably do, given a bit of time and practice, but so outside the normal day-to-day stuff they normally do that it doesn't tell you much other than they spent time preparing. Which is fine if that's what you're looking to measure. I think the point that the grandparent is trying to make is that if you take a group of developers with a good amount of experience and who everyone agrees are skilled in their field and administer a test with no warning/preparation, and none of them score very well, are they suddenly not very good developers or is the test not a very good predictor of whether someone is a skilled developer?