2 ms·
Indeed. Studying vocab is even the example I've used for years on the dual n-back mailing list to explain how a previously valid measure can be rendered meaning
by gwern 13y ago
Indeed. Studying vocab is even the example I've used for years on the dual n-back mailing list to explain how a previously valid measure can be rendered meaningless by training: just memorize vocab and boost your scores on the vocab subtest. Of course, if you took a bunch of people who studied SAT/GRE words and looked at the factorization on a battery of tests, you'd find that the g-loading of the vocab subtest had gone way down, but that can't be done in this sort of case and so all you're left with is a meaningless measure.
this was in relation to Moody's critique of the original n-back study: he thought that all the n-back training, which involved tracking moving squares in a grid, was effectively training you to be able to manipulate shapes on a grid better than you would normally, and destroying part of the validity of the matrix tests. And indeed, in the later studies which used more than just matrix tests, the n-back training tends to show more limited gains. It's not officially out yet, but an example of this would be "Adaptive n‐back training does not improve fluid intelligence at the construct level; gains on individual tests suggest training may enhance visuospatial processing", Colom et al 2013.