3 ms·
You’re kinda smushing together two different things: reliability and merit. The SAT (which is similar to the TJ test) is highly reliable—meaning that it’s measu
by rayiner 6y ago
You’re kinda smushing together two different things: reliability and merit. The SAT (which is similar to the TJ test) is highly reliable—meaning that it’s measuring something real, and not just producing a “random” result.
https://joss.tcnj.edu/wp-content/uploads/sites/176/2012/04/2007-Stickler-SAT-Critical-Review.pdf https://joss.tcnj.edu/wp-content/uploads/sites/176/2012/04/2...
> How reliable is the SAT? The College Board website reports that a student’s true score is within 30 points of his or her measured score (SEM = 30; College Board, FAQ, 2005). A test with a standard deviation of 100 and a SEM of 30 has an internal consistency coefficient of .91 (α = .91). In a study specifically investigating SAT I score change upon repeated testing, 1,120,563 students in the 1997 college- bound cohort who took the SAT I one to five times in their junior and senior years gained an average of 7 to 13 points on the verbal section and 8 to 16 points on the mathematics section (Nathan & Camara, 1998). Thus, a student who retested at the higher end of both ranges still would not breach the standard error of measurement, indicating high test-retest reliability. Additionally, in a study of the effects of commercial coaching on SAT scores, Powers and Rock (1999) found that the average increase in student scores after intensive coaching was only 30 points on the verbal section and 50 points on the mathematics section, demonstrating that SAT scores remain relatively stable even with intervening practice.
With an SEM of 30, there is less than a 1% chance that a 100 point difference, between a 600 and a 500 on a section is due to random chance. The difference between a median score and and a TJ level score is 250.
Validity, whether the test measures something we care about is a separate issue.