4 ms·

It might not matter what is on the exams. SAT could just be testing for test-taking ability. I'd assume test-taking ability would be a good predictor of colle

by labster 11y ago

It might not matter what is on the exams. SAT could just be testing for test-taking ability. I'd assume test-taking ability would be a good predictor of college performance too, as exams are typically 40-90% of college grades in my experience. Is there any data out there which tries to measure test skills independently?

threatofrain 11y ago
What do you propose are the factors behind test-taking ability between-subjects? I would propose that a major contributor is g, or general mental ability, and I propose it is for that reason that the SAT II's have a non-trivial relationship to college performance, more than 4 years of high school grades.
Spooky23 11y ago
They don't test for test-taking ability, they test for wherewithal to enroll in test prep classes. I can't speak to the current state of affairs, but in the mid 90s, taking a 5 week Kaplan course in my peer group raised scores from the 1150-1300 range to the 1300-1500 range. That meant the difference between average state school and high probability of hitting a "reach" school. Many of the ultra achievers on these tests start prep at age 10 or younger.
- threatofrain 11y ago
  That just means that we should think about accessibility issues, but for whatever reason, teachers as a whole, despite intimate access for 4 years to students, are still less predictive of future performance than a test developed from afar.
- thedufer 11y ago
  On my phone so I don't have the paper handy, but all available research shows no more than 50 point gains on SAT from test prep.
  - seanmcdirmid 11y ago
    But is that Chinese prep? Cram school is quite different from what we call prep in the west.
    - thedufer 11y ago
      Well, no, since we're talking about the SAT it is pretty US-centric. Unfortunately, Duke has broken broken the link to the paper (the entire subdomain it's on appears to have gotten the redirection treatment), but [here](http://marginalrevolution.com/marginalrevolution/2014/03/the-sat-test-prep-income-and-race.html http://marginalrevolution.com/marginalrevolution/2014/03/the...) is an article about it.
cbr 11y ago
I'm imagining a test-taking skills test. You have multiple choice questions, drawn from an incredible range of domains that people couldn't possibly be familiar with most of. I'd love to see how predictive this would be!
- ZeroGravitas 11y ago
  There's a multiple choice quiz in which the questions and answers are all basically gibberish yet you can work out the answers based on "test wiseness": https://edtechdev.wordpress.com/2007/10/11/test-wiseness-writing-better-multiple-choice-quiz-questions/ https://edtechdev.wordpress.com/2007/10/11/test-wiseness-wri...

by labster 11y ago

threatofrain 11y ago

What do you propose are the factors behind test-taking ability between-subjects? I would propose that a major contributor is g, or general mental ability, and I propose it is for that reason that the SAT II's have a non-trivial relationship to college performance, more than 4 years of high school grades.

Spooky23 11y ago

They don't test for test-taking ability, they test for wherewithal to enroll in test prep classes. I can't speak to the current state of affairs, but in the mid 90s, taking a 5 week Kaplan course in my peer group raised scores from the 1150-1300 range to the 1300-1500 range. That meant the difference between average state school and high probability of hitting a "reach" school. Many of the ultra achievers on these tests start prep at age 10 or younger.

threatofrain 11y ago
That just means that we should think about accessibility issues, but for whatever reason, teachers as a whole, despite intimate access for 4 years to students, are still less predictive of future performance than a test developed from afar.
thedufer 11y ago
On my phone so I don't have the paper handy, but all available research shows no more than 50 point gains on SAT from test prep.
- seanmcdirmid 11y ago
  But is that Chinese prep? Cram school is quite different from what we call prep in the west.
  - thedufer 11y ago
    Well, no, since we're talking about the SAT it is pretty US-centric. Unfortunately, Duke has broken broken the link to the paper (the entire subdomain it's on appears to have gotten the redirection treatment), but [here](http://marginalrevolution.com/marginalrevolution/2014/03/the-sat-test-prep-income-and-race.html http://marginalrevolution.com/marginalrevolution/2014/03/the...) is an article about it.

cbr 11y ago

I'm imagining a test-taking skills test. You have multiple choice questions, drawn from an incredible range of domains that people couldn't possibly be familiar with most of. I'd love to see how predictive this would be!

ZeroGravitas 11y ago
There's a multiple choice quiz in which the questions and answers are all basically gibberish yet you can work out the answers based on "test wiseness": https://edtechdev.wordpress.com/2007/10/11/test-wiseness-writing-better-multiple-choice-quiz-questions/ https://edtechdev.wordpress.com/2007/10/11/test-wiseness-wri...