30 ms·

How are the SAT/ACT not objective tests? Are they not objective measures of how you perform on the SAT/ACT? There are abundant low cost practice guides and s

by _brnu 5y ago

How are the SAT/ACT not objective tests? Are they not objective measures of how you perform on the SAT/ACT? There are abundant low cost practice guides and study materials. I came from a very poor family and with some practice at home scored very well on the SAT. It was one of the few tools I had to attract college admissions boards. I feel like taking that away will severely disadvantage poor kids who are otherwise bright with strong work ethics.

dastbe 5y ago
put a big IMO/IMU around all of this, but the SAT/ACT isn't objective in the sense that studying for them is something you can do to improve your score and your prospects. However, it is also transparent in how you do that: you take a course, get study books, and/or take practice tests and there are programs out that have been trying to help disadvantaged students get access to these tools. its also much more cost effective compared to the other solutions like going to a private school (difficult, can be costly without aid) or getting into expensive niche sports. so imo getting rid of standardized tests does get rid of a subjective rating of an applicant which is good in theory, but when you leave in all of the other subjective ratings that are much more difficult for disadvantaged students to do well on it who is it actually benefiting?
- sokoloff 5y ago
  It’s well-known how to train for foot races and that training will improve your natural/untrained performance. That doesn’t make a timed foot race a subjective measure, IMO.
  - dastbe 5y ago
    subjective/objective are probably not the right adjectives here, but to use your example I would say that a timed foot race is an objective measure of how well you do in timed foot races, but is not an objective measure of your innate athletic ability because of training/coaching and you have to factor those to attempt to back into physical ability. the same is true for the SAT, in that people treat it like an intelligence/general knowledge test when you have to factor in the specific training that person has had for the SAT itself.
    - OnlineGladiator 5y ago
      So what do you propose is more objective (or whatever qualifier you prefer) than the SAT? Pointing out shortcomings of standardized tests is one thing. Advocating for eliminating what seems to be our most equal/standardized/objective/whatever measurement without a replacement seems like a bad idea to me.
      - afiori 5y ago
        Not really about SATs, but about a different direction to take the concept of college and higher education. https://slatestarcodex.com/2015/06/06/against-tulip-subsidies/ https://slatestarcodex.com/2015/06/06/against-tulip-subsidie...
        pgwhalen 5y ago
        That's an interesting read, and I personally am a fan of Scott's somewhat radical views on college education in America, but surely you recognize that you are directly side-stepping the question? The problem at hand here is not about "how to decide who to get into undergrad", it's "how to decide who to train for the 'elite' jobs in society".
      - dastbe 5y ago
        if you read my first comment i never argued for getting rid of the sat, though they should get rid of the usurious prices for tests and sending your scores out. in fact, i argued that of all the subjective tests the sat has the most legible and accessible path to maximizing your score compared to all the other subjective things colleges look at.
    - PeterisP 5y ago
      A timed foot race is a quite reasonable measurement of athletic capability - like, it's not a perfect one and a more thorough and diverse evaluation would be better, but if you don't have a better evaluation option then it's obvious that it has some information; if you'd take a hundred random people and split them into two halves based on foot race times, then that would be a far better separation than one based on randomness or any factors that measure athleticness even less. I mean, it's okay to use the best imperfect metric you have available; so acknowledging "the metric is not fully correlated and depends on other factors" does not mean anything, the only argument that has weight is "hey, this other metric has better correlation with The True Thing and less bias, use that instead"; if the remaining alternative metrics are even less objective and more biased than SAT/ACT tests (which IMHO is the case), then removing SAT tests won't and can't improve things, despite these tests being flawed.
- pgwhalen 5y ago
  While it's true that one can improve their standardized test scores by studying or taking a course, and the ability to do those things is correlated to a student's socio-economic situation; if you label standardized tests as subjective, then for me it follows that there is no objective measure of a student period. Which begs the question, who exactly do you admit to your school, and why?
- WalterBright 5y ago
  I haunt thrift stores, and almost always there are SAT/ACT prep books for sale for a couple dollars. You can probably get them for free from your local library. In my not-so-humble opinion, however, the best way to prep for the verbal SAT is to read books. Novels are fine. For the math, prep by taking all the math classes your school offers, and make an effort to learn all the material. Do that, and you won't need prep classes, and you'll also be well prepared for college.
jedberg 5y ago
> I feel like taking that away will severely disadvantage poor kids who are otherwise bright with strong work ethics. They can still take the test and get a high score. > How are the SAT/ACT not objective tests? Are they not objective measures of how you perform on the SAT/ACT? Most of the studies on this show that all of the cultural references in the reading passages and math word problems were things that wealthy people would have experienced in real life, but poor people would not, so it gave an advantage to wealthy kids. Also a lot of the vocab were words that wealthy kids had heard at home but not poor kids. So while it was supposed to be a test of intelligence, it is really a test of how wealthy you were growing up. Edit: Here is the main study from 2003 from Harvard: https://meridian.allenpress.com/her/article-abstract/73/1/1/31807/Correcting-the-SAT-s-Ethnic-and-Social-Class-Bias https://meridian.allenpress.com/her/article-abstract/73/1/1/... Here is a writeup of someone who replicated it with more recent test scores: https://www.insidehighered.com/news/2010/06/21/new-evidence-racial-bias-sat https://www.insidehighered.com/news/2010/06/21/new-evidence-...
- hcurtiss 5y ago
  No, it's a measure of how well you perform on the subjects tested by the SAT. There are many things that correlate with high SAT scores (e.g., grades, being Asian?), but it is definitely true that you can prep for the SAT without any meaningful amount of money. I don't believe that anybody has ever alleged that SAT scores are a proxy for intelligence, though I suspect there is a decent correlation. In my mind it is a useful, objective, universal measure of how an individual student is able to prepare for and execute on a difficult task.
  - rayiner 5y ago
    SAT's are closely correlated with IQ tests: https://slate.com/technology/2014/04/what-do-sat-and-iq-tests-measure-general-intelligence-predicts-school-and-life-success.html https://slate.com/technology/2014/04/what-do-sat-and-iq-test...
  - jedberg 5y ago
    > I don't believe that anybody has ever alleged that SAT scores are a proxy for intelligence MENSA accepts it for admission to their high IQ club.
xapata 5y ago
There is some truth to the argument that standardized tests are a flawed proxy for the combination of intelligence and motivation to learn that one wants in a student, and that the test may be biased towards some cultures that have a tradition of preparing their children for those standardized tests, whether directly or indirectly. Despite the relative low cost of practice, it is not free. Also, each attempt of the test has a cost. Still, I'm a fan of standardized tests. If there's a problem, we should try to improve the tests rather than doing away with them.
- usaar333 5y ago
  I'm not aware of any significant cultural test bias (which I'll define as differential prediction).
  - xapata 5y ago
    Statistically significant, such that it created some academic literature and enough cover for Harvard to make this new policy. Practically significant, maybe not. Here's a comment from a different branch of this thread: > The findings in general is that much of the disparity between minorities and whites goes away for harder analogies that are more technical or scientific in nature, in fact contrary to what I expected, the recommendation of the study is to simply eliminate the bottom third of verbal questions (ranked by difficulty) involving analogies or sentence completion, as those are the questions that are most likely to involve cultural bias. https://news.ycombinator.com/item?id=29599455#29600100 https://news.ycombinator.com/item?id=29599455#29600100
    - usaar333 5y ago
      First off, the SAT has gone through many revisions in the last 20 years so it's unclear this still applies. The SAT doesn't even have analogies any more, perhaps as a result of this research. Secondly, that doesn't address differential prediction abilities. It addresses a concerning aspect of the verbal test, but is somewhat irrelevant when your goal is to predict college success. I'm not aware of differences on that metric.

by _brnu 5y ago

dastbe 5y ago

put a big IMO/IMU around all of this, but the SAT/ACT isn't objective in the sense that studying for them is something you can do to improve your score and your prospects. However, it is also transparent in how you do that: you take a course, get study books, and/or take practice tests and there are programs out that have been trying to help disadvantaged students get access to these tools. its also much more cost effective compared to the other solutions like going to a private school (difficult, can be costly without aid) or getting into expensive niche sports. so imo getting rid of standardized tests does get rid of a subjective rating of an applicant which is good in theory, but when you leave in all of the other subjective ratings that are much more difficult for disadvantaged students to do well on it who is it actually benefiting?

sokoloff 5y ago
It’s well-known how to train for foot races and that training will improve your natural/untrained performance. That doesn’t make a timed foot race a subjective measure, IMO.
- dastbe 5y ago
  subjective/objective are probably not the right adjectives here, but to use your example I would say that a timed foot race is an objective measure of how well you do in timed foot races, but is not an objective measure of your innate athletic ability because of training/coaching and you have to factor those to attempt to back into physical ability. the same is true for the SAT, in that people treat it like an intelligence/general knowledge test when you have to factor in the specific training that person has had for the SAT itself.
  - OnlineGladiator 5y ago
    So what do you propose is more objective (or whatever qualifier you prefer) than the SAT? Pointing out shortcomings of standardized tests is one thing. Advocating for eliminating what seems to be our most equal/standardized/objective/whatever measurement without a replacement seems like a bad idea to me.
    - afiori 5y ago
      Not really about SATs, but about a different direction to take the concept of college and higher education. https://slatestarcodex.com/2015/06/06/against-tulip-subsidies/ https://slatestarcodex.com/2015/06/06/against-tulip-subsidie...
      - pgwhalen 5y ago
        That's an interesting read, and I personally am a fan of Scott's somewhat radical views on college education in America, but surely you recognize that you are directly side-stepping the question? The problem at hand here is not about "how to decide who to get into undergrad", it's "how to decide who to train for the 'elite' jobs in society".
    - dastbe 5y ago
      if you read my first comment i never argued for getting rid of the sat, though they should get rid of the usurious prices for tests and sending your scores out. in fact, i argued that of all the subjective tests the sat has the most legible and accessible path to maximizing your score compared to all the other subjective things colleges look at.
  - PeterisP 5y ago
    A timed foot race is a quite reasonable measurement of athletic capability - like, it's not a perfect one and a more thorough and diverse evaluation would be better, but if you don't have a better evaluation option then it's obvious that it has some information; if you'd take a hundred random people and split them into two halves based on foot race times, then that would be a far better separation than one based on randomness or any factors that measure athleticness even less. I mean, it's okay to use the best imperfect metric you have available; so acknowledging "the metric is not fully correlated and depends on other factors" does not mean anything, the only argument that has weight is "hey, this other metric has better correlation with The True Thing and less bias, use that instead"; if the remaining alternative metrics are even less objective and more biased than SAT/ACT tests (which IMHO is the case), then removing SAT tests won't and can't improve things, despite these tests being flawed.
pgwhalen 5y ago
While it's true that one can improve their standardized test scores by studying or taking a course, and the ability to do those things is correlated to a student's socio-economic situation; if you label standardized tests as subjective, then for me it follows that there is no objective measure of a student period. Which begs the question, who exactly do you admit to your school, and why?
WalterBright 5y ago
I haunt thrift stores, and almost always there are SAT/ACT prep books for sale for a couple dollars. You can probably get them for free from your local library. In my not-so-humble opinion, however, the best way to prep for the verbal SAT is to read books. Novels are fine. For the math, prep by taking all the math classes your school offers, and make an effort to learn all the material. Do that, and you won't need prep classes, and you'll also be well prepared for college.

jedberg 5y ago

> I feel like taking that away will severely disadvantage poor kids who are otherwise bright with strong work ethics. They can still take the test and get a high score. > How are the SAT/ACT not objective tests? Are they not objective measures of how you perform on the SAT/ACT? Most of the studies on this show that all of the cultural references in the reading passages and math word problems were things that wealthy people would have experienced in real life, but poor people would not, so it gave an advantage to wealthy kids. Also a lot of the vocab were words that wealthy kids had heard at home but not poor kids. So while it was supposed to be a test of intelligence, it is really a test of how wealthy you were growing up. Edit: Here is the main study from 2003 from Harvard: https://meridian.allenpress.com/her/article-abstract/73/1/1/31807/Correcting-the-SAT-s-Ethnic-and-Social-Class-Bias https://meridian.allenpress.com/her/article-abstract/73/1/1/... Here is a writeup of someone who replicated it with more recent test scores: https://www.insidehighered.com/news/2010/06/21/new-evidence-racial-bias-sat https://www.insidehighered.com/news/2010/06/21/new-evidence-...

hcurtiss 5y ago
No, it's a measure of how well you perform on the subjects tested by the SAT. There are many things that correlate with high SAT scores (e.g., grades, being Asian?), but it is definitely true that you can prep for the SAT without any meaningful amount of money. I don't believe that anybody has ever alleged that SAT scores are a proxy for intelligence, though I suspect there is a decent correlation. In my mind it is a useful, objective, universal measure of how an individual student is able to prepare for and execute on a difficult task.
- rayiner 5y ago
  SAT's are closely correlated with IQ tests: https://slate.com/technology/2014/04/what-do-sat-and-iq-tests-measure-general-intelligence-predicts-school-and-life-success.html https://slate.com/technology/2014/04/what-do-sat-and-iq-test...
- jedberg 5y ago
  > I don't believe that anybody has ever alleged that SAT scores are a proxy for intelligence MENSA accepts it for admission to their high IQ club.

xapata 5y ago

There is some truth to the argument that standardized tests are a flawed proxy for the combination of intelligence and motivation to learn that one wants in a student, and that the test may be biased towards some cultures that have a tradition of preparing their children for those standardized tests, whether directly or indirectly. Despite the relative low cost of practice, it is not free. Also, each attempt of the test has a cost. Still, I'm a fan of standardized tests. If there's a problem, we should try to improve the tests rather than doing away with them.

usaar333 5y ago
I'm not aware of any significant cultural test bias (which I'll define as differential prediction).
- xapata 5y ago
  Statistically significant, such that it created some academic literature and enough cover for Harvard to make this new policy. Practically significant, maybe not. Here's a comment from a different branch of this thread: > The findings in general is that much of the disparity between minorities and whites goes away for harder analogies that are more technical or scientific in nature, in fact contrary to what I expected, the recommendation of the study is to simply eliminate the bottom third of verbal questions (ranked by difficulty) involving analogies or sentence completion, as those are the questions that are most likely to involve cultural bias. https://news.ycombinator.com/item?id=29599455#29600100 https://news.ycombinator.com/item?id=29599455#29600100
  - usaar333 5y ago
    First off, the SAT has gone through many revisions in the last 20 years so it's unclear this still applies. The SAT doesn't even have analogies any more, perhaps as a result of this research. Secondly, that doesn't address differential prediction abilities. It addresses a concerning aspect of the verbal test, but is somewhat irrelevant when your goal is to predict college success. I'm not aware of differences on that metric.