6 ms·

SAT: Getting the lowest score possible

109 points by solipsist 14y ago

deleted 14y ago
[deleted]
maeon3 14y ago
The SAT is an important measure of a humans intelligence and performance in the real world, that's why the best companies always filter candidates and select the ones with highest SAT scores. It's a good thing we have tests like these and pay large sums of money to the people who maintain it. Otherwise interviewing might be totally screwed up and completely fail at its intended purpose in this country.
- hkmurakami 14y ago
  The SAT is good for measuring how good one is at taking the SAT, which in turn is an indicator of how much the test taker took preparing for the exam seriously, which is an indirect indicator of the person's work ethic and attitude towards putting in effort for a means to an end (yes, despite the pointlessness of this exam). It does do a decent job at finding such people who strive to excel within the limits of a (flawed) system, whether or not they realize the pointlessness of said structure. If an institution is looking for an indicator for such people, the SAT is an acceptable tool. Of course, what it's truly good for is to measure how affluent the student's family is.
  - Mithrandir 14y ago
    In particular from the post: >"The purpose of the SAT is to predict how well a high school student would perform as a freshman at a college or university in the United States of America (USA)." >"...Another implication of the purpose of the SAT is that taking preparation courses that focus on the exact, narrow content of the SAT, no more and no less, is not "morally wrong". If a person does better on the SAT as a result of specific training or coaching, then, in essence, the SAT has indicated that the person has a quality (such as initiative, motivation, or money) that correlates with good performance in a college or university in the United States of America (USA)."
  - phaus 14y ago
    While I'm a different person at the age of 30 (Just finished my A.S. two days ago with a 3.7 GPA), I have to admit that I was extremely lazy as a teenager. As a matter of fact, I was almost certainly the laziest person in my class. Keeping that in mind, I achieved the second highest score on the PSAT in my school. I don't think that a standardized test as easy as the SATs is a good indicator of work ethic. In my opinion, the completion of a college degree in combination with one's GPA add up to a pretty good, but not perfect, indicator of one's work ethic. Standardized tests on the other hand, could identify intelligence, but probably not in their current forms.
  - seacond 14y ago
    Well said. Except I'd make the following corrections: s/[(]yes.*exam.//; s/.flawed.//; $d I'd add that sometimes the "end" may be pointless, though you might not see it as such while working toward it; years later, in hindsight you might realise it was the "means", i.e., the process of working toward it, i.e. the "hard work", that was the point. Interestingly, there's no "hard work" in taking a test you do not prepare for, e.g., an IQ test.
    - Evbn 14y ago
      Everyone I know with a high IQ did plenty of IQ test practice.
- ComputerGuru 14y ago
  Yes, it's clear (to me at any rate) that you're being sarcastic. But, speaking as someone who once was a SAT taker and now is in a position to hire people based on their SAT scores, your sarcasm is unwarranted and you're looking at the SAT in an incorrect light: the SAT, all things considered, does reflect something. It's not a random number the College Board pulls out of their backside. At the end of the day, having an independent company running standardized tests across the nation in dozens of subjects (there's SAT and then there's SAT II) is really a good thing (TM). Interviews are good, but they're not great. You can't have a good interview without good metrics at hand to qualify what you're seeing. There are geniuses that fail at social interaction and will buckle under the pressure of an interview at CalTech, MIT, Harvard, Stanford, etc. or Google, Microsoft, Facebook, & co. but you'd still want; just as there are interviewees that will ace any verbal interview with flying colors (yes, including technical questions) that you wouldn't. At the end of the day: the SAT score, as it was intended to be used, is a datapoint. It's not the be-all end-all: it's an additional piece of information that should be used responsibly to arrive at a good conclusion whether or not to accept/hire an individual. A good score on the SAT + a good score at an interview means that this person not only knows their stuff and can think on their feet, it also means they're hard-working and will do what it takes to do great (believe me, it takes MONTHS (or even years) of dedication to actually ace the PSAT or SAT). A good score on the SAT and poor interviewing means this person is a hard-worker but not particularly great at thinking on his or her feet. A good interview w/ poor SAT scores indicates someone that couldn't be bothered to study for even a month for an important event in their high school career - can you trust him/her to research thoroughly before taking on a big project? The problem is not with the SAT. The problem is with the interviewers at colleges and workplaces around the country that treat it as the Holy Grail. Don't blame the College Board for this one. Anyway, what would you have the SAT replaced with? Don't tell me you'd have colleges accept students based solely on the grades they achieved - in the United States we have some the highest educational disparity across the nation, and even across neighboring cities and suburbs. An A+ at one school may not even be a C at another. Teachers who's pay depends on the evaluation of their kids' academic performance are not necessarily the people you want giving them a number you'd bet the farm on. Just look at the recent Chicago teachers' strike: at the end of the day, pay and compensations aside, it rested on the fact that teachers don't want standardized tests to be used as metrics to determine the quality of their teaching, and they wanted to set the evaluations themselves. The statistical balance of the scores on the PSAT/ACT/SAT/SATII is very well-studied and well-designed, just like with the USMLE medical board examinations and all other engineered standardized tests. There are easy questions, there are medium questions, there are hard questions, and there are flawed questions which can't be answered - before and after every examination, the questions are classified and then re-classified to avoid anomalous or unfair results. Coming up with these questions, organizing the examinations, coming up with the statistical review methods, grading the exams, and getting the results to the universities is not cheap - I paid for all these exams when I was in highschool myself (and my family was lower middle class at best) and while I then considered them to be pretty damn expensive (and that's why I did my best to get the mark I needed the first time around and worked hard to make sure money was not being thrown away), in retrospect the pricing is very fair (although I hear USMLEs are outrageously expensive, but then again, so is everything associated with the medical industry).
- rayiner 14y ago
  We basically do that. Harvard, Yale, etc, select based on SAT scores, then top employers select based on Harvard, Yale, etc.
dangoldin 14y ago
I've thought of doing this but never had the courage unfortunately. Maybe when I retire I'll have the time to go back and try this out.
beatgammit 14y ago
I admit, I didn't read the whole thing, it was super long. I personally think the ACT (and by association the SAT) is a better measure of intelligence than a GPA, and I would love it if companies used that more often as a filtering metric. Obviously, grades in general aren't the best metric, but they're simple and generally pretty reliable. If I were hiring somebody, I'd look at a combination of test scores and personal achievements. If I were hiring a programmer, I'd filter by test scores, then look at examples of projects the applicant works on/is associated with. Grades aren't as important in my opinion. If that's the premise of the post, I totally agree. If not, I'm not willing to read a forever long post that forces me to relive the horror of the testing I went through before I got into college.
- thaumasiotes 14y ago
  You're looking for the canned tokenadult post which summarizes as follows: [paraphrased] "Decades of research have established that the most effective indicators of whether someone will be a good hire are (on the one hand) a general intelligence test, and (on the other hand) a work sample. Outside the US, the use of intelligence tests in hiring is common; within the US, it subjects the hiring agent to legal difficulties. If you are hiring outside the US, use both. If you are hiring inside the US, use work samples."
  - tokenadult 14y ago
    Actually, the effective hiring procedures of either kind are rare everywhere, but, yes, that is otherwise a fair summary of my FAQ post on company hiring procedures, http://news.ycombinator.com/item?id=4613543 http://news.ycombinator.com/item?id=4613543 which I too thought seems to fit in with several of the comments in this thread. Regrettably, most companies miss out on opportunities to use the best available hiring procedures, preferring the traditional methods to methods validated by research.
- blackhole 14y ago
  Anything involving a test will simply measure one's skill at taking tests instead of what the test is actually about. If this wasn't true at some point in the past, it's certainly true now. Furthermore, seeing as you didn't read the whole thing, you would have missed the part where the SAT itself explicitly states that it is intended not to measure intellect, but rather how well one would perform at a university.
mehulkar 14y ago
This reminds of me playing Hearts on a PC. You can try really hard to get a good score, or you can get the worst score possible and win the round[1]. I didn't read the whole article, but it was pretty clear that to get the lowest possible score you have to know the correct response to every question. (A little less black & white for the essay section, but you get the gist.) Colleges should have a lottery admission available to people who can get a perfect score on the SAT/ACT. Students would inadvertently study harder and learn proportionally more than if they were to study hard enough to get a perfect score. [1] http://en.wikipedia.org/wiki/Hearts#Shooting_the_moon http://en.wikipedia.org/wiki/Hearts#Shooting_the_moon
- greatzebu 14y ago
  You don't really need to identify the correct answer; you only have to be able to find an answer that you're sure isn't correct. Of course, that's much easier if you know what the right answer is, but there's pretty commonly one obviously wrong option.
  - mehulkar 14y ago
    The "obviously wrong option" often turns out not to be as wrong as I first thought.
- gizmo686 14y ago
  > but it was pretty clear that to get the lowest possible score you have to know the correct response to every question. Not exactly. To get the lowest possible score, you only need to know a incorrect response to every questions. This is a very different thing, as it is not uncommon for a question to have some obviously incorrect answers.
  - mehulkar 14y ago
    Knowing an absolutely incorrect answer to every question is difficult to ensure. It leaves things up to chance since you cannot prepare for the exact questions you get. On the other hand, knowing all the correct answers is perfectly doable.
swordswinger12 14y ago
I honestly don't know whether to be impressed or frightened by the level of obsessive attention to detail on display here.
- devcpp 14y ago
  I visited some of the article from his homepage. This guy is an all-around genius! He writes every page on his website in this manner, on so many different fields. Bookmarked. I hope no one will only look at his SAT.
Steko 14y ago
Spoliers (it's very long): he got one question correct, sadly. I feel for him given the preparation and detail he put into it. I missed a few SAT questions my senior year but helpfully, The College Board decided to make the test easier the following year and magically recalibrate my scores to perfect. It was too late for colleges to care and the expected thousand girlfriends never materialized but it did make me feel warm and fuzzy inside.
- Evbn 14y ago
  I didn't get the recalibration :-( one off perfect forever. same as my LSAT.
nateberkopec 14y ago
A lot of the SAT-apologist comments here missed one of the most amazing parts of the post: > "The correlation between [...] combined verbal and math scores and freshman GPA is .52;" .52! And it's pulled straight from the College Board's Terms and Conditions! And later on, it goes on to explain that high school GPA's correlation is just .54! The graph he produced to visualize the scatter involved with a .52 correlation is both hilarious and horrifying.
- tzs 14y ago
  Ideally, shouldn't there be weak or no correlation between score and college grades, when your sample is all SAT takers? My reasoning (not entirely non-serious) is that people who score high on the SAT get into hard schools and so get their GPA smacked down to average, whereas those with poor SAT scores get into easy schools where they can earn an average GPA.
  - brianchu 14y ago
    The problem with that line of reasoning is that GPA is a relative measure - your GPA tells people (theoretically) how you stack up against your classmates - the concept of grading classes on a "curve." Furthermore, there is a correlation between grade inflation and both the selectivity of a school and whether a school is private, with private schools exhibiting more grade inflation (http://gradeinflation.com http://gradeinflation.com).
  - yummyfajitas 14y ago
    Harder schools don't necessarily have harder grading. I taught at both Rutgers and NYU - if anything, Rutgers was tougher than NYU. At Rutgers we failed people regularly. At NYU this would raise red flags and generate huge numbers of complaints (from students and parents).
    - rdl 14y ago
      I thought Rutgers was actually a harder/better school than NYU, at least for engineering. Rutgers is obviously better for film studies.
pbiggar 14y ago
tl;dr: he got one wrong, discussed in section 14.2.4.
tokenadult 14y ago
A lot of the comments here are related to the idea of whether or not the SAT can be regarded as being much like an IQ test. It can, and psychologists routinely think of the SAT that way. Despite a number of statements to the contrary in the various comments here, taking SAT scores as an informative correlate (proxy) of what psychologists call "general intelligence" is a procedure often found in the professional literature of psychology, with the warrant of studies specifically on that issue. Note that it is standard usage among psychologists to treat "general intelligence" as a term that basically equates with "scoring well on IQ tests and good proxies of IQ tests," which is the point of some of the comments here. http://www.iapsych.com/iqmr/koening2008.pdf http://www.iapsych.com/iqmr/koening2008.pdf "Frey and Detterman (2004) showed that the SAT was correlated with measures of general intelligence .82 (.87 when corrected for nonlinearity)" http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144549/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144549/ "Indeed, research suggests that SAT scores load highly on the first principal factor of a factor analysis of cognitive measures; a finding that strongly suggests that the SAT is g loaded (Frey & Detterman, 2004)." http://www.nytimes.com/roomfordebate/2011/12/04/why-should-sats-matter/the-sat-is-a-good-intelligence-test http://www.nytimes.com/roomfordebate/2011/12/04/why-should-s... "Furthermore, the SAT is largely a measure of general intelligence. Scores on the SAT correlate very highly with scores on standardized tests of intelligence, and like IQ scores, are stable across time and not easily increased through training, coaching or practice." http://faculty.psy.ohio-state.edu/peters/lab/pubs/publications/2012_Peters_Beyond_comprehension_The_role_of_numeracy_in_J_and_D.pdf http://faculty.psy.ohio-state.edu/peters/lab/pubs/publicatio... "Numeracy’s effects can be examined when controlling for other proxies of general intelligence (e.g., SAT scores; Stanovich & West, 2008)." As I have heard the issue discussed in the local "journal club" I participate in with professors and graduate students of psychology who focus on human behavioral genetics (including the genetics of IQ), one thing that makes the SAT a very good proxy of general intelligence is that its item content is disclosed (in released previous tests that can be used as practice tests), so that almost the only difference between one test-taker and another in performance on the SAT is generally and consistently getting all of the various items correct, which certainly takes cognitive strengths. Psychologist Keith R. Stanovich makes the interesting point that there are very strong correlations with IQ scores and SAT scores with some of what everyone regards as "smart" behavior (and which psychologists by convention call "general intelligence") while there are still other kinds of tests that plainly have indisputable right answers that high-IQ people are able to muff. Thus Stanovich distinguishes "intelligence" (essentially, IQ) from "rationality" (making correct decisions that overcome human cognitive biases) as distinct aspects of human cognition. He has a whole book on the subject, What Intelligence Tests Miss, that is quite thought-provoking and informative. http://www.amazon.com/What-Intelligence-Tests-Miss-Psychology/dp/0300164629/ http://www.amazon.com/What-Intelligence-Tests-Miss-Psycholog... (Disclosure: I enjoy this kind of research discussion partly because I am acquainted with one large group of high-IQ young people http://cty.jhu.edu/set/ http://cty.jhu.edu/set/ and am interested in how such young people develop over the course of life.)
protomyth 14y ago
My high school required students to take the ASVAB in addition to the ACT (SAT wasn't offered, so I had to go to a testing center for it). I know a guy who honestly tried on the math section. He got the single point for signing his name, but missed all the questions. The first question is "2+2".
ekm2 14y ago
>Scores on the SAT correlate very highly with scores on standardized tests of intelligence, and like IQ scores, are stable across time and not easily increased through training, coaching or practice. That is interesting.How would you account for the fact that i increased my score by 130 points upon a retake without the help of a tutor(Too bad my new score was useless since i already had a fullride)?These tests can be gamed.Heck ,even iQ scores arent stable over a lifetime if anyone has bothered to read current research(Fynn Effect blah blah)
- Evbn 14y ago
  Flynn effect compared genereations, not individual's changes.
  - ekm2 14y ago
    Arent generational changes just the average of individual changes in the same way that the character of a nation emanates from that of individual nations?
ghshephard 14y ago
I had to laugh when reading about the Fantasy "Calculators" - included as a fantasy is a "Slide Rule" - when I took my grade 12 physics provincial finals in 1987, I didn't have a calculator with me, but that was not unexpected, so students were also provided with a booklet of log tables - a predecessor, of course, to the slide rule. The log tables, of course, were more than sufficient for whatever complex multiplication and division that had to be done.
- narsil 14y ago
  These are still used frequently in high schools across the world. For example, Indian high schools following the CBSE system do not allow calculators, but log tables are permitted for some courses.
  - archangel_one 14y ago
    Yep, at my high school in New Zealand we were required to learn how to use log tables, even though we all had calculators. I don't know if it was a compulsory part of the curriculum though, and I wouldn't be surprised if they don't do it any longer.
    - Evbn 14y ago
      Sad. Learning to use of tables helps you learn the concept of logarithm. Way better than a calculator.
imjk 14y ago
Isn't it considerably easier to get every question wrong than it would be to get a perfect score (every question right)? For each question, there is only one correct answer but four incorrect ones. If you're aiming for a perfect score, you have to choose the only one correct answer among the five choices. But if you're trying to get every question wrong, you can choose one of four different answers to get the outcome you want (much better margin for error).
- spicyj 14y ago
  That would be true, but you are in fact allowed to miss a couple of questions and still get a perfect score.
  - Serplat 14y ago
    That depends on the section of the test. Critical reading you can get several wrong, writing depends on your essay score, and math you typically can't get any wrong.
    - ohashi 14y ago
      I made 1 mistake and math and ended up with 790. I was quite annoyed with myself when I walked out knowing exactly which question I messed up too.
      - Evbn 14y ago
        Did it hurt your future prospects?
        ohashi 14y ago
        Ruined my life. Just kidding. It would be impossible to tell what, if any, effect it had. Do I feel like it actually had any effect? Not really.
zheng 14y ago
This is possibly the funniest thing I have read this year. As an aside, the amount of thought and focus this guy obviously can devote to a single topic is amazing. I really hope someone is paying him lots of money (or whatever he prefers) to use his talents for good.
- jobu 14y ago
  It was funny, but in a somewhat disturbing, manic way. I skimmed much of it, but the reference to Weekly World News caught my eye and still can't figure out the point of the tangent even after going back and thoroughly reading most of the article. Section 8.3.10 starts with possible numeric encodings for answers, but then jumps into a philosophical question about admission tests for religion and ends with the reference to WWN and the headline: "10 MORE COMMANDMENTS FOUND! YOU WON'T BELIEVE WHAT THEY SAY!"
ekm2 14y ago
Isnt amazing that at the time you apply to Harvard,Yale and other top schools,admission officials brag at just how many perfect SAT scorers they have turned down,yet a few years down the line turn back and claim to have sifted out the best and the brightest and gladly hint at scores as the measure of brilliance.I even remember reading an article about Harvard adcoms pointing out how they do not want to turn their school into an Ecole Nomale Superieure .Either they lie about inputs or the outputs.They cant have it both ways.
- Evbn 14y ago
  Can you praphrase? I couldn't follow all your pronoun referents.
scotty79 14y ago
Instead of points for correct answers the result of the test should be defined as probability of such set of responses to be achieved by chance only. :-)
oofabz 14y ago
Web Page Making the ugliest one possible Colin Fahey
nitrogen 14y ago
For anyone wondering what is meant by the "12700-choice" questions mentioned starting in section 5.1, they are the questions in the "student-produced response" format analyzed in section 8.
cyphersanctus 14y ago
Collin Fahey has won the internet today in my book. Every single thing he wrote had amazing attention to detail and in many cases was charged with extra meaning. Genius.
dbaupp 14y ago
Mostly off topic, but do USA schoolschildren get taught to use things like "number of degrees of arc", "measures in degrees" and "inverse logarithm" instead of just "degrees" and "exponentiation"/"exponential" (and the use of the last one is the geometric mean, which is a much pithier explanation and concept). And, a correlation of .52 isn't the same as a coin toss: that would be a correlation of 0.
- grecy 14y ago
  Yes, they do. After 35 years teaching in Australia, my parents are now in NYC running seminars for teachers there. They are constantly shocked at the amount of time spent in classes memorizing lexicon, dates, etc. without actually learning to apply any of it. The students can very confidently regurgitate what the "number of degrees of arc" means, but they have no idea how to apply that to anything. They'll also spit out the exact date of some war, but have no idea who fought, why, or what any outcome was.
  - Evbn 14y ago
    I have never seen a teacher or student like you describe. The ones who don't know the (purported) reason for the Civil War don't know when it started either.
jgeralnik 14y ago
Tangentially related, on one of my (national) highschool math finals, I decided to bring an abacus instead of a calculator. The proctor gave me a weird look, I explained I was using it as a calculator, and he gave me no problems.
- Evbn 14y ago
  Did you use it? Aren't abaci rather loud?
  - jgeralnik 14y ago
    No, it was just for show.
unimpressive 14y ago
While reading this, I slowly started to tune out and pressed the back button after realizing just how much material was on the page. I realized that this must be what my friends go through when I explain stuff I'm interested in. Ouch.
pirateking 14y ago
I have done this before on high school and college exams (not the SAT unfortunately), along with other similar experiments like writing answers backwards, using red pen to write my answers to muddle grading attempts, and writing essays on completely different subjects than the one assigned. The level of disbelief expressed by peers and teachers when you challenge their value system led to perhaps one of the most important lessons I learned in my time spent locked in the school system. Thinking outside the box means you are still stuck in a box.

109 points by solipsist 14y ago

deleted 14y ago

[deleted]

The SAT is an important measure of a humans intelligence and performance in the real world, that's why the best companies always filter candidates and select the ones with highest SAT scores. It's a good thing we have tests like these and pay large sums of money to the people who maintain it. Otherwise interviewing might be totally screwed up and completely fail at its intended purpose in this country.

hkmurakami 14y ago
The SAT is good for measuring how good one is at taking the SAT, which in turn is an indicator of how much the test taker took preparing for the exam seriously, which is an indirect indicator of the person's work ethic and attitude towards putting in effort for a means to an end (yes, despite the pointlessness of this exam). It does do a decent job at finding such people who strive to excel within the limits of a (flawed) system, whether or not they realize the pointlessness of said structure. If an institution is looking for an indicator for such people, the SAT is an acceptable tool. Of course, what it's truly good for is to measure how affluent the student's family is.
- Mithrandir 14y ago
  In particular from the post: >"The purpose of the SAT is to predict how well a high school student would perform as a freshman at a college or university in the United States of America (USA)." >"...Another implication of the purpose of the SAT is that taking preparation courses that focus on the exact, narrow content of the SAT, no more and no less, is not "morally wrong". If a person does better on the SAT as a result of specific training or coaching, then, in essence, the SAT has indicated that the person has a quality (such as initiative, motivation, or money) that correlates with good performance in a college or university in the United States of America (USA)."
- phaus 14y ago
  While I'm a different person at the age of 30 (Just finished my A.S. two days ago with a 3.7 GPA), I have to admit that I was extremely lazy as a teenager. As a matter of fact, I was almost certainly the laziest person in my class. Keeping that in mind, I achieved the second highest score on the PSAT in my school. I don't think that a standardized test as easy as the SATs is a good indicator of work ethic. In my opinion, the completion of a college degree in combination with one's GPA add up to a pretty good, but not perfect, indicator of one's work ethic. Standardized tests on the other hand, could identify intelligence, but probably not in their current forms.
- seacond 14y ago
  Well said. Except I'd make the following corrections: s/[(]yes.*exam.//; s/.flawed.//; $d I'd add that sometimes the "end" may be pointless, though you might not see it as such while working toward it; years later, in hindsight you might realise it was the "means", i.e., the process of working toward it, i.e. the "hard work", that was the point. Interestingly, there's no "hard work" in taking a test you do not prepare for, e.g., an IQ test.
  - Evbn 14y ago
    Everyone I know with a high IQ did plenty of IQ test practice.
ComputerGuru 14y ago
Yes, it's clear (to me at any rate) that you're being sarcastic. But, speaking as someone who once was a SAT taker and now is in a position to hire people based on their SAT scores, your sarcasm is unwarranted and you're looking at the SAT in an incorrect light: the SAT, all things considered, does reflect something. It's not a random number the College Board pulls out of their backside. At the end of the day, having an independent company running standardized tests across the nation in dozens of subjects (there's SAT and then there's SAT II) is really a good thing (TM). Interviews are good, but they're not great. You can't have a good interview without good metrics at hand to qualify what you're seeing. There are geniuses that fail at social interaction and will buckle under the pressure of an interview at CalTech, MIT, Harvard, Stanford, etc. or Google, Microsoft, Facebook, & co. but you'd still want; just as there are interviewees that will ace any verbal interview with flying colors (yes, including technical questions) that you wouldn't. At the end of the day: the SAT score, as it was intended to be used, is a datapoint. It's not the be-all end-all: it's an additional piece of information that should be used responsibly to arrive at a good conclusion whether or not to accept/hire an individual. A good score on the SAT + a good score at an interview means that this person not only knows their stuff and can think on their feet, it also means they're hard-working and will do what it takes to do great (believe me, it takes MONTHS (or even years) of dedication to actually ace the PSAT or SAT). A good score on the SAT and poor interviewing means this person is a hard-worker but not particularly great at thinking on his or her feet. A good interview w/ poor SAT scores indicates someone that couldn't be bothered to study for even a month for an important event in their high school career - can you trust him/her to research thoroughly before taking on a big project? The problem is not with the SAT. The problem is with the interviewers at colleges and workplaces around the country that treat it as the Holy Grail. Don't blame the College Board for this one. Anyway, what would you have the SAT replaced with? Don't tell me you'd have colleges accept students based solely on the grades they achieved - in the United States we have some the highest educational disparity across the nation, and even across neighboring cities and suburbs. An A+ at one school may not even be a C at another. Teachers who's pay depends on the evaluation of their kids' academic performance are not necessarily the people you want giving them a number you'd bet the farm on. Just look at the recent Chicago teachers' strike: at the end of the day, pay and compensations aside, it rested on the fact that teachers don't want standardized tests to be used as metrics to determine the quality of their teaching, and they wanted to set the evaluations themselves. The statistical balance of the scores on the PSAT/ACT/SAT/SATII is very well-studied and well-designed, just like with the USMLE medical board examinations and all other engineered standardized tests. There are easy questions, there are medium questions, there are hard questions, and there are flawed questions which can't be answered - before and after every examination, the questions are classified and then re-classified to avoid anomalous or unfair results. Coming up with these questions, organizing the examinations, coming up with the statistical review methods, grading the exams, and getting the results to the universities is not cheap - I paid for all these exams when I was in highschool myself (and my family was lower middle class at best) and while I then considered them to be pretty damn expensive (and that's why I did my best to get the mark I needed the first time around and worked hard to make sure money was not being thrown away), in retrospect the pricing is very fair (although I hear USMLEs are outrageously expensive, but then again, so is everything associated with the medical industry).
rayiner 14y ago
We basically do that. Harvard, Yale, etc, select based on SAT scores, then top employers select based on Harvard, Yale, etc.

dangoldin 14y ago

I've thought of doing this but never had the courage unfortunately. Maybe when I retire I'll have the time to go back and try this out.

beatgammit 14y ago

I admit, I didn't read the whole thing, it was super long. I personally think the ACT (and by association the SAT) is a better measure of intelligence than a GPA, and I would love it if companies used that more often as a filtering metric. Obviously, grades in general aren't the best metric, but they're simple and generally pretty reliable. If I were hiring somebody, I'd look at a combination of test scores and personal achievements. If I were hiring a programmer, I'd filter by test scores, then look at examples of projects the applicant works on/is associated with. Grades aren't as important in my opinion. If that's the premise of the post, I totally agree. If not, I'm not willing to read a forever long post that forces me to relive the horror of the testing I went through before I got into college.

thaumasiotes 14y ago
You're looking for the canned tokenadult post which summarizes as follows: [paraphrased] "Decades of research have established that the most effective indicators of whether someone will be a good hire are (on the one hand) a general intelligence test, and (on the other hand) a work sample. Outside the US, the use of intelligence tests in hiring is common; within the US, it subjects the hiring agent to legal difficulties. If you are hiring outside the US, use both. If you are hiring inside the US, use work samples."
- tokenadult 14y ago
  Actually, the effective hiring procedures of either kind are rare everywhere, but, yes, that is otherwise a fair summary of my FAQ post on company hiring procedures, http://news.ycombinator.com/item?id=4613543 http://news.ycombinator.com/item?id=4613543 which I too thought seems to fit in with several of the comments in this thread. Regrettably, most companies miss out on opportunities to use the best available hiring procedures, preferring the traditional methods to methods validated by research.
blackhole 14y ago
Anything involving a test will simply measure one's skill at taking tests instead of what the test is actually about. If this wasn't true at some point in the past, it's certainly true now. Furthermore, seeing as you didn't read the whole thing, you would have missed the part where the SAT itself explicitly states that it is intended not to measure intellect, but rather how well one would perform at a university.

mehulkar 14y ago

This reminds of me playing Hearts on a PC. You can try really hard to get a good score, or you can get the worst score possible and win the round[1]. I didn't read the whole article, but it was pretty clear that to get the lowest possible score you have to know the correct response to every question. (A little less black & white for the essay section, but you get the gist.) Colleges should have a lottery admission available to people who can get a perfect score on the SAT/ACT. Students would inadvertently study harder and learn proportionally more than if they were to study hard enough to get a perfect score. [1] http://en.wikipedia.org/wiki/Hearts#Shooting_the_moon http://en.wikipedia.org/wiki/Hearts#Shooting_the_moon

greatzebu 14y ago
You don't really need to identify the correct answer; you only have to be able to find an answer that you're sure isn't correct. Of course, that's much easier if you know what the right answer is, but there's pretty commonly one obviously wrong option.
- mehulkar 14y ago
  The "obviously wrong option" often turns out not to be as wrong as I first thought.
gizmo686 14y ago
> but it was pretty clear that to get the lowest possible score you have to know the correct response to every question. Not exactly. To get the lowest possible score, you only need to know a incorrect response to every questions. This is a very different thing, as it is not uncommon for a question to have some obviously incorrect answers.
- mehulkar 14y ago
  Knowing an absolutely incorrect answer to every question is difficult to ensure. It leaves things up to chance since you cannot prepare for the exact questions you get. On the other hand, knowing all the correct answers is perfectly doable.

swordswinger12 14y ago

I honestly don't know whether to be impressed or frightened by the level of obsessive attention to detail on display here.

devcpp 14y ago
I visited some of the article from his homepage. This guy is an all-around genius! He writes every page on his website in this manner, on so many different fields. Bookmarked. I hope no one will only look at his SAT.

Steko 14y ago

Spoliers (it's very long): he got one question correct, sadly. I feel for him given the preparation and detail he put into it. I missed a few SAT questions my senior year but helpfully, The College Board decided to make the test easier the following year and magically recalibrate my scores to perfect. It was too late for colleges to care and the expected thousand girlfriends never materialized but it did make me feel warm and fuzzy inside.

Evbn 14y ago
I didn't get the recalibration :-( one off perfect forever. same as my LSAT.

nateberkopec 14y ago

A lot of the SAT-apologist comments here missed one of the most amazing parts of the post: > "The correlation between [...] combined verbal and math scores and freshman GPA is .52;" .52! And it's pulled straight from the College Board's Terms and Conditions! And later on, it goes on to explain that high school GPA's correlation is just .54! The graph he produced to visualize the scatter involved with a .52 correlation is both hilarious and horrifying.

tzs 14y ago
Ideally, shouldn't there be weak or no correlation between score and college grades, when your sample is all SAT takers? My reasoning (not entirely non-serious) is that people who score high on the SAT get into hard schools and so get their GPA smacked down to average, whereas those with poor SAT scores get into easy schools where they can earn an average GPA.
- brianchu 14y ago
  The problem with that line of reasoning is that GPA is a relative measure - your GPA tells people (theoretically) how you stack up against your classmates - the concept of grading classes on a "curve." Furthermore, there is a correlation between grade inflation and both the selectivity of a school and whether a school is private, with private schools exhibiting more grade inflation (http://gradeinflation.com http://gradeinflation.com).
- yummyfajitas 14y ago
  Harder schools don't necessarily have harder grading. I taught at both Rutgers and NYU - if anything, Rutgers was tougher than NYU. At Rutgers we failed people regularly. At NYU this would raise red flags and generate huge numbers of complaints (from students and parents).
  - rdl 14y ago
    I thought Rutgers was actually a harder/better school than NYU, at least for engineering. Rutgers is obviously better for film studies.

pbiggar 14y ago

tl;dr: he got one wrong, discussed in section 14.2.4.

tokenadult 14y ago

A lot of the comments here are related to the idea of whether or not the SAT can be regarded as being much like an IQ test. It can, and psychologists routinely think of the SAT that way. Despite a number of statements to the contrary in the various comments here, taking SAT scores as an informative correlate (proxy) of what psychologists call "general intelligence" is a procedure often found in the professional literature of psychology, with the warrant of studies specifically on that issue. Note that it is standard usage among psychologists to treat "general intelligence" as a term that basically equates with "scoring well on IQ tests and good proxies of IQ tests," which is the point of some of the comments here. http://www.iapsych.com/iqmr/koening2008.pdf http://www.iapsych.com/iqmr/koening2008.pdf "Frey and Detterman (2004) showed that the SAT was correlated with measures of general intelligence .82 (.87 when corrected for nonlinearity)" http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144549/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144549/ "Indeed, research suggests that SAT scores load highly on the first principal factor of a factor analysis of cognitive measures; a finding that strongly suggests that the SAT is g loaded (Frey & Detterman, 2004)." http://www.nytimes.com/roomfordebate/2011/12/04/why-should-sats-matter/the-sat-is-a-good-intelligence-test http://www.nytimes.com/roomfordebate/2011/12/04/why-should-s... "Furthermore, the SAT is largely a measure of general intelligence. Scores on the SAT correlate very highly with scores on standardized tests of intelligence, and like IQ scores, are stable across time and not easily increased through training, coaching or practice." http://faculty.psy.ohio-state.edu/peters/lab/pubs/publications/2012_Peters_Beyond_comprehension_The_role_of_numeracy_in_J_and_D.pdf http://faculty.psy.ohio-state.edu/peters/lab/pubs/publicatio... "Numeracy’s effects can be examined when controlling for other proxies of general intelligence (e.g., SAT scores; Stanovich & West, 2008)." As I have heard the issue discussed in the local "journal club" I participate in with professors and graduate students of psychology who focus on human behavioral genetics (including the genetics of IQ), one thing that makes the SAT a very good proxy of general intelligence is that its item content is disclosed (in released previous tests that can be used as practice tests), so that almost the only difference between one test-taker and another in performance on the SAT is generally and consistently getting all of the various items correct, which certainly takes cognitive strengths. Psychologist Keith R. Stanovich makes the interesting point that there are very strong correlations with IQ scores and SAT scores with some of what everyone regards as "smart" behavior (and which psychologists by convention call "general intelligence") while there are still other kinds of tests that plainly have indisputable right answers that high-IQ people are able to muff. Thus Stanovich distinguishes "intelligence" (essentially, IQ) from "rationality" (making correct decisions that overcome human cognitive biases) as distinct aspects of human cognition. He has a whole book on the subject, What Intelligence Tests Miss, that is quite thought-provoking and informative. http://www.amazon.com/What-Intelligence-Tests-Miss-Psychology/dp/0300164629/ http://www.amazon.com/What-Intelligence-Tests-Miss-Psycholog... (Disclosure: I enjoy this kind of research discussion partly because I am acquainted with one large group of high-IQ young people http://cty.jhu.edu/set/ http://cty.jhu.edu/set/ and am interested in how such young people develop over the course of life.)

protomyth 14y ago

My high school required students to take the ASVAB in addition to the ACT (SAT wasn't offered, so I had to go to a testing center for it). I know a guy who honestly tried on the math section. He got the single point for signing his name, but missed all the questions. The first question is "2+2".

ekm2 14y ago

>Scores on the SAT correlate very highly with scores on standardized tests of intelligence, and like IQ scores, are stable across time and not easily increased through training, coaching or practice. That is interesting.How would you account for the fact that i increased my score by 130 points upon a retake without the help of a tutor(Too bad my new score was useless since i already had a fullride)?These tests can be gamed.Heck ,even iQ scores arent stable over a lifetime if anyone has bothered to read current research(Fynn Effect blah blah)

Evbn 14y ago
Flynn effect compared genereations, not individual's changes.
- ekm2 14y ago
  Arent generational changes just the average of individual changes in the same way that the character of a nation emanates from that of individual nations?

ghshephard 14y ago

I had to laugh when reading about the Fantasy "Calculators" - included as a fantasy is a "Slide Rule" - when I took my grade 12 physics provincial finals in 1987, I didn't have a calculator with me, but that was not unexpected, so students were also provided with a booklet of log tables - a predecessor, of course, to the slide rule. The log tables, of course, were more than sufficient for whatever complex multiplication and division that had to be done.

narsil 14y ago
These are still used frequently in high schools across the world. For example, Indian high schools following the CBSE system do not allow calculators, but log tables are permitted for some courses.
- archangel_one 14y ago
  Yep, at my high school in New Zealand we were required to learn how to use log tables, even though we all had calculators. I don't know if it was a compulsory part of the curriculum though, and I wouldn't be surprised if they don't do it any longer.
  - Evbn 14y ago
    Sad. Learning to use of tables helps you learn the concept of logarithm. Way better than a calculator.

imjk 14y ago

Isn't it considerably easier to get every question wrong than it would be to get a perfect score (every question right)? For each question, there is only one correct answer but four incorrect ones. If you're aiming for a perfect score, you have to choose the only one correct answer among the five choices. But if you're trying to get every question wrong, you can choose one of four different answers to get the outcome you want (much better margin for error).

spicyj 14y ago
That would be true, but you are in fact allowed to miss a couple of questions and still get a perfect score.
- Serplat 14y ago
  That depends on the section of the test. Critical reading you can get several wrong, writing depends on your essay score, and math you typically can't get any wrong.
  - ohashi 14y ago
    I made 1 mistake and math and ended up with 790. I was quite annoyed with myself when I walked out knowing exactly which question I messed up too.
    - Evbn 14y ago
      Did it hurt your future prospects?
      - ohashi 14y ago
        Ruined my life. Just kidding. It would be impossible to tell what, if any, effect it had. Do I feel like it actually had any effect? Not really.

zheng 14y ago

This is possibly the funniest thing I have read this year. As an aside, the amount of thought and focus this guy obviously can devote to a single topic is amazing. I really hope someone is paying him lots of money (or whatever he prefers) to use his talents for good.

jobu 14y ago
It was funny, but in a somewhat disturbing, manic way. I skimmed much of it, but the reference to Weekly World News caught my eye and still can't figure out the point of the tangent even after going back and thoroughly reading most of the article. Section 8.3.10 starts with possible numeric encodings for answers, but then jumps into a philosophical question about admission tests for religion and ends with the reference to WWN and the headline: "10 MORE COMMANDMENTS FOUND! YOU WON'T BELIEVE WHAT THEY SAY!"

ekm2 14y ago

Isnt amazing that at the time you apply to Harvard,Yale and other top schools,admission officials brag at just how many perfect SAT scorers they have turned down,yet a few years down the line turn back and claim to have sifted out the best and the brightest and gladly hint at scores as the measure of brilliance.I even remember reading an article about Harvard adcoms pointing out how they do not want to turn their school into an Ecole Nomale Superieure .Either they lie about inputs or the outputs.They cant have it both ways.

Evbn 14y ago
Can you praphrase? I couldn't follow all your pronoun referents.

scotty79 14y ago

Instead of points for correct answers the result of the test should be defined as probability of such set of responses to be achieved by chance only. :-)

oofabz 14y ago

Web Page Making the ugliest one possible Colin Fahey

nitrogen 14y ago

For anyone wondering what is meant by the "12700-choice" questions mentioned starting in section 5.1, they are the questions in the "student-produced response" format analyzed in section 8.

cyphersanctus 14y ago

Collin Fahey has won the internet today in my book. Every single thing he wrote had amazing attention to detail and in many cases was charged with extra meaning. Genius.

dbaupp 14y ago

Mostly off topic, but do USA schoolschildren get taught to use things like "number of degrees of arc", "measures in degrees" and "inverse logarithm" instead of just "degrees" and "exponentiation"/"exponential" (and the use of the last one is the geometric mean, which is a much pithier explanation and concept). And, a correlation of .52 isn't the same as a coin toss: that would be a correlation of 0.

grecy 14y ago
Yes, they do. After 35 years teaching in Australia, my parents are now in NYC running seminars for teachers there. They are constantly shocked at the amount of time spent in classes memorizing lexicon, dates, etc. without actually learning to apply any of it. The students can very confidently regurgitate what the "number of degrees of arc" means, but they have no idea how to apply that to anything. They'll also spit out the exact date of some war, but have no idea who fought, why, or what any outcome was.
- Evbn 14y ago
  I have never seen a teacher or student like you describe. The ones who don't know the (purported) reason for the Civil War don't know when it started either.

jgeralnik 14y ago

Tangentially related, on one of my (national) highschool math finals, I decided to bring an abacus instead of a calculator. The proctor gave me a weird look, I explained I was using it as a calculator, and he gave me no problems.

Evbn 14y ago
Did you use it? Aren't abaci rather loud?
- jgeralnik 14y ago
  No, it was just for show.

unimpressive 14y ago

While reading this, I slowly started to tune out and pressed the back button after realizing just how much material was on the page. I realized that this must be what my friends go through when I explain stuff I'm interested in. Ouch.

pirateking 14y ago

I have done this before on high school and college exams (not the SAT unfortunately), along with other similar experiments like writing answers backwards, using red pen to write my answers to muddle grading attempts, and writing essays on completely different subjects than the one assigned. The level of disbelief expressed by peers and teachers when you challenge their value system led to perhaps one of the most important lessons I learned in my time spent locked in the school system. Thinking outside the box means you are still stuck in a box.