10 ms·

I used to think the SATs were fair. Until I found out how much money was being spent on SAT prep. And those expensive prep courses had the potential to increase

by yardie 7y ago

I used to think the SATs were fair. Until I found out how much money was being spent on SAT prep. And those expensive prep courses had the potential to increase scores over 100 points. I took the recommended SAT prep course through my high school. It was 2 weekends of going over material that might be on the exam and a workbook recommended by The College Board. Imagine my surprise going to university and meeting students much richer than I who had multi year SAT prep courses with actual exam questions!

duckMuppet 7y ago
Anyone can study the SAT prep books for free at a bookstore a few hours at a time daily. If your offended by that, what about professors or HS teachers who teach certain sections of the textbook because it's going to be on the exam. One can self-study the SAT prep book or self-study their biology book and in both cases can potentially be more knowledgeable than those who attended a formal university class or prep workshop. Most students aren't driven to do that though. This goes to the fundamental problem, that our university system at present is really just a credentialing and signaling system.
- nicoburns 7y ago
  > If your offended by that, what about professors or HS teachers who teach certain sections of the textbook because it's going to be on the exam. Yes, this another big problem caused by (too much emphasis on) standardised testing. I learnt the most in classes that didn't do this. But people do better on tests when they do do this. Standardised tests are destroying our education system.
  - naniwaduni 7y ago
    Goodhart's law in action!
  - mattkrause 7y ago
    Surely the standardization matters? Teaching to an aptitude test is clearly counterproductive (and ditto tricks for hacking the test--like just testing each multiple choice answer instead of solving an equation). On the other hand. the AP classes I took in high school (admittedly, a long time ago) seemed pretty reasonable. Perhaps you could quibble with how the syllabus weighted various subjects, or the (mercifully few) classes on how structure exam answers. Overall though, I felt like I got a decent education in Spanish/History/Physics/etc from those courses.
    - pbhjpbhj 7y ago
      Diversity in knowledge across the working population would seem preferable to homogeneity to me. Sure you want some core things, like all engineers being able to solve 2nd order DEs, but outside that you want diversity if you're looking for innovation and application to diverse fields.
      - mattkrause 7y ago
        In general, sure, but how much diversity you can reasonably expect for say, high school calculus? It’s not as though schools abandoned their infinitesimal-based curricula to match the AP syllabus better. The AP US History curriculum also kind of anticipates your critique: “As has been the case for all prior versions of the AP U.S. History course, this AP U.S. History course framework includes a minimal number of individual names: the founders, several presidents and party leaders, and other individuals who are almost universally taught in college-level U.S. history courses. As history teachers know well, the material in this framework cannot be taught without careful attention to the individuals, events, and documents of American history; however, to ensure teachers have flexibility to teach specific content that is valued locally and individually, the course avoids prescribing details that would require all teachers to teach the same historical examples. Each teacher is responsible for selecting specific individuals, events, and documents for student investigation of the material in the course framework.” https://apstudents.collegeboard.org/ap/pdf/ap-us-history-course-and-exam-description.pdf https://apstudents.collegeboard.org/ap/pdf/ap-us-history-cou... (Page 11/15) I would argue that it’s possible to have a good yet flexible standardized system; it’s just hard.
    - nicoburns 7y ago
      I think it's the testing that's the problem, more so than the standardization (although standardization can definitely be taken too far to the point that it's unhelpful). You can of course have a standardized curriculum without any testing at all. Countries like Finland put far less emphasis on test results, and a lot more emphasis on teacher's assessment of their pupils. It seems like a much healthier system to me.
      - 7y ago
askafriend 7y ago
That's true, but at the same time it's a standardized test so anyone can replicate those prep courses. There's tons of prep books with practice questions, tips, etc. It's a matter of awareness and taking it seriously (obviously a simplification, but hope you get my point).
- thaumasiotes 7y ago
  I mean, it's not really true. People spend a lot of money on test prep, but this isn't even a case where the evidence hasn't come in yet. We know perfectly well that test prep has negligible influence on SAT scores.
  - seanmcdirmid 7y ago
    Those studies were mostly funded and influenced by the college board, and they needed that to be true, so that’s what they found. But there is prep and there is prep!, they are different with different influences. Studying the former (say a few weekend long prep course) to make conclusions about the latter (intensive cram school for a couple of years) is criminal. I’ve seen an SAT cram school in China, and yes...it will influence your scores significantly because they pull out all of the stops.
    - thaumasiotes 7y ago
      > Those studies were mostly funded and influenced by the college board, and they needed that to be true, so that’s what they found. The SAT draws a huge amount of overtly hostile attention. So does SAT prep. But these highly motivated SAT-hostile researchers haven't been able to produce the results they need.
      - seanmcdirmid 7y ago
        That simply isn’t true, eg see https://www.washingtonpost.com/news/answer-sheet/wp/2017/05/09/can-coaching-truly-boost-sat-scores-for-years-the-college-board-said-no-now-it-says-yes/ https://www.washingtonpost.com/news/answer-sheet/wp/2017/05/... The College Board can’t really have it both ways. And of course: https://www.reuters.com/investigates/special-report/college-sat-two/ https://www.reuters.com/investigates/special-report/college-... Which is just one example of pulling out all the stops.
Symmetry 7y ago
SAT prep courses advertise that they can increase scores by 100 points but generally they're lying in order to sell more test courses. But the real number is closer to "11 to 15 points on the math section and 6 to 9 points on the verbal."[1] [1]https://www.researchgate.net/publication/228337033_Using_Linear_Regression_and_Propensity_Score_Matching_to_Estimate_the_Effect_of_Coaching_on_the_SAT https://www.researchgate.net/publication/228337033_Using_Lin...
- thaumasiotes 7y ago
  Note also that the measurement error of a section score is 30 points, which dwarfs the effect of prep.
- asdfman123 7y ago
  As a former SAT tutor (and a guy who gamed the hell out of the SAT) I have trouble believing that. There are only a few classes of questions they ask on the test, and if you master them the whole test is a walk in the park. Maybe the problem is most students just don't give a damn about the classes their parents make them take? Can someone explain the findings in this paper?
  - graeme 7y ago
    My impression as a former SAT instructor is a lot of students don’t spend that much time prepping. And if they have no good reading/math background it can take some effort to master things. That said, typically saw increases well above that in my classes. Even with zero instruction I’d expect a self studying student to improve more than that from familiarization. It did say upper class people improve more.
  - closeparen 7y ago
    The SAT is designed to be an intelligence test, not a course final, and it's possible that the large team of education and psychometrics people who've been working on it for decades know what they're doing. Armchair speculation: there are probably some really bright kids out there with poor test-taking skills, and prep likely has a dramatic effect on scores for them, but most students are not that. Did you actually sit an SAT before you started gaming? What was your improvement?
albntomat0 7y ago
How is that functionally different from most things considered by college admissions? Money helps a ton for normal class, extra-curriculars, etc.
realtalk_sp 7y ago
While it's true that prep courses can help dramatically, the SAT subject tests are also not really that hard. I don't consider myself terribly smart (certainly not MIT level) and I managed to get an 800 on the Math II and 800 on Chemistry with a minimum of studying using pretty inexpensive books. This applied to many of my friends as well.
- UncleMeat 7y ago
  This is one of the reasons the subject tests are not especially useful. And since they test knowledge rather than aptitude, making them more difficult doesn't actually find smarter people but instead people who had access to more rigorous high school coursework. At the GRE level this is even more extreme. For competitive programs a perfect score doesn't distinguish you so taking the test can only harm you. When applying to grad school I was explicitly advised not to take the GRE subject tests.
  - applecrazy 7y ago
    The subject tests, IMO, are terrible. I took the Chemistry one right after taking AP (college level) Chemistry, and while I got a full score on that, I got absolutely destroyed on the subject test. They asked trivia-like questions and barely tested for actual chemistry knowledge.
    - saagarjha 7y ago
      Interestingly, I had the opposite experience, finding the AP Chemistry exam more difficult than the subject exam. I think part of it is that I have no love for chemistry, so I just memorized the things I needed for the subject test.
- jimmyvalmer 7y ago
  I was a child of the 80s, and while I agree the Math "Level 2" was a cakewalk, getting an 800 on Chem was definitely something to write home about. I get that scaling may have changed in the past thirty years, but still!
aliston 7y ago
The real scandal that I can't believe hasn't gotten more attention is that wealthy families are hiring psychologists to claim their child has a learning disability. They can then get more time to take the test and due to a policy change a few years ago, this information is hidden from the schools when students report their scores.
- Spivak 7y ago
  Look, faking a disability is deplorable but this stance is bonkers. I wish my parents had the money when I was growing up to get me checked and diagnose my ADHD. And of course you wouldn't want to report to the college if a person has a disability, are you trying to scarlet letter everyone who needs a little more time to read from dyslexia?
  - hoka-one-one 7y ago
    As someone who has used similar accommodations (though not specifically on the SAT), yes. Dyslexia will hold you back in college. SAT is intended to measure odds of success in college.
    - Spivak 7y ago
      You're not wrong but I think that's due to how much of university is based around timed tests. The artificial difficulty is maddening. I was fortunate that my school's disability office had fangs and that the maths program wasn't super competitive and wanted students to learn more than get marks. My favorite professor was teaching number theory and always scheduled his classes for the end day and gave unlimited time for exams. Every time I would sit there for at least 5 hours (which wasn't super uncommon) and it was the nicest thing to not be freaking out and trying to force myself to focus.
      - hoka-one-one 7y ago
        I disagree that it's artificial. There's only so much time in a day. A programmer who can do a task in one hour will in the long run outsprint one who needs two. Yes, you can (like many, many others) spend an extra couple hours at work every day, but it'll all come back to center eventually. Sometimes, or even much of the time, you won't even have that option. These are times when people seriously depend on you to perform. It's not personal. I get that it can be stressful and I've been there but I don't kid myself that I'm as good as the people who can work quickly and under pressure.
asdfman123 7y ago
I've worked as an SAT prep tutor and I've worked in the oil industry. Actually, I felt way more guilt about my SAT tutoring work than the oil industry. The world runs on oil, but no one needs expensive SAT tutoring to give rich kids an edge over the poor ones.
- Spivak 7y ago
  This seems like such a weird stance since pretty much all formal education isn't free. Even without SAT prep rich families can still afford better schools, private tutors, set up apprenticeships. It's a weird world where people are mad that those with means are using them to educate themselves and their children. As an adult I choose to pay for lots of different forms of education that gives me "advantage" -- bleh -- over my friends, peers, coworkers, etc.. Why should I feel guilty that I have the means to buy culinary textbooks so I can bake better?
  - asdfman123 7y ago
    I'm not angry at the people who want to help their kids, but at the system that we're making. Our "meritocracy" has become a caste system in which a small group of people are doing very well and everyone else is scrambling to make it. It's a big social problem. Look at how suicide rates in middle America has skyrocketed, look at how much more divisive our politics have become. I have no idea what the solution is, and can't say for sure where we should place the blame, but I have an innate distaste for the passing along of inherited advantages.
  - kaibee 7y ago
    I think the difference is that, better schools and private tutors, are at least in theory, teaching useful skills and ultimately creating a better person at the end. Being good at the SATs doesn't have the same excuse, so its much more obvious when you're basically just paying to get a better score.
    - Spivak 7y ago
      Look, I don't disagree with you -- other than the score SAT prep is fantastically bad education investment. But I think this is a problem with the testing more than some larger class issue. We've rewarded learning a bunch of useless facts and test taking strategies. You could probably switch out the SAT for an entire test of movie trivia and you'd probably get the same distribution of performance since ACT/SAT just reward the people who study for it the most. You can't accidentally prepare for these tests by other schooling that's not geared for it.
neaden 7y ago
The question shouldn't be is the SAT unfairly biased towards families with money, since of course it is, but is the SAT less biased towards families with money then other criteria. Having parents with money and connections lets you get tutoring to increase your GPA, tutoring to write essays, connections to get internships, the free time to do clubs, training to excel in sports, and so many other things.
- Spivak 7y ago
  Right! I don't understand why people in this thread are apparently mad rich families for this? Like a family with means is doing all the things to raise a socialized, well rounded, experienced, well-read and tutored, worldly person. Like god damn I can't think of a better use for the money as a parent. I get that college admissions is a contest and performative but hot damn this thread is full of people mad at other people's happiness.
  - int_19h 7y ago
    People are mad, because that happiness comes at their expense. Not because rich people prep their kids per se, but because society is a zero-sum game in many ways, and the end result is a de facto caste system. People are often willing to endure unfair hardships, if their children get a better chance. But if they know that not even their great-grandchildren are likely to be significantly better off, the focus shifts back to here and now.
  - Dylan16807 7y ago
    > socialized, well rounded, experienced, well-read and tutored, worldly person Test prep doesn't help with any of that.
hackinthebochs 7y ago
I'm an underrepresented minority and I increased my SAT score by 200-300 from the PSAT to SAT, depending on how you want to measure the change. This was just by doing self-study with off the shelf test prep material. The classes aren't what increases your score, its the consistent structured study and ones innate potential. For some people, no amount of study is going to get them into the 90+th percentile. While others who have the capacity to do well but haven't had the best instruction over the years can cover a lot of ground with the right prep. But this is exactly what SAT is intended to measure, scholastic potential. That some people can increase scores dramatically through preparation does not indicate a failure of the test, but rather its success.
- SkyBelow 7y ago
  >But this is exactly what SAT is intended to measure, scholastic potential. But if you define the variable like this, you end up opening a can of worms. Do rich kids have more scholastic potential because they have better access to resources? Or should scholastic potential be the measurement of how a child performs assuming they had the same access to resources? Should children who belong to a racial/ethnic group that have social stereotypes of having more scholastic potential be defined as having more scholastic potential because of the effect of stereotypes and labels, or should the measure of scholastic potential be of the underlying potential if the effects of stereotypes and labels were equal? Normally measurements of potential aim to be to measure the factors least impacted by other factors, because if we account for those factors we are measuring actual instead of potential.
  - askafriend 7y ago
    > Do rich kids have more scholastic potential because they have better access to resources? Yes. They do. It's maybe not the answer you were looking for, but if you are better prepared, you'll do better at the task at hand. Seems fairly straightforward. The next question would be how easy are those resources to access (in the case of the SAT)? And the answer is you can do pretty well with a $20 test prep book that you can order from Amazon. The rest is time, effort, awareness etc. Then we start getting into factors that are much harder to control or evaluate - parental involvement, guidance, nutrition, emotional support, etc.
sushid 7y ago
This reads more like a humblebrag to me than a thoughtful criticism against standardized testing.
indigochill 7y ago
The main thing that frustrated me about the SAT, at least when I took it, was that it essentially boiled down to an IQ test, not a true "scholastic aptitude" test. I did barely any prep (aside from, I think, a flip calendar with questions), but I scored 1510 out of 1600 at the time which qualified me for a prestigious program in my university. Then I narrowly avoided flunking out because I'd never actually learned to study or manage my time properly in high school because I'd never needed to until college. Study skills and time management seem like pretty important factors in a true measure of scholastic aptitude.
- colinmhayes 7y ago
  Similar story to you except I graduated with a 3.4 from a top 10 university without learning to study or manage my time. I guess the test was right?
hangonhn 7y ago
These advantages are more subtle than even that. I was shamed by some older fellow alums when I told people that my parents had hired someone to go over my application and essays. Their stance was that my parents gave me an unfair advantage (note, both of my parents are immigrants who did not even finish high school). My retort to them was point out that most of them didn't exactly grow up in the crime ridden rural parts of the world (quite the opposite in fact). Their parents paid a higher tax rate to support the world class public education they got, which they just assumed is the norm. That shut them up rather quickly. SAT prep classes or not, coming from a more advantageous social-economic background grants you so many benefits in many ways. My parents wouldn't have to hire someone to review my applications if my school had a good college counselor (50% of my classmates do not go on to college).
- pbhjpbhj 7y ago
  >coming from a more advantageous social-economic background grants you so many benefits in many ways // It's not all benefit: you can be taught to be lazy, not to be self-reliant, you can lack tenacity and resilience, you can have perverse expectations and a sheltered view of the World. Such things can turn out to be a hindrance to self-fulfillment.
  - hangonhn 7y ago
    Yeah that's a fair point for sure and I've seen examples of those too. However, even then some well resourced parents will still clear a path for these children to get into selective schools as the recent college admissions scandals revealed.
losvedir 7y ago
100 points isn't that much. The average score is 1000-1100. The elite universities expect 1500. Tutoring, even if it gave you 100 points, isn't going to cross that gap. I'd say it's still relatively fair. Fairer than essentially any other admissions criterion: grades aren't comparable across schools, extra-curriculars are much more about money spent, etc.
saithound 7y ago
The SAT is not fair. By taking an expensive prep course, you can potentially increase your SAT score by 100 (although that's quite extreme, the average improvement attributed to SAT prep is closer to 20 points). That's a significant, but not earth-shattering improvement: it moves you from the 75th percentile to the 84th, or from the 97th percentile to the 99th. Now let's compare that to other criteria used in the college admissions process. It's way easier to have impressive and relevant extracurricular activities if you're rich and go to a good school. And unlike SAT prep, acquiring good extracurriculars will definitely take years of your time and money. Well-written admissions essay? MIT requires one. But you do it at home, instead of a tightly controlled testing center. So if you're rich enough, you can have ghost-written admissions essays. Needless to say, this process can turn even a completely worthless essay into an impressive one. Creative portfolio? Unless you're in the 99th percentile of musical talent, the difference between "I write songs that I play on an old guitar" and "an orchestra performed my composition" is money. The former is probably not even 75th percentile; the latter, probably 95+. Alumni parents who would be likely to donate big bucks? You don't need an expensive prep course to get that, yet it can provide your application with a much bigger boost than 100 SAT points. And unlike the SAT, if you don't have alumni parents to begin with, then you'll never get this boost, no matter how much extra work you put in. The SAT is not fair. But it's the fairest admissions criterion used by U.S. universities today. (note: MIT still relies on the SAT; today's announcement concerns the SAT Subject Tests)
- lollercoasr2020 7y ago
  Truth be told: the culture of those institutions is based on extremely synthetic virtue-signalling and superficial values and is very far from being meritocratic or open to social mobility. This might be great for rich kids, though in case you are ambitious, clever, gritty but come from not-so-privileged background (not necessarily low-class or third world - just temporary economic turmoil is enough) - you are going to hate every single second you spend there with all your heart. So maybe this way it is a win-win in the end. The aristocracy get their titles and the plebs their opportunities for self-realisation. No harm done.
- commandlinefan 7y ago
  > it's the fairest admissions criterion I have yet to see an "abolish standardized tests" type suggest a workable alternative (or even an unworkable one).
scythe 7y ago
The question isn't "can the SATs be gamed?", though. The question is "can the SATs be gamed more than whatever other criteria you use?". Simply analyzing one half of the equation won't produce a correct answer. Rich people can game standardized tests. They can also game grades. They can game admission essays. They can game interviews. If colleges admitted students based on brain fMRI data, I wouldn't be at all surprised to find out that rich people managed to exploit that, too. Are the SATs unfair? Absolutely. Are they more unfair than the alternative? I'm not convinced. I usually prefer the devil I know.
bermanoid 7y ago
That's a fair criticism of the general SAT - I used to teach those classes, and the test is very gameable in the sense that there really are "A FEW SIMPLE TRICKS!" to learn that have nothing to do with actually knowing the material in a useful way. But at least some of the subject SATs are not like that at all, specifically the math/science ones. There really aren't many tricks or traps, they really are like normal school tests (if multiple choice) where doing well on them requires you to know the material they cover. Nobody who is "good at tests" is going to 800 the physics one without knowing physics well enough that they'd do well in a freshman mechanics course, and someone who gets a 400 either slept through class or is going to struggle at college level.
CapmCrackaWaka 7y ago
I work for a company that collects aggregated SAT data. I was pretty upset when I discovered there is a very strong, direct correlation between family income and SAT score, with a credible spread of several hundred points between the average scores of the highest and lowest income groups. I knew that there would be a correlation, but seeing how strong it was was pretty depressing.
- fastball 7y ago
  Why is that depressing?
- jimmyvalmer 7y ago
  Just so we're clear: IQ does correlate with income, for obvious reasons.
koboll 7y ago
>And those expensive prep courses had the potential to increase scores over 100 points. I took one of those expensive scores and got the exact same score after the course as I did before the course. It turned out the cheap study book was every bit as effective as the expensive course.
JamesBarney 7y ago
You can get like 80% of the value of an expensive sat prep course from a cheap sat book. But money also buys you grades, gpa, extracurriculars, and recommendations.
devonkim 7y ago
I realized how much of a scam standardized testing was when I used to do the PSATs and increased my score mostly by using a different strategy rather than getting smarter or anything. The SAT prep class my high school offered for like $40 was a complete joke because they were designed for public school students where you get around 1250 and it's pretty good for state school admissions. Until I learned to treat the test as a game and understand its scoring I went from 1330 to 1520 simply by _not_ answering anything I wasn't fully confident with my answer - this is literally the opposite strategy I was instructed to do where they encouraged people to guess. To get to the higher end, you are best off never getting even a partial deduction for getting something wrong. When I got my best score, I only got 3 questions wrong but far more unanswered.
thedance 7y ago
What really increases your score is having a good night's sleep and a healthy breakfast before school every day of your life prior to taking the test. The test doesn't really measure aptitude because it can't control for those inputs.
- lonelappde 7y ago
  Those inputs cause aptitude.
manfredo 7y ago
Coming from someone who took said expensive prep courses, I can tell you that just taking practice exam after practice exam is easily 90% of the benefit. The test taking strategy they teach is something anyone with serious prospects of getting into MIT would have learned years ago.
outlace 7y ago
I think the courses mostly help with discipline in studying for the test. There’s nothing magical about them. I’ve taken many standardized exams with and without prep courses and it’s quite possible to do very well with inexpensive self study material if you’re incredibly disciplined about studying.
RedBeetDeadpool 7y ago
If you gave me a student who had enough motivation to succeed, I bet I could raise his score far far far more than just 100.
underpand 7y ago
> And those expensive prep courses had the potential to increase scores over 100 points. As compared with self-studying with free online practice tests and resources having the potential to increase scores over 100 points.

by yardie 7y ago

duckMuppet 7y ago

Anyone can study the SAT prep books for free at a bookstore a few hours at a time daily. If your offended by that, what about professors or HS teachers who teach certain sections of the textbook because it's going to be on the exam. One can self-study the SAT prep book or self-study their biology book and in both cases can potentially be more knowledgeable than those who attended a formal university class or prep workshop. Most students aren't driven to do that though. This goes to the fundamental problem, that our university system at present is really just a credentialing and signaling system.

nicoburns 7y ago
> If your offended by that, what about professors or HS teachers who teach certain sections of the textbook because it's going to be on the exam. Yes, this another big problem caused by (too much emphasis on) standardised testing. I learnt the most in classes that didn't do this. But people do better on tests when they do do this. Standardised tests are destroying our education system.
- naniwaduni 7y ago
  Goodhart's law in action!
- mattkrause 7y ago
  Surely the standardization matters? Teaching to an aptitude test is clearly counterproductive (and ditto tricks for hacking the test--like just testing each multiple choice answer instead of solving an equation). On the other hand. the AP classes I took in high school (admittedly, a long time ago) seemed pretty reasonable. Perhaps you could quibble with how the syllabus weighted various subjects, or the (mercifully few) classes on how structure exam answers. Overall though, I felt like I got a decent education in Spanish/History/Physics/etc from those courses.
  - pbhjpbhj 7y ago
    Diversity in knowledge across the working population would seem preferable to homogeneity to me. Sure you want some core things, like all engineers being able to solve 2nd order DEs, but outside that you want diversity if you're looking for innovation and application to diverse fields.
    - mattkrause 7y ago
      In general, sure, but how much diversity you can reasonably expect for say, high school calculus? It’s not as though schools abandoned their infinitesimal-based curricula to match the AP syllabus better. The AP US History curriculum also kind of anticipates your critique: “As has been the case for all prior versions of the AP U.S. History course, this AP U.S. History course framework includes a minimal number of individual names: the founders, several presidents and party leaders, and other individuals who are almost universally taught in college-level U.S. history courses. As history teachers know well, the material in this framework cannot be taught without careful attention to the individuals, events, and documents of American history; however, to ensure teachers have flexibility to teach specific content that is valued locally and individually, the course avoids prescribing details that would require all teachers to teach the same historical examples. Each teacher is responsible for selecting specific individuals, events, and documents for student investigation of the material in the course framework.” https://apstudents.collegeboard.org/ap/pdf/ap-us-history-course-and-exam-description.pdf https://apstudents.collegeboard.org/ap/pdf/ap-us-history-cou... (Page 11/15) I would argue that it’s possible to have a good yet flexible standardized system; it’s just hard.
  - nicoburns 7y ago
    I think it's the testing that's the problem, more so than the standardization (although standardization can definitely be taken too far to the point that it's unhelpful). You can of course have a standardized curriculum without any testing at all. Countries like Finland put far less emphasis on test results, and a lot more emphasis on teacher's assessment of their pupils. It seems like a much healthier system to me.
    - 7y ago

askafriend 7y ago

That's true, but at the same time it's a standardized test so anyone can replicate those prep courses. There's tons of prep books with practice questions, tips, etc. It's a matter of awareness and taking it seriously (obviously a simplification, but hope you get my point).

thaumasiotes 7y ago
I mean, it's not really true. People spend a lot of money on test prep, but this isn't even a case where the evidence hasn't come in yet. We know perfectly well that test prep has negligible influence on SAT scores.
- seanmcdirmid 7y ago
  Those studies were mostly funded and influenced by the college board, and they needed that to be true, so that’s what they found. But there is prep and there is prep!, they are different with different influences. Studying the former (say a few weekend long prep course) to make conclusions about the latter (intensive cram school for a couple of years) is criminal. I’ve seen an SAT cram school in China, and yes...it will influence your scores significantly because they pull out all of the stops.
  - thaumasiotes 7y ago
    > Those studies were mostly funded and influenced by the college board, and they needed that to be true, so that’s what they found. The SAT draws a huge amount of overtly hostile attention. So does SAT prep. But these highly motivated SAT-hostile researchers haven't been able to produce the results they need.
    - seanmcdirmid 7y ago
      That simply isn’t true, eg see https://www.washingtonpost.com/news/answer-sheet/wp/2017/05/09/can-coaching-truly-boost-sat-scores-for-years-the-college-board-said-no-now-it-says-yes/ https://www.washingtonpost.com/news/answer-sheet/wp/2017/05/... The College Board can’t really have it both ways. And of course: https://www.reuters.com/investigates/special-report/college-sat-two/ https://www.reuters.com/investigates/special-report/college-... Which is just one example of pulling out all the stops.

Symmetry 7y ago

SAT prep courses advertise that they can increase scores by 100 points but generally they're lying in order to sell more test courses. But the real number is closer to "11 to 15 points on the math section and 6 to 9 points on the verbal."[1] [1]https://www.researchgate.net/publication/228337033_Using_Linear_Regression_and_Propensity_Score_Matching_to_Estimate_the_Effect_of_Coaching_on_the_SAT https://www.researchgate.net/publication/228337033_Using_Lin...

thaumasiotes 7y ago
Note also that the measurement error of a section score is 30 points, which dwarfs the effect of prep.
asdfman123 7y ago
As a former SAT tutor (and a guy who gamed the hell out of the SAT) I have trouble believing that. There are only a few classes of questions they ask on the test, and if you master them the whole test is a walk in the park. Maybe the problem is most students just don't give a damn about the classes their parents make them take? Can someone explain the findings in this paper?
- graeme 7y ago
  My impression as a former SAT instructor is a lot of students don’t spend that much time prepping. And if they have no good reading/math background it can take some effort to master things. That said, typically saw increases well above that in my classes. Even with zero instruction I’d expect a self studying student to improve more than that from familiarization. It did say upper class people improve more.
- closeparen 7y ago
  The SAT is designed to be an intelligence test, not a course final, and it's possible that the large team of education and psychometrics people who've been working on it for decades know what they're doing. Armchair speculation: there are probably some really bright kids out there with poor test-taking skills, and prep likely has a dramatic effect on scores for them, but most students are not that. Did you actually sit an SAT before you started gaming? What was your improvement?

albntomat0 7y ago

How is that functionally different from most things considered by college admissions? Money helps a ton for normal class, extra-curriculars, etc.

realtalk_sp 7y ago

While it's true that prep courses can help dramatically, the SAT subject tests are also not really that hard. I don't consider myself terribly smart (certainly not MIT level) and I managed to get an 800 on the Math II and 800 on Chemistry with a minimum of studying using pretty inexpensive books. This applied to many of my friends as well.

UncleMeat 7y ago
This is one of the reasons the subject tests are not especially useful. And since they test knowledge rather than aptitude, making them more difficult doesn't actually find smarter people but instead people who had access to more rigorous high school coursework. At the GRE level this is even more extreme. For competitive programs a perfect score doesn't distinguish you so taking the test can only harm you. When applying to grad school I was explicitly advised not to take the GRE subject tests.
- applecrazy 7y ago
  The subject tests, IMO, are terrible. I took the Chemistry one right after taking AP (college level) Chemistry, and while I got a full score on that, I got absolutely destroyed on the subject test. They asked trivia-like questions and barely tested for actual chemistry knowledge.
  - saagarjha 7y ago
    Interestingly, I had the opposite experience, finding the AP Chemistry exam more difficult than the subject exam. I think part of it is that I have no love for chemistry, so I just memorized the things I needed for the subject test.
jimmyvalmer 7y ago
I was a child of the 80s, and while I agree the Math "Level 2" was a cakewalk, getting an 800 on Chem was definitely something to write home about. I get that scaling may have changed in the past thirty years, but still!

aliston 7y ago

The real scandal that I can't believe hasn't gotten more attention is that wealthy families are hiring psychologists to claim their child has a learning disability. They can then get more time to take the test and due to a policy change a few years ago, this information is hidden from the schools when students report their scores.

Spivak 7y ago
Look, faking a disability is deplorable but this stance is bonkers. I wish my parents had the money when I was growing up to get me checked and diagnose my ADHD. And of course you wouldn't want to report to the college if a person has a disability, are you trying to scarlet letter everyone who needs a little more time to read from dyslexia?
- hoka-one-one 7y ago
  As someone who has used similar accommodations (though not specifically on the SAT), yes. Dyslexia will hold you back in college. SAT is intended to measure odds of success in college.
  - Spivak 7y ago
    You're not wrong but I think that's due to how much of university is based around timed tests. The artificial difficulty is maddening. I was fortunate that my school's disability office had fangs and that the maths program wasn't super competitive and wanted students to learn more than get marks. My favorite professor was teaching number theory and always scheduled his classes for the end day and gave unlimited time for exams. Every time I would sit there for at least 5 hours (which wasn't super uncommon) and it was the nicest thing to not be freaking out and trying to force myself to focus.
    - hoka-one-one 7y ago
      I disagree that it's artificial. There's only so much time in a day. A programmer who can do a task in one hour will in the long run outsprint one who needs two. Yes, you can (like many, many others) spend an extra couple hours at work every day, but it'll all come back to center eventually. Sometimes, or even much of the time, you won't even have that option. These are times when people seriously depend on you to perform. It's not personal. I get that it can be stressful and I've been there but I don't kid myself that I'm as good as the people who can work quickly and under pressure.

asdfman123 7y ago

I've worked as an SAT prep tutor and I've worked in the oil industry. Actually, I felt way more guilt about my SAT tutoring work than the oil industry. The world runs on oil, but no one needs expensive SAT tutoring to give rich kids an edge over the poor ones.

Spivak 7y ago
This seems like such a weird stance since pretty much all formal education isn't free. Even without SAT prep rich families can still afford better schools, private tutors, set up apprenticeships. It's a weird world where people are mad that those with means are using them to educate themselves and their children. As an adult I choose to pay for lots of different forms of education that gives me "advantage" -- bleh -- over my friends, peers, coworkers, etc.. Why should I feel guilty that I have the means to buy culinary textbooks so I can bake better?
- asdfman123 7y ago
  I'm not angry at the people who want to help their kids, but at the system that we're making. Our "meritocracy" has become a caste system in which a small group of people are doing very well and everyone else is scrambling to make it. It's a big social problem. Look at how suicide rates in middle America has skyrocketed, look at how much more divisive our politics have become. I have no idea what the solution is, and can't say for sure where we should place the blame, but I have an innate distaste for the passing along of inherited advantages.
- kaibee 7y ago
  I think the difference is that, better schools and private tutors, are at least in theory, teaching useful skills and ultimately creating a better person at the end. Being good at the SATs doesn't have the same excuse, so its much more obvious when you're basically just paying to get a better score.
  - Spivak 7y ago
    Look, I don't disagree with you -- other than the score SAT prep is fantastically bad education investment. But I think this is a problem with the testing more than some larger class issue. We've rewarded learning a bunch of useless facts and test taking strategies. You could probably switch out the SAT for an entire test of movie trivia and you'd probably get the same distribution of performance since ACT/SAT just reward the people who study for it the most. You can't accidentally prepare for these tests by other schooling that's not geared for it.

neaden 7y ago

The question shouldn't be is the SAT unfairly biased towards families with money, since of course it is, but is the SAT less biased towards families with money then other criteria. Having parents with money and connections lets you get tutoring to increase your GPA, tutoring to write essays, connections to get internships, the free time to do clubs, training to excel in sports, and so many other things.

Spivak 7y ago
Right! I don't understand why people in this thread are apparently mad rich families for this? Like a family with means is doing all the things to raise a socialized, well rounded, experienced, well-read and tutored, worldly person. Like god damn I can't think of a better use for the money as a parent. I get that college admissions is a contest and performative but hot damn this thread is full of people mad at other people's happiness.
- int_19h 7y ago
  People are mad, because that happiness comes at their expense. Not because rich people prep their kids per se, but because society is a zero-sum game in many ways, and the end result is a de facto caste system. People are often willing to endure unfair hardships, if their children get a better chance. But if they know that not even their great-grandchildren are likely to be significantly better off, the focus shifts back to here and now.
- Dylan16807 7y ago
  > socialized, well rounded, experienced, well-read and tutored, worldly person Test prep doesn't help with any of that.

hackinthebochs 7y ago

I'm an underrepresented minority and I increased my SAT score by 200-300 from the PSAT to SAT, depending on how you want to measure the change. This was just by doing self-study with off the shelf test prep material. The classes aren't what increases your score, its the consistent structured study and ones innate potential. For some people, no amount of study is going to get them into the 90+th percentile. While others who have the capacity to do well but haven't had the best instruction over the years can cover a lot of ground with the right prep. But this is exactly what SAT is intended to measure, scholastic potential. That some people can increase scores dramatically through preparation does not indicate a failure of the test, but rather its success.

SkyBelow 7y ago
>But this is exactly what SAT is intended to measure, scholastic potential. But if you define the variable like this, you end up opening a can of worms. Do rich kids have more scholastic potential because they have better access to resources? Or should scholastic potential be the measurement of how a child performs assuming they had the same access to resources? Should children who belong to a racial/ethnic group that have social stereotypes of having more scholastic potential be defined as having more scholastic potential because of the effect of stereotypes and labels, or should the measure of scholastic potential be of the underlying potential if the effects of stereotypes and labels were equal? Normally measurements of potential aim to be to measure the factors least impacted by other factors, because if we account for those factors we are measuring actual instead of potential.
- askafriend 7y ago
  > Do rich kids have more scholastic potential because they have better access to resources? Yes. They do. It's maybe not the answer you were looking for, but if you are better prepared, you'll do better at the task at hand. Seems fairly straightforward. The next question would be how easy are those resources to access (in the case of the SAT)? And the answer is you can do pretty well with a $20 test prep book that you can order from Amazon. The rest is time, effort, awareness etc. Then we start getting into factors that are much harder to control or evaluate - parental involvement, guidance, nutrition, emotional support, etc.

sushid 7y ago

This reads more like a humblebrag to me than a thoughtful criticism against standardized testing.

indigochill 7y ago

The main thing that frustrated me about the SAT, at least when I took it, was that it essentially boiled down to an IQ test, not a true "scholastic aptitude" test. I did barely any prep (aside from, I think, a flip calendar with questions), but I scored 1510 out of 1600 at the time which qualified me for a prestigious program in my university. Then I narrowly avoided flunking out because I'd never actually learned to study or manage my time properly in high school because I'd never needed to until college. Study skills and time management seem like pretty important factors in a true measure of scholastic aptitude.

colinmhayes 7y ago
Similar story to you except I graduated with a 3.4 from a top 10 university without learning to study or manage my time. I guess the test was right?

hangonhn 7y ago

These advantages are more subtle than even that. I was shamed by some older fellow alums when I told people that my parents had hired someone to go over my application and essays. Their stance was that my parents gave me an unfair advantage (note, both of my parents are immigrants who did not even finish high school). My retort to them was point out that most of them didn't exactly grow up in the crime ridden rural parts of the world (quite the opposite in fact). Their parents paid a higher tax rate to support the world class public education they got, which they just assumed is the norm. That shut them up rather quickly. SAT prep classes or not, coming from a more advantageous social-economic background grants you so many benefits in many ways. My parents wouldn't have to hire someone to review my applications if my school had a good college counselor (50% of my classmates do not go on to college).

pbhjpbhj 7y ago
>coming from a more advantageous social-economic background grants you so many benefits in many ways // It's not all benefit: you can be taught to be lazy, not to be self-reliant, you can lack tenacity and resilience, you can have perverse expectations and a sheltered view of the World. Such things can turn out to be a hindrance to self-fulfillment.
- hangonhn 7y ago
  Yeah that's a fair point for sure and I've seen examples of those too. However, even then some well resourced parents will still clear a path for these children to get into selective schools as the recent college admissions scandals revealed.

losvedir 7y ago

100 points isn't that much. The average score is 1000-1100. The elite universities expect 1500. Tutoring, even if it gave you 100 points, isn't going to cross that gap. I'd say it's still relatively fair. Fairer than essentially any other admissions criterion: grades aren't comparable across schools, extra-curriculars are much more about money spent, etc.

saithound 7y ago

The SAT is not fair. By taking an expensive prep course, you can potentially increase your SAT score by 100 (although that's quite extreme, the average improvement attributed to SAT prep is closer to 20 points). That's a significant, but not earth-shattering improvement: it moves you from the 75th percentile to the 84th, or from the 97th percentile to the 99th. Now let's compare that to other criteria used in the college admissions process. It's way easier to have impressive and relevant extracurricular activities if you're rich and go to a good school. And unlike SAT prep, acquiring good extracurriculars will definitely take years of your time and money. Well-written admissions essay? MIT requires one. But you do it at home, instead of a tightly controlled testing center. So if you're rich enough, you can have ghost-written admissions essays. Needless to say, this process can turn even a completely worthless essay into an impressive one. Creative portfolio? Unless you're in the 99th percentile of musical talent, the difference between "I write songs that I play on an old guitar" and "an orchestra performed my composition" is money. The former is probably not even 75th percentile; the latter, probably 95+. Alumni parents who would be likely to donate big bucks? You don't need an expensive prep course to get that, yet it can provide your application with a much bigger boost than 100 SAT points. And unlike the SAT, if you don't have alumni parents to begin with, then you'll never get this boost, no matter how much extra work you put in. The SAT is not fair. But it's the fairest admissions criterion used by U.S. universities today. (note: MIT still relies on the SAT; today's announcement concerns the SAT Subject Tests)

lollercoasr2020 7y ago
Truth be told: the culture of those institutions is based on extremely synthetic virtue-signalling and superficial values and is very far from being meritocratic or open to social mobility. This might be great for rich kids, though in case you are ambitious, clever, gritty but come from not-so-privileged background (not necessarily low-class or third world - just temporary economic turmoil is enough) - you are going to hate every single second you spend there with all your heart. So maybe this way it is a win-win in the end. The aristocracy get their titles and the plebs their opportunities for self-realisation. No harm done.
commandlinefan 7y ago
> it's the fairest admissions criterion I have yet to see an "abolish standardized tests" type suggest a workable alternative (or even an unworkable one).

scythe 7y ago

The question isn't "can the SATs be gamed?", though. The question is "can the SATs be gamed more than whatever other criteria you use?". Simply analyzing one half of the equation won't produce a correct answer. Rich people can game standardized tests. They can also game grades. They can game admission essays. They can game interviews. If colleges admitted students based on brain fMRI data, I wouldn't be at all surprised to find out that rich people managed to exploit that, too. Are the SATs unfair? Absolutely. Are they more unfair than the alternative? I'm not convinced. I usually prefer the devil I know.

bermanoid 7y ago

That's a fair criticism of the general SAT - I used to teach those classes, and the test is very gameable in the sense that there really are "A FEW SIMPLE TRICKS!" to learn that have nothing to do with actually knowing the material in a useful way. But at least some of the subject SATs are not like that at all, specifically the math/science ones. There really aren't many tricks or traps, they really are like normal school tests (if multiple choice) where doing well on them requires you to know the material they cover. Nobody who is "good at tests" is going to 800 the physics one without knowing physics well enough that they'd do well in a freshman mechanics course, and someone who gets a 400 either slept through class or is going to struggle at college level.

CapmCrackaWaka 7y ago

I work for a company that collects aggregated SAT data. I was pretty upset when I discovered there is a very strong, direct correlation between family income and SAT score, with a credible spread of several hundred points between the average scores of the highest and lowest income groups. I knew that there would be a correlation, but seeing how strong it was was pretty depressing.

fastball 7y ago
Why is that depressing?
jimmyvalmer 7y ago
Just so we're clear: IQ does correlate with income, for obvious reasons.

koboll 7y ago

>And those expensive prep courses had the potential to increase scores over 100 points. I took one of those expensive scores and got the exact same score after the course as I did before the course. It turned out the cheap study book was every bit as effective as the expensive course.

JamesBarney 7y ago

You can get like 80% of the value of an expensive sat prep course from a cheap sat book. But money also buys you grades, gpa, extracurriculars, and recommendations.

devonkim 7y ago

I realized how much of a scam standardized testing was when I used to do the PSATs and increased my score mostly by using a different strategy rather than getting smarter or anything. The SAT prep class my high school offered for like $40 was a complete joke because they were designed for public school students where you get around 1250 and it's pretty good for state school admissions. Until I learned to treat the test as a game and understand its scoring I went from 1330 to 1520 simply by _not_ answering anything I wasn't fully confident with my answer - this is literally the opposite strategy I was instructed to do where they encouraged people to guess. To get to the higher end, you are best off never getting even a partial deduction for getting something wrong. When I got my best score, I only got 3 questions wrong but far more unanswered.

thedance 7y ago

What really increases your score is having a good night's sleep and a healthy breakfast before school every day of your life prior to taking the test. The test doesn't really measure aptitude because it can't control for those inputs.

lonelappde 7y ago
Those inputs cause aptitude.

manfredo 7y ago

Coming from someone who took said expensive prep courses, I can tell you that just taking practice exam after practice exam is easily 90% of the benefit. The test taking strategy they teach is something anyone with serious prospects of getting into MIT would have learned years ago.

outlace 7y ago

I think the courses mostly help with discipline in studying for the test. There’s nothing magical about them. I’ve taken many standardized exams with and without prep courses and it’s quite possible to do very well with inexpensive self study material if you’re incredibly disciplined about studying.

RedBeetDeadpool 7y ago

If you gave me a student who had enough motivation to succeed, I bet I could raise his score far far far more than just 100.

underpand 7y ago

> And those expensive prep courses had the potential to increase scores over 100 points. As compared with self-studying with free online practice tests and resources having the potential to increase scores over 100 points.