4 ms·

I don't know how the industry could claim the SAT isn't coachable without knowing full well they were lying. The parents, teachers, and students knew when I was

I don't know how the industry could claim the SAT isn't coachable without knowing full well they were lying. The parents, teachers, and students knew when I was in high school 20 years ago, yet it was a secret to the industry? These people are the gatekeepers to a higher education, they should be held to higher standards.

rayiner 6y ago
The SAT isn’t really coachable for most people: https://www.wsj.com/articles/SB124278685697537839 https://www.wsj.com/articles/SB124278685697537839 > The college counselors' report concludes that, on average, prep courses yield only a modest benefit, "contrary to the claims made by many test-preparation providers." It found that SAT coaching resulted in about 30 points in score improvement on the SAT, out of a possible 1600, and less than one point out of a possible 36 on the ACT, the other main college-entrance exam, says Derek Briggs, chairman of the research and methodology department at the University of Colorado in Boulder and author of the admissions counselors' report. https://slate.com/technology/2019/04/sat-prep-courses-do-they-work-bias.html https://slate.com/technology/2019/04/sat-prep-courses-do-the... > Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University. > Some estimates run a little higher: A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37.
- an_opabinia 6y ago
  The exam is totally coachable, you just have to do cram school. Those studies look at private tutoring which spends possibly 1/20th of the time kids spend in cram school. Cram school is often conflated with private tutoring but they are completely different things.
  - caddemon 6y ago
    Sure but the amount of time you have to spend on cram school starts to make it not worth the points gain. The top schools all reject a bunch of 1600 scorers every year. If all you have going for you are good grades + good SAT you probably aren't going to get into an elite place. Sure it'll help you get into a tier 2 school, but at that point you're usually better off with a public option.
    - learc83 6y ago
      That’s the point of the entire argument. Elite schools don’t go by scores and grades alone. Parents who sent their kids to cram school, or tutoring or whatever, and kids with perfect scores are mad that schools aren’t basing admissions solely on scores and grades. What it boils down to is that Asian American kids are outperforming most other groups on the SAT for whatever reason. Elite colleges don’t want student bodies that are 70% Asian American.
      - caddemon 6y ago
        Elite colleges also don't want the type of people that will spend 4 hours a day cramming for an uninteresting test.
        an_opabinia 6y ago
        Yes, but elite public high schools do. Which feed the elite colleges. Also, middle schoolers don’t really have a choice in the matter. If it were up to them, they’d play Roblox and watch TikToks all day.
        learc83 6y ago
        No they probably don’t. That’s one side of the argument.
- jessedhillon 6y ago
  Well what they do is move people from the 96th to the 98th percentile without a meaningful increase in skill or attitude. The bigger issue is that schools have their heads up their asses when they insist on only selecting from the 99th percentile, when slumming it in and going down to the 93rd would be more meritorious and not negligible to their graduation rates.
- choeger 6y ago
  I am an outsider to that whole system, but when the SAT is not coachable, then what is the purpose of schools?
  - theamk 6y ago
    This talks about coaching for SAT specifically, as opposed to general education. For example, looking at specific question formats; learning the only topics that would be covered by tests and nothing else.
    - learc83 6y ago
      Specifically 1-3 hours a week for a few months isn’t enough to dramatically improve the average score. That’s all the referenced studies show. That doesn’t mean that the SAT isn’t “coachable” with much intensive efforts, or that more minimal efforts don’t result in large improvements for some subset of test takers.
- learc83 6y ago
  1. The results you list are just average. There could be a bimodal distribution. 2. What those studies call coaching isn’t remotely comparable to years of 20 hour a week cram school. 3. Of course the SAT is coachable. How do you think you learn the math techniques and vocabulary in the first place? If it’s not possible to increase your vocabulary, how do people learn new languages? 4. The percentage of students who make a perfect score on a similar test, the ACT, has increased by 5x in the last five years while the average score has only risen slightly. Clearly a subset of test takers has figured out a way to reliably prep for the test.
- csa 6y ago
  > The SAT isn’t really coachable for most people Based on info from a buddy of mine who runs an SAT prep business, this is a half truth. He says: - It is tough to get gains from a kid who doesn’t want to be there — that is, the parents are pushing the students to prep when the student doesn’t want to. My friend actually tries to filter out these folks in his sales funnel. - Motivated kids can get ridiculous amounts of improvement (40-100 per skill), especially if they are starting off in the lower or middle part of the scoring range (people at the top are capped in improvement by max scores). - I have done some minimal tutoring for a friend’s kid, and his SAT score went up by 110 in math and 80 in verbal from a middling starting point. Anecdote, but still... My “curriculum” was just have him take a practice test every week under test like conditions, review the principles behind the math problems that he missed, review any test strategy issues that pop up, and give him 50 vocab words a week with skills on how to search for how these words are used in a natural setting. With a longer runway, I think he may have been able to double that improvement (the kid was smart enough to get a perfect math score, but didn’t have the confidence to execute in a testing context).
  - deleted 6y ago
    [deleted]
  - 2sk21 6y ago
    I don't know why you're getting downvoted. You are correct. It's very much possible to improve scores by practice. By simply taking a lot of practice tests over the summer, my daughter was able to substantially improve her ACT score from 30 to 34 - which is in fact what she got in the actual test.
- calibas 6y ago
  A single second is a very small amount of time, but in a race it can be the difference between winning and losing. When you're in a competition with a thousand other people whose scores are almost the same as yours, 30 points can be huge. And it should be noted that they changed the tests before these studies too. > Many of the key studies of coaching effects were based on kids who took the SAT or ACT in the 1990s and 2000s, Briggs told me. The tests have since been changed a fair amount, in an effort to make them more like statewide achievement tests and to minimize their similarity to intelligence tests.
- tptacek 6y ago
  Just a quick note than your analysis (here, I don't know about elsewhere) depends on the effectiveness of test prep courses, but those aren't the only advantage well-off student have in the SAT; other advantages, all of which might be more important than coaching, include stuff as simple as "expectations and forcing functions to take the SAT more than one" and as lurid as "phony IEPs to enable the taking of the test over multiple days", which is a thing. "The SAT isn't gameable" is an extraordinary claim. And "suburban families try hard to game the SAT" is not so much a "claim" as "a fact your own analysis acknowledges". I could be persuaded, but what you've presented here doesn't constitute extraordinary evidence. (Your other argument in this thread, about the possible Cobra Effect of eliminating the SAT, is much more persuasive).
RestlessMind 6y ago
That is only if you focus on the curriculum covered in SAT. If you instead see SAT as a proxy for the eventual success in the society, then I believe SAT is a great indicator of the factors which lead to this success - family wealth, willingness of parents to invest in their children, diligence and rote work required to ace such tests, ability to find loopholes to strategize and game the system (we love "hackers" here, no?) etc etc. In other words, if someone is going to succeed because of SAT, they will succeed even if you remove SAT from the picture.

rayiner 6y ago

The SAT isn’t really coachable for most people: https://www.wsj.com/articles/SB124278685697537839 https://www.wsj.com/articles/SB124278685697537839 > The college counselors' report concludes that, on average, prep courses yield only a modest benefit, "contrary to the claims made by many test-preparation providers." It found that SAT coaching resulted in about 30 points in score improvement on the SAT, out of a possible 1600, and less than one point out of a possible 36 on the ACT, the other main college-entrance exam, says Derek Briggs, chairman of the research and methodology department at the University of Colorado in Boulder and author of the admissions counselors' report. https://slate.com/technology/2019/04/sat-prep-courses-do-they-work-bias.html https://slate.com/technology/2019/04/sat-prep-courses-do-the... > Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University. > Some estimates run a little higher: A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37.

an_opabinia 6y ago
The exam is totally coachable, you just have to do cram school. Those studies look at private tutoring which spends possibly 1/20th of the time kids spend in cram school. Cram school is often conflated with private tutoring but they are completely different things.
- caddemon 6y ago
  Sure but the amount of time you have to spend on cram school starts to make it not worth the points gain. The top schools all reject a bunch of 1600 scorers every year. If all you have going for you are good grades + good SAT you probably aren't going to get into an elite place. Sure it'll help you get into a tier 2 school, but at that point you're usually better off with a public option.
  - learc83 6y ago
    That’s the point of the entire argument. Elite schools don’t go by scores and grades alone. Parents who sent their kids to cram school, or tutoring or whatever, and kids with perfect scores are mad that schools aren’t basing admissions solely on scores and grades. What it boils down to is that Asian American kids are outperforming most other groups on the SAT for whatever reason. Elite colleges don’t want student bodies that are 70% Asian American.
    - caddemon 6y ago
      Elite colleges also don't want the type of people that will spend 4 hours a day cramming for an uninteresting test.
      - an_opabinia 6y ago
        Yes, but elite public high schools do. Which feed the elite colleges. Also, middle schoolers don’t really have a choice in the matter. If it were up to them, they’d play Roblox and watch TikToks all day.
      - learc83 6y ago
        No they probably don’t. That’s one side of the argument.
jessedhillon 6y ago
Well what they do is move people from the 96th to the 98th percentile without a meaningful increase in skill or attitude. The bigger issue is that schools have their heads up their asses when they insist on only selecting from the 99th percentile, when slumming it in and going down to the 93rd would be more meritorious and not negligible to their graduation rates.
choeger 6y ago
I am an outsider to that whole system, but when the SAT is not coachable, then what is the purpose of schools?
- theamk 6y ago
  This talks about coaching for SAT specifically, as opposed to general education. For example, looking at specific question formats; learning the only topics that would be covered by tests and nothing else.
  - learc83 6y ago
    Specifically 1-3 hours a week for a few months isn’t enough to dramatically improve the average score. That’s all the referenced studies show. That doesn’t mean that the SAT isn’t “coachable” with much intensive efforts, or that more minimal efforts don’t result in large improvements for some subset of test takers.
learc83 6y ago
1. The results you list are just average. There could be a bimodal distribution. 2. What those studies call coaching isn’t remotely comparable to years of 20 hour a week cram school. 3. Of course the SAT is coachable. How do you think you learn the math techniques and vocabulary in the first place? If it’s not possible to increase your vocabulary, how do people learn new languages? 4. The percentage of students who make a perfect score on a similar test, the ACT, has increased by 5x in the last five years while the average score has only risen slightly. Clearly a subset of test takers has figured out a way to reliably prep for the test.
csa 6y ago
> The SAT isn’t really coachable for most people Based on info from a buddy of mine who runs an SAT prep business, this is a half truth. He says: - It is tough to get gains from a kid who doesn’t want to be there — that is, the parents are pushing the students to prep when the student doesn’t want to. My friend actually tries to filter out these folks in his sales funnel. - Motivated kids can get ridiculous amounts of improvement (40-100 per skill), especially if they are starting off in the lower or middle part of the scoring range (people at the top are capped in improvement by max scores). - I have done some minimal tutoring for a friend’s kid, and his SAT score went up by 110 in math and 80 in verbal from a middling starting point. Anecdote, but still... My “curriculum” was just have him take a practice test every week under test like conditions, review the principles behind the math problems that he missed, review any test strategy issues that pop up, and give him 50 vocab words a week with skills on how to search for how these words are used in a natural setting. With a longer runway, I think he may have been able to double that improvement (the kid was smart enough to get a perfect math score, but didn’t have the confidence to execute in a testing context).
- deleted 6y ago
  [deleted]
- 2sk21 6y ago
  I don't know why you're getting downvoted. You are correct. It's very much possible to improve scores by practice. By simply taking a lot of practice tests over the summer, my daughter was able to substantially improve her ACT score from 30 to 34 - which is in fact what she got in the actual test.
calibas 6y ago
A single second is a very small amount of time, but in a race it can be the difference between winning and losing. When you're in a competition with a thousand other people whose scores are almost the same as yours, 30 points can be huge. And it should be noted that they changed the tests before these studies too. > Many of the key studies of coaching effects were based on kids who took the SAT or ACT in the 1990s and 2000s, Briggs told me. The tests have since been changed a fair amount, in an effort to make them more like statewide achievement tests and to minimize their similarity to intelligence tests.
tptacek 6y ago
Just a quick note than your analysis (here, I don't know about elsewhere) depends on the effectiveness of test prep courses, but those aren't the only advantage well-off student have in the SAT; other advantages, all of which might be more important than coaching, include stuff as simple as "expectations and forcing functions to take the SAT more than one" and as lurid as "phony IEPs to enable the taking of the test over multiple days", which is a thing. "The SAT isn't gameable" is an extraordinary claim. And "suburban families try hard to game the SAT" is not so much a "claim" as "a fact your own analysis acknowledges". I could be persuaded, but what you've presented here doesn't constitute extraordinary evidence. (Your other argument in this thread, about the possible Cobra Effect of eliminating the SAT, is much more persuasive).

RestlessMind 6y ago

That is only if you focus on the curriculum covered in SAT. If you instead see SAT as a proxy for the eventual success in the society, then I believe SAT is a great indicator of the factors which lead to this success - family wealth, willingness of parents to invest in their children, diligence and rote work required to ace such tests, ability to find loopholes to strategize and game the system (we love "hackers" here, no?) etc etc. In other words, if someone is going to succeed because of SAT, they will succeed even if you remove SAT from the picture.