12 ms·

> I worry that the proposed changes will simply create a new market for consultants to help affluent parents prepare their children to ace that competition, too

by jal278 12y ago

> I worry that the proposed changes will simply create a new market for consultants to help affluent parents prepare their children to ace that competition, too. I wonder how well disadvantaged students and their parents would be able to navigate a complex and difficult maze of details, requirements and tasks. It's interesting how standardized tests seem to birth eco-systems of 'test-prep' designed to circumvent to some degree what the test is designed to measure. For example, how many of us simply memorized stacks of words to achieve higher scores on the SAT -- did that kind of memorization somehow quickly make us more 'prepared' for college? Of course not, it was just a way to game the metric. And many of us joined clubs in HS and attempted to become officers not because we really wanted to, but simply because it would look good on our college apps. While one could make the argument that ability and motivation to jump through these exploitative 'hoops' is indicative of ability to achieve in college, it is far from the original motivation of the test, and distorts high school life in strange ways; i.e. makes participation in H.S. clubs and organizations disingenuous, and needlessly wastes hours upon hours on vocab flash cards. More on topic: I'm not sure what distinguished the original test that the author took -- what made it less amenable to buying performance through specialized classes or courses -- but that perhaps is where the interesting research lies: In trying to create incorruptible tests, if that is even possible; for example, perhaps if the format and material changed wildly from year to year?

WalterBright 12y ago
> how many of us simply memorized stacks of words to achieve higher scores on the SAT I didn't. > And many of us joined clubs in HS and attempted to become officers not because we really wanted to, but simply because it would look good on our college apps. I did related things. But is that really a negative? I do a lot of things that I don't particularly want to, but are necessary for my career. So does everyone else with a career. Even the best job in the world is going to have scut work that needs doing despite not wanting to. Wouldn't you look negatively at a job applicant who eschewed doing things he didn't particularly want to do?
- burkaman 12y ago
  I didn't either, but "how many of us" really means "how common is it" and the answer is very common, so individual experience isn't that relevant. And doing things you don't want to is definitely not a negative, but trying to get a leadership position in something you have no interest in is. This isn't "I don't want to but it needs to be done", this is "I don't want to but I want to look like I want to". The "ideal" candidate was the president of the French club because they're passionate about the language and culture and their peers see them as a competent leader, not because they wanted to be president of something.
- untog 12y ago
  Wouldn't you look negatively at a job applicant who eschewed doing things he didn't particularly want to do? What about those that had no time to do? There are very bright kids out there that have to cook dinner when they get home. Or babysit their younger siblings. Or any and all of the above. Basically, the system rewards those who have the opportunity to do more, not the desire to.
  - WalterBright 12y ago
    That's true. I'd like to hear a plan for accurately admitting those with certain attributes that exhibit no signs of them whatsoever, because I can't think of one.
    - judk 12y ago
      In the absence of quality evidence, randomness is better than prejudicial bias.
      - xenadu02 12y ago
        In a way, randomness would be preferable because those with ability (who otherwise wouldn't meet the criteria or pass the tests) would get a chance to succeed. The ones that flunk out would make room for some more random attempts. Eventually the system would settle into a steady-state of X% of previously-found high performers and Y% of ever-rotating randoms, where occasionally one of the randoms moves into the high-performers group. This stuff also continues to reinforce and highlight how arbitrary and capricious success often is; many people are successful as much for their family/money/connections/opportunities as for their actual talent.
sliverstorm 12y ago
It would be hard to change the material dramatically year-over-year. For example the literary section is predicated upon you having read a number of books from a certain selection, and along with other things essentially tests that you are well-read. But if the scope of books they might inquire about is too broad, many test-takers (very well-read individuals included) will fail.
bradleyjg 12y ago
There was nothing fundamentally un-gamable about the tests they used in the early 80s, if anything on the contrary. But it was a very different time and culture. The notion that spending tens of thousands of dollars in test prep for a gifted and talented program was something a reasonable person would ever consider doing hadn't taken hold yet. Certainly not in NYC, which at the time was experiencing the height of "white flight", had just gone through a major funding crisis, and was in the midst of a crime wave.
yummyfajitas 12y ago
There may be an ecosystem designed to circumvent the test, but it's mostly snake oil. Test prep generates very small improvements in scores. http://marginalrevolution.com/marginalrevolution/2014/03/the-sat-test-prep-income-and-race.html http://marginalrevolution.com/marginalrevolution/2014/03/the... If test prep did work, it would reduce racial gaps since blacks do more test prep than average.
- bradleyjg 12y ago
  > Test prep generates very small improvements in scores. > If test prep did work, it would reduce racial gaps since blacks do more test prep than average. You have some evidence for either of these propositions vis-a-vis NYC specialized school admissions tests, or did you just want to grind your axe regardless of relevance?
  - yummyfajitas 12y ago
    Do you believe that this particular narrowly specialized test with little data available is wildly different from the mainstream tests on which we have data? If so, why?
    - bradleyjg 12y ago
      Is it really your claim that SAT prep rates by race are identical to prep rates for every narrow specialized test out there? No confounding factors having to do with how well known the SATs are in various communities as compared to other tests? None to do with public and charitable funding for SAT prep that doesn't exist for other tests? Nothing like that? It's a standardized test so let's extrapolate from whatever data happens to be handy? Do you have data on any narrow, specialized exam to posit this correlation with the SAT?
      - yummyfajitas 12y ago
        I don't claim it's identical, I claim it's the best point estimate we have.
mathattack 12y ago
It is amenable to test prep. Perhaps the one difference is that since it's on an off year (7th grade rather than 9th), is only for one school, and there aren't many spots, it isn't as widely prepped for as the 9th grade exam that covers the rest of the top schools.
JoshTriplett 12y ago
> For example, how many of us simply memorized stacks of words to achieve higher scores on the SAT -- did that kind of memorization somehow quickly make us more 'prepared' for college? Of course not, it was just a way to game the metric. Personally, I studied vocabulary extensively for the SAT and GRE, and retained the majority of it; that preparation made a major difference in my baseline everyday vocabulary. Not all test preparation needs to go to waste immediately after the test.
eli_gottlieb 12y ago
>For example, how many of us simply memorized stacks of words to achieve higher scores on the SAT -- did that kind of memorization somehow quickly make us more 'prepared' for college? Dear God, no. I just had that large a vocabulary.
- walshemj 12y ago
  :-) me to - at my junior school I was in the small 3-4 group that where allowed to go off and read by our selves as we where so far ahead. Probably get labelled as a trouble maker and be managed out of the school nowadays.
lotsofmangos 12y ago
Successfully gaming an aptitude test seems like a good demonstration of aptitude anyway.

by jal278 12y ago

WalterBright 12y ago

> how many of us simply memorized stacks of words to achieve higher scores on the SAT I didn't. > And many of us joined clubs in HS and attempted to become officers not because we really wanted to, but simply because it would look good on our college apps. I did related things. But is that really a negative? I do a lot of things that I don't particularly want to, but are necessary for my career. So does everyone else with a career. Even the best job in the world is going to have scut work that needs doing despite not wanting to. Wouldn't you look negatively at a job applicant who eschewed doing things he didn't particularly want to do?

burkaman 12y ago
I didn't either, but "how many of us" really means "how common is it" and the answer is very common, so individual experience isn't that relevant. And doing things you don't want to is definitely not a negative, but trying to get a leadership position in something you have no interest in is. This isn't "I don't want to but it needs to be done", this is "I don't want to but I want to look like I want to". The "ideal" candidate was the president of the French club because they're passionate about the language and culture and their peers see them as a competent leader, not because they wanted to be president of something.
untog 12y ago
Wouldn't you look negatively at a job applicant who eschewed doing things he didn't particularly want to do? What about those that had no time to do? There are very bright kids out there that have to cook dinner when they get home. Or babysit their younger siblings. Or any and all of the above. Basically, the system rewards those who have the opportunity to do more, not the desire to.
- WalterBright 12y ago
  That's true. I'd like to hear a plan for accurately admitting those with certain attributes that exhibit no signs of them whatsoever, because I can't think of one.
  - judk 12y ago
    In the absence of quality evidence, randomness is better than prejudicial bias.
    - xenadu02 12y ago
      In a way, randomness would be preferable because those with ability (who otherwise wouldn't meet the criteria or pass the tests) would get a chance to succeed. The ones that flunk out would make room for some more random attempts. Eventually the system would settle into a steady-state of X% of previously-found high performers and Y% of ever-rotating randoms, where occasionally one of the randoms moves into the high-performers group. This stuff also continues to reinforce and highlight how arbitrary and capricious success often is; many people are successful as much for their family/money/connections/opportunities as for their actual talent.

sliverstorm 12y ago

It would be hard to change the material dramatically year-over-year. For example the literary section is predicated upon you having read a number of books from a certain selection, and along with other things essentially tests that you are well-read. But if the scope of books they might inquire about is too broad, many test-takers (very well-read individuals included) will fail.

bradleyjg 12y ago

There was nothing fundamentally un-gamable about the tests they used in the early 80s, if anything on the contrary. But it was a very different time and culture. The notion that spending tens of thousands of dollars in test prep for a gifted and talented program was something a reasonable person would ever consider doing hadn't taken hold yet. Certainly not in NYC, which at the time was experiencing the height of "white flight", had just gone through a major funding crisis, and was in the midst of a crime wave.

yummyfajitas 12y ago

There may be an ecosystem designed to circumvent the test, but it's mostly snake oil. Test prep generates very small improvements in scores. http://marginalrevolution.com/marginalrevolution/2014/03/the-sat-test-prep-income-and-race.html http://marginalrevolution.com/marginalrevolution/2014/03/the... If test prep did work, it would reduce racial gaps since blacks do more test prep than average.

bradleyjg 12y ago
> Test prep generates very small improvements in scores. > If test prep did work, it would reduce racial gaps since blacks do more test prep than average. You have some evidence for either of these propositions vis-a-vis NYC specialized school admissions tests, or did you just want to grind your axe regardless of relevance?
- yummyfajitas 12y ago
  Do you believe that this particular narrowly specialized test with little data available is wildly different from the mainstream tests on which we have data? If so, why?
  - bradleyjg 12y ago
    Is it really your claim that SAT prep rates by race are identical to prep rates for every narrow specialized test out there? No confounding factors having to do with how well known the SATs are in various communities as compared to other tests? None to do with public and charitable funding for SAT prep that doesn't exist for other tests? Nothing like that? It's a standardized test so let's extrapolate from whatever data happens to be handy? Do you have data on any narrow, specialized exam to posit this correlation with the SAT?
    - yummyfajitas 12y ago
      I don't claim it's identical, I claim it's the best point estimate we have.

mathattack 12y ago

It is amenable to test prep. Perhaps the one difference is that since it's on an off year (7th grade rather than 9th), is only for one school, and there aren't many spots, it isn't as widely prepped for as the 9th grade exam that covers the rest of the top schools.

JoshTriplett 12y ago

> For example, how many of us simply memorized stacks of words to achieve higher scores on the SAT -- did that kind of memorization somehow quickly make us more 'prepared' for college? Of course not, it was just a way to game the metric. Personally, I studied vocabulary extensively for the SAT and GRE, and retained the majority of it; that preparation made a major difference in my baseline everyday vocabulary. Not all test preparation needs to go to waste immediately after the test.

eli_gottlieb 12y ago

walshemj 12y ago
:-) me to - at my junior school I was in the small 3-4 group that where allowed to go off and read by our selves as we where so far ahead. Probably get labelled as a trouble maker and be managed out of the school nowadays.

lotsofmangos 12y ago

Successfully gaming an aptitude test seems like a good demonstration of aptitude anyway.