8 ms·

How Asian test-prep companies swiftly exposed the brand-new SAT

42 points by phsource 11y ago

epalmer 11y ago
It is time we abandon SAT and ACT tests.
- threatofrain 11y ago
  SAT II's are more predictive of college performance on their own than 4 years of high school grades, but standardized tests like these take a fraction of the time, as opposed to years of intimate teacher evaluations. The compromised operational security of these exams don't stop them from being more predictive than teachers. So if we want to retire these tests, then we need something better. What we should really be talking about is improving these tests and beefing up operational security.
  - EliRivers 11y ago
    Serious question; on what basis is "how well they will do at college" the right metric to use for selecting who to accept?
    - danieltillett 11y ago
      If you run a college then selecting people who will do well should maximise your graduation rate. Of course if this is what should be optimised is an open question.
      - adrianN 11y ago
        Or you could keep graduation rate constant and make the taught material harder.
        kuschku 11y ago
        Which will lead to a revolution if you have paid colleges. People apply, pay for a year or two, inevitably fail, but are now also 30k in debt. What do you think those people will do? Accepting anyone only works in countries without paid universities.
    - jasode 11y ago
      >on what basis is "how well they will do at college" the right metric use for selecting who to accept? Well, the underlying basis is the finite number of seats for students. If you have a limited number of slots, you must reject some applicants. Therefore, the question flipped around is, "what criteria to use to reject applicants?" That leads into what I guess your usage of "basis" is actually about: the philosophy of selecting on the "doing well" instead of some other criteria such as... "economic disadvantages?"... or "local student over out-of-state or foreign student?"..."athletic abilities?", etc. You'd have to expand on what alternatives you had in mind. I think the "doing well" criteria can be defended by filling up the limited seats with students who will get the most out of college. For example, a student taking "Physics I" needs to know what derivatives are (Calculus). Having some slots used up by students that should have been in remedial math courses (Algebra) instead of flunking out of PHYICS1 is unfair to the potential students who already have the baseline math skills required for solving physics equations. Multiply that example across the entire curriculum required for a degree and the "doing well" criteria seems justified. You want to reject potential students who are not able to succeed at college-level course work. Maybe tests like SAT are bad/flawed but so far, they appear to be the "least bad" tool to predict which students will do well in college. One can create institutions that don't care about the assessment of "doing well" as an entrance criteria as long as the students pay the fees. That signal is provided by many colleges[1] that don't require SAT/ACT scores. However, the prestige and respect of those institutions don't match the ones that do. That negative perception in the marketplace is not uncorrelated to their acceptance criteria. (E.g. University of Phoenix.) [1]http://www.collegexpress.com/lists/list/colleges-that-do-not-consider-standardized-test-scores/2083/ http://www.collegexpress.com/lists/list/colleges-that-do-not...
    - kaitai 11y ago
      An excellent question. In practice, that's not who we select. One college in the Midwest I know of selected 1) kids related to alumni, 2) people of a certain religion, 3) people who would add to the sports and music programs, 4) kids of color with good grades and high SAT scores, and last 5) people who didn't fit the above, especially from California, Texas, or Florida, who could pay full tuition. A major goal at some small colleges is balancing kids who will pay full price and kids who will do well at college, which creates a funny situation where you've got a well off heavy-majority-white party crowd with low test scores and a smaller and more diverse group of subsidized kids who bring many academic and non-academic talents (the music/sports/math/science/high school leadership crowd that may not have well-off parents).
    - jerf 11y ago
      Flip it around; how is it good to accept someone to college who does not have a good chance of doing well? How good is it to accept their money, probably put them in debt (even if not the catastrophic $100,000+ debt, even $20,000 debt can be quite a lot for a young person, and it's hard to get out of 4 years without at least that much), and claim very significant portions of some of the most important years of their life [1] if you already have good reason to believe they are unlikely to do well? (And remember we're judging lots of people so statistical bases are all we can use.) It may not be the only criterion of use, but "do we have good reason to believe they will succeed in college?" is at the very least a critical bar that ought to be passed. It's can be the difference between college as the empowering, education experience it's supposed to be, and college as the meat-grinder wealth-extraction tool it can be when you feed in students who shouldn't be there. [1]: Not necessarily "best", but being "earliest" in a world of compounding interest makes them quite valuable in their own way.
      - jtbigwoo 11y ago
        It's also true that the better less-selective (or non-selective) colleges provide more help than very selective colleges. I went to community college for a year and then went to a selective private college. The community college had far more help sessions, tutors, and study aids available. There was even a program at the community college that specifically sought out first-generation students to make sure they were getting extra support.
    - superuser2 11y ago
      I can try: universities differ in rigor and students differ in capability. A student gets the most out of education (and becomes more capable) when presented with a challenge just below his threshold. Put him in a program that's too hard, and he'll have a shitty time while learning nothing and wasting a lot of money before washing out. Put him in a program that's not hard enough, and he'll succeed within it but his growth will be suboptimal. Match students to the colleges that will be hard enough to get them to grow, but not so hard that they fail.
  - labster 11y ago
    It might not matter what is on the exams. SAT could just be testing for test-taking ability. I'd assume test-taking ability would be a good predictor of college performance too, as exams are typically 40-90% of college grades in my experience. Is there any data out there which tries to measure test skills independently?
    - threatofrain 11y ago
      What do you propose are the factors behind test-taking ability between-subjects? I would propose that a major contributor is g, or general mental ability, and I propose it is for that reason that the SAT II's have a non-trivial relationship to college performance, more than 4 years of high school grades.
    - Spooky23 11y ago
      They don't test for test-taking ability, they test for wherewithal to enroll in test prep classes. I can't speak to the current state of affairs, but in the mid 90s, taking a 5 week Kaplan course in my peer group raised scores from the 1150-1300 range to the 1300-1500 range. That meant the difference between average state school and high probability of hitting a "reach" school. Many of the ultra achievers on these tests start prep at age 10 or younger.
      - threatofrain 11y ago
        That just means that we should think about accessibility issues, but for whatever reason, teachers as a whole, despite intimate access for 4 years to students, are still less predictive of future performance than a test developed from afar.
      - thedufer 11y ago
        On my phone so I don't have the paper handy, but all available research shows no more than 50 point gains on SAT from test prep.
        seanmcdirmid 11y ago
  - guard-of-terra 11y ago
    "standardized tests like these take a fraction of the time" Preparing for standardized tests ruins a fraction of life.
    - logicchains 11y ago
      Is this a criticism, or a positive via the implication that high school ruins a much large fraction?
      - deleted 11y ago
        [deleted]
  - crispyambulance 11y ago
    SAT's are not used universally, nor are they the only factor in determining acceptance when they are used. Nor SHOULD they be. Whatever the case, it is not surprising that sketchy cram schools are cheating.
    - threatofrain 11y ago
      Whatever is the most performant set of predictive instruments should be used as a factor for acceptance for colleges seeking to optimize on whatever they feel are the "best" students. College essays? Sure, if the instrument is performant. SAT II's? Sure, if the instrument is performant. Grades? Sure, if the instrument is performant. Or extra-curriculars, or whatever.
      - crispyambulance 11y ago
        "Whatever" is certainly the case. College admissions at many of the better schools don't simply try to get "all the best" students that they can. They look to balance many factors, including admitting some fraction of students who have connections or can pay full tuition, including taking a risk on some students who would otherwise not be admitted but show some promise, including throttling-down the acceptance of over-represented groups. This is a fact. "predictive performant instruments" sure sounds very rigorous and objective, but you're talking about young people who are undergoing a lot of change. They're moving targets and predicting future performance based on one test only goes so far. A test like the SAT is always going to be a blunt instrument as is any singular factor. Admissions officers recognize this.
        superuser2 11y ago
        >or can pay full tuition Can you name some "better schools" whose admissions are not need-blind? My understanding is that this is pretty much universal among prestigious schools.
        crispyambulance 11y ago
  - jerf 11y ago
    "What we should really be talking about is improving these tests and beefing up operational security." We have a real-world instance of the DRM problem here. Once you've put a static exam physically into the hands of the students, it's compromised. The attackers have massively, massively more resources than the defenders. The attackers can hire a hundred students to each take one picture of one page, and put the test back together themselves. And there's no way that the proctors can prevent students sneaking one picture of one page nowadays. (Also, I'm not saying that's an optimal attack. I just put about 5 seconds' thought into it. But it's a sufficient attack already, so the existence of even further clever attacks only makes my point stronger.) Even presenting the problems one at a time on a screen (massively more expensive, btw, due to brand new hardware requirements, plus introduction of other attack vectors) would not be sufficient. Clever games played with contrast or whatever won't work because the test needs to be highly accessible, so you can't play games like the old copy protection booklets [1], either. Basically, if the test is static, it's game over. Not that students aren't fairly good at cheating with dynamic questions. But those really only work well with the math questions, and even the dumbed-down calculators they still sell today can be convinced to run programs sufficient to "crack" any dynamic problems that the test companies can get their hands on for the math portions. It's basically hopeless. The adversaries have gotten too darned powerful. They're armed with too much gear that would make old-school spies weep tears of joy. [1]: http://i.imgur.com/eU8Rvhh.jpg http://i.imgur.com/eU8Rvhh.jpg
  - 1337biz 11y ago
    Might be true, still trying to measure the academic potential of a highschool student by a private company is absurd. I am all for participants doing their very best to "hack" the system. Kudos to those companies whelping them by take up the battle with the testing industry.
- deleted 11y ago
  [deleted]
- VLM 11y ago
  Wait for the higher ed crash. All things that can't go on forever, eventually stop. At that point the ACT/SAT problem will automatically solve itself.
deleted 11y ago
[deleted]
dietrichepp 11y ago
> We’re working against cartel-like companies in China and other countries that will stop at nothing to enrich themselves, That's how I felt about College Board.
- jaflo 11y ago
  I still don't understand how then can call themselves a non-profit while charging ridiculous prizes and monopolizing the standardized test market.
  - MereInterest 11y ago
    Given that Ikea is legally run and registered as a non-profit charity, I think that the term is fairly meaningless at this point. We can only judge them by their actions, just as we do any other company.
    - manyxcxi 11y ago
      Not for profit and non-profit are actually different in different tax and law scenarios. I'm not going to try and butcher the explanation, but it pretty much boils down to how employees get paid and expenses get deducted- which, in my opinion, has made me look at not for profit companies as companies that want to appear nicer than they really are.
      - fulafel 11y ago
        That sounds like a US tax law distinction, but the IKEA structure is non-US (as are the owners/founders).
  - jccalhoun 11y ago
    they are super sketchy. https://en.wikipedia.org/wiki/College_Board#Criticism https://en.wikipedia.org/wiki/College_Board#Criticism
  - xlm1717 11y ago
    Also, helping a handful of calculator companies maintain a monopoly on the graphing calculator market.
guard-of-terra 11y ago
As a society we should totally rethink how we handle children. Employment is as personal and customizable as ever, but we increasingly treat kinds not even as blue-collar workers but as poultry on industrial farm. Stick them in cage with artificial light, force feed with unnatural excuse for knowledge, disregard their personality, rank on how good they pass tests (we weight poultry at this point)
- threatofrain 11y ago
  There's better education if you can afford it. There are schools where doctorates lecture the promising young on cryptography. There are schools with great faculty:student ratios. And great psychological resources. Owing to how few of such environments society can sustainably create, I think it would be implausible to suggest that all children warrant access to such limited resources. At the very least, it would call for a rethinking of what % of our employable society should be employed as teachers, and how and where our most elite teachers ought be allocated.
  - guard-of-terra 11y ago
    I don't see why such environments got to be scarce. What's the limiting resource here?
    - threatofrain 11y ago
      The limiting factor is talent, bureaucratic and pedagogical, and money.
      - deleted 11y ago
        [deleted]
    - atemerev 11y ago
      Teachers. Good teachers are rare and it's hard to measure teaching talent. Bad teachers are commonplace and unionized to teeth. Sounds like a perfect environment for a disruptive AI tech. Like Coursera, but better and fully automated. Or, leaving AI aside, at least another "Uber for X" app, where X = teaching.
wodenokoto 11y ago
Why don't they try and develop a model that takes into account that students will study directly against the test? The current model seems to rely on what could be called "security by obscurity". Of course students shouldn't be able to spend hours analysing a single text before the test, but if you publish a corpus of 200 texts, then anyone who read and analysed them thoroughly before the test aren't really cheating anymore, they are just educating themselves.
- ikeboy 11y ago
  No multiple choice test can protect against a list of questions and answers.
  - ATsch 11y ago
    America uses a multiple choice test for university admission?
    - ikeboy 11y ago
      Both the SAT and the ACT are mostly multiple choice, with one essay.
    - superuser2 11y ago
      Alongside high school transcript, essays, extracurricular activities, and demographics, yes. It's usually not a simple threshold or ranking, but it gives admissions officers a normalized view of something approximating intellectual horsepower (rigor and quality of high schools varies enormously and mostly responds to socioeconomic status of the neighborhood). SAT and ACT each have a timed handwritten essay component, but it's not very well-regarded. There is no long-form working of problems, though that does appear on AP/IB subject tests which are also necessary to be competitive at highly selective schools.
    - serge2k 11y ago
      The american education system seems obsessed with standardized tests.
roymurdock 11y ago
Saw an interesting proposal on HN a few months ago WRT entrance exams: My university just allows anyone into first semester, but then has quite hard courses and exams there already – by the end of the second semester, 90% have already failed. That’s another way to solve the >10x applicants issue. Seems like a great way to reduce administrative overhead, judge based on merit, and get students to realistically self-select a college (if you don't, you waste a semester and the associated costs). Problem is that the exams select only for people that are good at taking exams - not always the most interesting/well rounded students.
- ATsch 11y ago
  Germany has such a system in place, but not without limits. How important your Grade is depends on the subject you are studying, how popular it is and how well grades correlate with success. For example, you can not be sure of a spot in a medicine school unless you have neat-perfect grades, whereas engineering or practical sciences are nearly a free-for-all, with 90% leaving after the first semester. I don't believe a purely merit-based system could work on a large scale with popular subjects such as medicine.
- umanwizard 11y ago
  Sounds like how medical school in France works, or at least did while I was there. Anyone with an "academic" HS diploma (40% of people or so) could sign up, but the overwhelming majority failed the first year. Their options were repeat the first year or go into a less prestigious track like nursing
- deleted 11y ago
  [deleted]
- jtbigwoo 11y ago
  This doesn't sound like a great idea for the U.S. system. Students would have to pay tens of thousands of dollars and spend many precious months for an experience with a 90% failure rate. I have a hard time thinking that anyone with other options would accept that proposal. On top of that, even if you're in the select 10%, you're attending classes where almost all the other students are failing. I can't imagine that class discussions are going to be helpful for you. If this school was free or very cheap, maybe it's an option for some, but with the way things work in the U.S., this seems more like a scam than a college.
iambateman 11y ago
If a test is this easy to game, perhaps it's not a very good test. Does Math and Language really correlate to ability? We know the answer is no. We sent humans into space, we can do better.
kenshaw 11y ago
Has IBM/Google/etc attempted to plug Watson or any of the other advanced knowledge AIs into the SAT/ACT yet? I would be curious to know if they could consistently score a "high" score on tests without having previously seen the material (obviously excluding the essay portion).
thrwy_01 11y ago
The bottom line... with rampant cheating on standardized tests, fabricated school records, and massive "assistance" in college applications, it has become almost impossible to accurately measure the merit of most applicants from east asia. First order effects arising from this are not that big a deal, those students will either wash out in in the university setting, or will end up cheating their way through school to the degree that they will be useless to many employers afterwards. The bigger issues is that, whether due to malice or not, schools have limited enrollment spots and people who try to "play fair" will be left at a disadvantage. Some might say that this is not all bad; that it prepares students for the real world. But I would like to think that we want to protect our culture/society from devolving into the sort of low trust free for all that exists in some other parts of the world. Anecdotally I've heard that this shift has already started to permeate our universities, where many students feel immense pressure to cheat because so many other people are doing it. I'd be lying if I said that this didn't have a big effect on the way I view recent east asian immigrants in the workforce. Let you say I'm xenophobic, "recent east asian immigrant" would accurately describe many people in my family and social circles, and if I didn't open my mouth, the average bystander might assume I would be in that category as well. The college board and individual colleges are not exactly innocent bystanders in all of this, and I'm glad I got through the system before these issues became acute, but I feel really badly for students from anywhere who actually play by the rules.
tedchs 11y ago
Broken link.

42 points by phsource 11y ago

epalmer 11y ago

It is time we abandon SAT and ACT tests.

threatofrain 11y ago
SAT II's are more predictive of college performance on their own than 4 years of high school grades, but standardized tests like these take a fraction of the time, as opposed to years of intimate teacher evaluations. The compromised operational security of these exams don't stop them from being more predictive than teachers. So if we want to retire these tests, then we need something better. What we should really be talking about is improving these tests and beefing up operational security.
- EliRivers 11y ago
  Serious question; on what basis is "how well they will do at college" the right metric to use for selecting who to accept?
  - danieltillett 11y ago
    If you run a college then selecting people who will do well should maximise your graduation rate. Of course if this is what should be optimised is an open question.
    - adrianN 11y ago
      Or you could keep graduation rate constant and make the taught material harder.
      - kuschku 11y ago
        Which will lead to a revolution if you have paid colleges. People apply, pay for a year or two, inevitably fail, but are now also 30k in debt. What do you think those people will do? Accepting anyone only works in countries without paid universities.
  - jasode 11y ago
    >on what basis is "how well they will do at college" the right metric use for selecting who to accept? Well, the underlying basis is the finite number of seats for students. If you have a limited number of slots, you must reject some applicants. Therefore, the question flipped around is, "what criteria to use to reject applicants?" That leads into what I guess your usage of "basis" is actually about: the philosophy of selecting on the "doing well" instead of some other criteria such as... "economic disadvantages?"... or "local student over out-of-state or foreign student?"..."athletic abilities?", etc. You'd have to expand on what alternatives you had in mind. I think the "doing well" criteria can be defended by filling up the limited seats with students who will get the most out of college. For example, a student taking "Physics I" needs to know what derivatives are (Calculus). Having some slots used up by students that should have been in remedial math courses (Algebra) instead of flunking out of PHYICS1 is unfair to the potential students who already have the baseline math skills required for solving physics equations. Multiply that example across the entire curriculum required for a degree and the "doing well" criteria seems justified. You want to reject potential students who are not able to succeed at college-level course work. Maybe tests like SAT are bad/flawed but so far, they appear to be the "least bad" tool to predict which students will do well in college. One can create institutions that don't care about the assessment of "doing well" as an entrance criteria as long as the students pay the fees. That signal is provided by many colleges[1] that don't require SAT/ACT scores. However, the prestige and respect of those institutions don't match the ones that do. That negative perception in the marketplace is not uncorrelated to their acceptance criteria. (E.g. University of Phoenix.) [1]http://www.collegexpress.com/lists/list/colleges-that-do-not-consider-standardized-test-scores/2083/ http://www.collegexpress.com/lists/list/colleges-that-do-not...
  - kaitai 11y ago
    An excellent question. In practice, that's not who we select. One college in the Midwest I know of selected 1) kids related to alumni, 2) people of a certain religion, 3) people who would add to the sports and music programs, 4) kids of color with good grades and high SAT scores, and last 5) people who didn't fit the above, especially from California, Texas, or Florida, who could pay full tuition. A major goal at some small colleges is balancing kids who will pay full price and kids who will do well at college, which creates a funny situation where you've got a well off heavy-majority-white party crowd with low test scores and a smaller and more diverse group of subsidized kids who bring many academic and non-academic talents (the music/sports/math/science/high school leadership crowd that may not have well-off parents).
  - jerf 11y ago
    Flip it around; how is it good to accept someone to college who does not have a good chance of doing well? How good is it to accept their money, probably put them in debt (even if not the catastrophic $100,000+ debt, even $20,000 debt can be quite a lot for a young person, and it's hard to get out of 4 years without at least that much), and claim very significant portions of some of the most important years of their life [1] if you already have good reason to believe they are unlikely to do well? (And remember we're judging lots of people so statistical bases are all we can use.) It may not be the only criterion of use, but "do we have good reason to believe they will succeed in college?" is at the very least a critical bar that ought to be passed. It's can be the difference between college as the empowering, education experience it's supposed to be, and college as the meat-grinder wealth-extraction tool it can be when you feed in students who shouldn't be there. [1]: Not necessarily "best", but being "earliest" in a world of compounding interest makes them quite valuable in their own way.
    - jtbigwoo 11y ago
      It's also true that the better less-selective (or non-selective) colleges provide more help than very selective colleges. I went to community college for a year and then went to a selective private college. The community college had far more help sessions, tutors, and study aids available. There was even a program at the community college that specifically sought out first-generation students to make sure they were getting extra support.
  - superuser2 11y ago
    I can try: universities differ in rigor and students differ in capability. A student gets the most out of education (and becomes more capable) when presented with a challenge just below his threshold. Put him in a program that's too hard, and he'll have a shitty time while learning nothing and wasting a lot of money before washing out. Put him in a program that's not hard enough, and he'll succeed within it but his growth will be suboptimal. Match students to the colleges that will be hard enough to get them to grow, but not so hard that they fail.
- labster 11y ago
  It might not matter what is on the exams. SAT could just be testing for test-taking ability. I'd assume test-taking ability would be a good predictor of college performance too, as exams are typically 40-90% of college grades in my experience. Is there any data out there which tries to measure test skills independently?
  - threatofrain 11y ago
    What do you propose are the factors behind test-taking ability between-subjects? I would propose that a major contributor is g, or general mental ability, and I propose it is for that reason that the SAT II's have a non-trivial relationship to college performance, more than 4 years of high school grades.
  - Spooky23 11y ago
    They don't test for test-taking ability, they test for wherewithal to enroll in test prep classes. I can't speak to the current state of affairs, but in the mid 90s, taking a 5 week Kaplan course in my peer group raised scores from the 1150-1300 range to the 1300-1500 range. That meant the difference between average state school and high probability of hitting a "reach" school. Many of the ultra achievers on these tests start prep at age 10 or younger.
    - threatofrain 11y ago
      That just means that we should think about accessibility issues, but for whatever reason, teachers as a whole, despite intimate access for 4 years to students, are still less predictive of future performance than a test developed from afar.
    - thedufer 11y ago
      On my phone so I don't have the paper handy, but all available research shows no more than 50 point gains on SAT from test prep.
      - seanmcdirmid 11y ago
- guard-of-terra 11y ago
  "standardized tests like these take a fraction of the time" Preparing for standardized tests ruins a fraction of life.
  - logicchains 11y ago
    Is this a criticism, or a positive via the implication that high school ruins a much large fraction?
    - deleted 11y ago
      [deleted]
- crispyambulance 11y ago
  SAT's are not used universally, nor are they the only factor in determining acceptance when they are used. Nor SHOULD they be. Whatever the case, it is not surprising that sketchy cram schools are cheating.
  - threatofrain 11y ago
    Whatever is the most performant set of predictive instruments should be used as a factor for acceptance for colleges seeking to optimize on whatever they feel are the "best" students. College essays? Sure, if the instrument is performant. SAT II's? Sure, if the instrument is performant. Grades? Sure, if the instrument is performant. Or extra-curriculars, or whatever.
    - crispyambulance 11y ago
      "Whatever" is certainly the case. College admissions at many of the better schools don't simply try to get "all the best" students that they can. They look to balance many factors, including admitting some fraction of students who have connections or can pay full tuition, including taking a risk on some students who would otherwise not be admitted but show some promise, including throttling-down the acceptance of over-represented groups. This is a fact. "predictive performant instruments" sure sounds very rigorous and objective, but you're talking about young people who are undergoing a lot of change. They're moving targets and predicting future performance based on one test only goes so far. A test like the SAT is always going to be a blunt instrument as is any singular factor. Admissions officers recognize this.
      - superuser2 11y ago
        >or can pay full tuition Can you name some "better schools" whose admissions are not need-blind? My understanding is that this is pretty much universal among prestigious schools.
        crispyambulance 11y ago
- jerf 11y ago
  "What we should really be talking about is improving these tests and beefing up operational security." We have a real-world instance of the DRM problem here. Once you've put a static exam physically into the hands of the students, it's compromised. The attackers have massively, massively more resources than the defenders. The attackers can hire a hundred students to each take one picture of one page, and put the test back together themselves. And there's no way that the proctors can prevent students sneaking one picture of one page nowadays. (Also, I'm not saying that's an optimal attack. I just put about 5 seconds' thought into it. But it's a sufficient attack already, so the existence of even further clever attacks only makes my point stronger.) Even presenting the problems one at a time on a screen (massively more expensive, btw, due to brand new hardware requirements, plus introduction of other attack vectors) would not be sufficient. Clever games played with contrast or whatever won't work because the test needs to be highly accessible, so you can't play games like the old copy protection booklets [1], either. Basically, if the test is static, it's game over. Not that students aren't fairly good at cheating with dynamic questions. But those really only work well with the math questions, and even the dumbed-down calculators they still sell today can be convinced to run programs sufficient to "crack" any dynamic problems that the test companies can get their hands on for the math portions. It's basically hopeless. The adversaries have gotten too darned powerful. They're armed with too much gear that would make old-school spies weep tears of joy. [1]: http://i.imgur.com/eU8Rvhh.jpg http://i.imgur.com/eU8Rvhh.jpg
- 1337biz 11y ago
  Might be true, still trying to measure the academic potential of a highschool student by a private company is absurd. I am all for participants doing their very best to "hack" the system. Kudos to those companies whelping them by take up the battle with the testing industry.
deleted 11y ago
[deleted]
VLM 11y ago
Wait for the higher ed crash. All things that can't go on forever, eventually stop. At that point the ACT/SAT problem will automatically solve itself.

deleted 11y ago

[deleted]

dietrichepp 11y ago

> We’re working against cartel-like companies in China and other countries that will stop at nothing to enrich themselves, That's how I felt about College Board.

jaflo 11y ago
I still don't understand how then can call themselves a non-profit while charging ridiculous prizes and monopolizing the standardized test market.
- MereInterest 11y ago
  Given that Ikea is legally run and registered as a non-profit charity, I think that the term is fairly meaningless at this point. We can only judge them by their actions, just as we do any other company.
  - manyxcxi 11y ago
    Not for profit and non-profit are actually different in different tax and law scenarios. I'm not going to try and butcher the explanation, but it pretty much boils down to how employees get paid and expenses get deducted- which, in my opinion, has made me look at not for profit companies as companies that want to appear nicer than they really are.
    - fulafel 11y ago
      That sounds like a US tax law distinction, but the IKEA structure is non-US (as are the owners/founders).
- jccalhoun 11y ago
  they are super sketchy. https://en.wikipedia.org/wiki/College_Board#Criticism https://en.wikipedia.org/wiki/College_Board#Criticism
- xlm1717 11y ago
  Also, helping a handful of calculator companies maintain a monopoly on the graphing calculator market.

guard-of-terra 11y ago

As a society we should totally rethink how we handle children. Employment is as personal and customizable as ever, but we increasingly treat kinds not even as blue-collar workers but as poultry on industrial farm. Stick them in cage with artificial light, force feed with unnatural excuse for knowledge, disregard their personality, rank on how good they pass tests (we weight poultry at this point)

threatofrain 11y ago
There's better education if you can afford it. There are schools where doctorates lecture the promising young on cryptography. There are schools with great faculty:student ratios. And great psychological resources. Owing to how few of such environments society can sustainably create, I think it would be implausible to suggest that all children warrant access to such limited resources. At the very least, it would call for a rethinking of what % of our employable society should be employed as teachers, and how and where our most elite teachers ought be allocated.
- guard-of-terra 11y ago
  I don't see why such environments got to be scarce. What's the limiting resource here?
  - threatofrain 11y ago
    The limiting factor is talent, bureaucratic and pedagogical, and money.
    - deleted 11y ago
      [deleted]
  - atemerev 11y ago
    Teachers. Good teachers are rare and it's hard to measure teaching talent. Bad teachers are commonplace and unionized to teeth. Sounds like a perfect environment for a disruptive AI tech. Like Coursera, but better and fully automated. Or, leaving AI aside, at least another "Uber for X" app, where X = teaching.

wodenokoto 11y ago

Why don't they try and develop a model that takes into account that students will study directly against the test? The current model seems to rely on what could be called "security by obscurity". Of course students shouldn't be able to spend hours analysing a single text before the test, but if you publish a corpus of 200 texts, then anyone who read and analysed them thoroughly before the test aren't really cheating anymore, they are just educating themselves.

ikeboy 11y ago
No multiple choice test can protect against a list of questions and answers.
- ATsch 11y ago
  America uses a multiple choice test for university admission?
  - ikeboy 11y ago
    Both the SAT and the ACT are mostly multiple choice, with one essay.
  - superuser2 11y ago
    Alongside high school transcript, essays, extracurricular activities, and demographics, yes. It's usually not a simple threshold or ranking, but it gives admissions officers a normalized view of something approximating intellectual horsepower (rigor and quality of high schools varies enormously and mostly responds to socioeconomic status of the neighborhood). SAT and ACT each have a timed handwritten essay component, but it's not very well-regarded. There is no long-form working of problems, though that does appear on AP/IB subject tests which are also necessary to be competitive at highly selective schools.
  - serge2k 11y ago
    The american education system seems obsessed with standardized tests.

roymurdock 11y ago

Saw an interesting proposal on HN a few months ago WRT entrance exams: My university just allows anyone into first semester, but then has quite hard courses and exams there already – by the end of the second semester, 90% have already failed. That’s another way to solve the >10x applicants issue. Seems like a great way to reduce administrative overhead, judge based on merit, and get students to realistically self-select a college (if you don't, you waste a semester and the associated costs). Problem is that the exams select only for people that are good at taking exams - not always the most interesting/well rounded students.

ATsch 11y ago
Germany has such a system in place, but not without limits. How important your Grade is depends on the subject you are studying, how popular it is and how well grades correlate with success. For example, you can not be sure of a spot in a medicine school unless you have neat-perfect grades, whereas engineering or practical sciences are nearly a free-for-all, with 90% leaving after the first semester. I don't believe a purely merit-based system could work on a large scale with popular subjects such as medicine.
umanwizard 11y ago
Sounds like how medical school in France works, or at least did while I was there. Anyone with an "academic" HS diploma (40% of people or so) could sign up, but the overwhelming majority failed the first year. Their options were repeat the first year or go into a less prestigious track like nursing
deleted 11y ago
[deleted]
jtbigwoo 11y ago
This doesn't sound like a great idea for the U.S. system. Students would have to pay tens of thousands of dollars and spend many precious months for an experience with a 90% failure rate. I have a hard time thinking that anyone with other options would accept that proposal. On top of that, even if you're in the select 10%, you're attending classes where almost all the other students are failing. I can't imagine that class discussions are going to be helpful for you. If this school was free or very cheap, maybe it's an option for some, but with the way things work in the U.S., this seems more like a scam than a college.

iambateman 11y ago

If a test is this easy to game, perhaps it's not a very good test. Does Math and Language really correlate to ability? We know the answer is no. We sent humans into space, we can do better.

kenshaw 11y ago

Has IBM/Google/etc attempted to plug Watson or any of the other advanced knowledge AIs into the SAT/ACT yet? I would be curious to know if they could consistently score a "high" score on tests without having previously seen the material (obviously excluding the essay portion).

thrwy_01 11y ago

The bottom line... with rampant cheating on standardized tests, fabricated school records, and massive "assistance" in college applications, it has become almost impossible to accurately measure the merit of most applicants from east asia. First order effects arising from this are not that big a deal, those students will either wash out in in the university setting, or will end up cheating their way through school to the degree that they will be useless to many employers afterwards. The bigger issues is that, whether due to malice or not, schools have limited enrollment spots and people who try to "play fair" will be left at a disadvantage. Some might say that this is not all bad; that it prepares students for the real world. But I would like to think that we want to protect our culture/society from devolving into the sort of low trust free for all that exists in some other parts of the world. Anecdotally I've heard that this shift has already started to permeate our universities, where many students feel immense pressure to cheat because so many other people are doing it. I'd be lying if I said that this didn't have a big effect on the way I view recent east asian immigrants in the workforce. Let you say I'm xenophobic, "recent east asian immigrant" would accurately describe many people in my family and social circles, and if I didn't open my mouth, the average bystander might assume I would be in that category as well. The college board and individual colleges are not exactly innocent bystanders in all of this, and I'm glad I got through the system before these issues became acute, but I feel really badly for students from anywhere who actually play by the rules.

tedchs 11y ago

Broken link.