3 ms·

Standardized testing is the benchmark for efficient signaling. Of course, it is in a college’s financial interest to insist that their “branded” signal is much

Standardized testing is the benchmark for efficient signaling. Of course, it is in a college’s financial interest to insist that their “branded” signal is much more valuable, since a test doesn’t care how you studied for it. A personal anecdote on this point: I enlisted in the US Navy after failing out of UIUC because I preferred to smoke weed and play video games instead of going to class. While on active duty I resumed my studies with online classes at military-friendly Strayer University, which accepted CLEP exams as credit for many lower level courses. I ended up taking like 8 different exams to take care of all my general requirements like sociology and Western civilization, each of which cost $0 thanks to military reimbursement.

gumby 3y ago
> Standardized testing is the benchmark for efficient signaling. I suppose it is indeed the benchmark, but standardized testing has mixed results. Aircraft pilot testing is, I believe, somewhat standardized and does demonstrate that at least when you it you met some baseline level of safety. The bar exam shows you have some baseline level of competency (as a non-lawyer who never read any law I am sure I would not pass) but I doubt it’s especially predictive as to how good a lawyer you will be. I took the SAT and got into a bunch of schools, but I was struck by how it didn’t seem to reflect much of anything we learned in school, except some of the math (we were not given any sat prep nor encouraged to get any). Note: I know MIT says SAT is somewhat predictive as to whether you will graduate, so…? I wonder about these full-cohort tests that control school admissions, such as in China, Korea, Japan, or France. Are the “winners” actually better in some useful way or is it simply a sorting hat that bins students into institutions with good/mediocre/bad networks?
- hibikir 3y ago
  I can tell you about the test in Spain: Ultimately it's actually testing what you learned in school, vs the SAT's IQ + Vocab proxy. Since it asks for so much, in practice it's testing for the quality of your school too, as if your classmates in high school were bad enough, or your teacher cared little about covering the material, you might not have even covered some of the questions. So in practice, some schools averaged extremely good results in some subjects even for the worst students, while in others the very best always scored badly: Imagine in the foreign language test, one school might have a teacher that could actually communicate well in English, while others didn't. Another fun part is that it limits how much a motivated high school can do regarding unfair grading: In my year, the average grades in high school and the grades in the University test, while relatively correlated, had some very big outliers. Do note that in Spain, more than a matter of which university you'll go to, it's also a matter of which major you'll be allowed to study. In general, high paying, low unemployment majors are basically unachievable by someone that tested badly, or had bad high school grades. So it's not a matter if you are going to the right university, but whether you can study anything you care about at all. I had friends that liked computer science, but were C students, so they had no prayer of studying CS. And also note that, even after you get into college, chances of flunking out are very high. About 15% of students are expelled every year from that CS program, to which you have to add the ones that drop out voluntarily. So more than sorting people into colleges, there's a whole lot of sorting into just getting a useful degree in a timely fashion.
- dmoy 3y ago
  > as in China, Korea, Japan, or France. Are the “winners” actually better in some useful way or is it simply a sorting hat that bins students into institutions with good/mediocre/bad networks? I can't speak to the latter three, but for China it's in large part whether you tested into a good middle school (at age 10 or whatever), and then tested into a good high school, and then spent a sufficient amount of time (read: 20hrs/day) cramming in the ~year before the exam supported by your good high school. Or have rich parents to pay your way into the good high school, or pay for top end tutoring / cram school if that route of corruption isn't available. Or I guess live in an area where the gap between <City Name> #1 High School and the next one down isn't as stark
  - gumby 3y ago
    > I can't speak to the latter three, but for China it's in large part whether you tested into a good middle school (at age 10 or whatever), and then tested into a good high school, and then spent a sufficient amount of time (read: 20hrs/day) cramming in the ~year before the exam supported by your good high school. > Or have rich parents So sounds like a useful test then: actual knowledge or skill is at best second order, but ability to fit into the upper tier networks has been demonstrated.
eru 3y ago
> Standardized testing is the benchmark for efficient signaling. No, it's not. And your anecdote proves it. Or rather, standardized testing just show how well you do on an exam. That means it tests how smart you are, and how well you can study the material on the test. But (according to Caplan) that's barely the point in the kind of signalling education is good for. You could mostly just give people an IQ test as part of your hiring process. Education takes years doing mostly pointless tasks, so that you can signal your long term _conformity_. You might be able to see how an employer might value someone who can credibly show that they can be a good cog in the machine for a long time (even if they don't see the point of the machine).
- rufus_foreman 3y ago
  >> You could mostly just give people an IQ test as part of your hiring process Not allowed. Griggs v. Duke Power Co. "What Congress has forbidden is giving these devices and mechanisms controlling force unless they are demonstrably a reasonable measure of job performance. Congress has not commanded that the less qualified be preferred over the better qualified simply because of minority origins. Far from disparaging job qualifications as such, Congress has made such qualifications the controlling factor, so that race, religion, nationality, and sex become irrelevant. What Congress has commanded is that any tests used must measure the person for the job and not the person in the abstract." So I guess under that decision, they considered IQ tests as measuring the person in the abstract.
  - eru 3y ago
    Eh, that decision hasn't hindered companies too much in practice. Eg software companies still give you 'programming problems' that are essentially badly designed IQ tests in disguise. If I remember right, Bryan Caplan addresses this court case in his book already.
    - rufus_foreman 3y ago
      >> software companies still give you 'programming problems' They've had to come up with all these ridiculous interview techniques when a simple "what's your IQ" would suffice.

gumby 3y ago

> Standardized testing is the benchmark for efficient signaling. I suppose it is indeed the benchmark, but standardized testing has mixed results. Aircraft pilot testing is, I believe, somewhat standardized and does demonstrate that at least when you it you met some baseline level of safety. The bar exam shows you have some baseline level of competency (as a non-lawyer who never read any law I am sure I would not pass) but I doubt it’s especially predictive as to how good a lawyer you will be. I took the SAT and got into a bunch of schools, but I was struck by how it didn’t seem to reflect much of anything we learned in school, except some of the math (we were not given any sat prep nor encouraged to get any). Note: I know MIT says SAT is somewhat predictive as to whether you will graduate, so…? I wonder about these full-cohort tests that control school admissions, such as in China, Korea, Japan, or France. Are the “winners” actually better in some useful way or is it simply a sorting hat that bins students into institutions with good/mediocre/bad networks?

hibikir 3y ago
I can tell you about the test in Spain: Ultimately it's actually testing what you learned in school, vs the SAT's IQ + Vocab proxy. Since it asks for so much, in practice it's testing for the quality of your school too, as if your classmates in high school were bad enough, or your teacher cared little about covering the material, you might not have even covered some of the questions. So in practice, some schools averaged extremely good results in some subjects even for the worst students, while in others the very best always scored badly: Imagine in the foreign language test, one school might have a teacher that could actually communicate well in English, while others didn't. Another fun part is that it limits how much a motivated high school can do regarding unfair grading: In my year, the average grades in high school and the grades in the University test, while relatively correlated, had some very big outliers. Do note that in Spain, more than a matter of which university you'll go to, it's also a matter of which major you'll be allowed to study. In general, high paying, low unemployment majors are basically unachievable by someone that tested badly, or had bad high school grades. So it's not a matter if you are going to the right university, but whether you can study anything you care about at all. I had friends that liked computer science, but were C students, so they had no prayer of studying CS. And also note that, even after you get into college, chances of flunking out are very high. About 15% of students are expelled every year from that CS program, to which you have to add the ones that drop out voluntarily. So more than sorting people into colleges, there's a whole lot of sorting into just getting a useful degree in a timely fashion.
dmoy 3y ago
> as in China, Korea, Japan, or France. Are the “winners” actually better in some useful way or is it simply a sorting hat that bins students into institutions with good/mediocre/bad networks? I can't speak to the latter three, but for China it's in large part whether you tested into a good middle school (at age 10 or whatever), and then tested into a good high school, and then spent a sufficient amount of time (read: 20hrs/day) cramming in the ~year before the exam supported by your good high school. Or have rich parents to pay your way into the good high school, or pay for top end tutoring / cram school if that route of corruption isn't available. Or I guess live in an area where the gap between <City Name> #1 High School and the next one down isn't as stark
- gumby 3y ago
  > I can't speak to the latter three, but for China it's in large part whether you tested into a good middle school (at age 10 or whatever), and then tested into a good high school, and then spent a sufficient amount of time (read: 20hrs/day) cramming in the ~year before the exam supported by your good high school. > Or have rich parents So sounds like a useful test then: actual knowledge or skill is at best second order, but ability to fit into the upper tier networks has been demonstrated.

eru 3y ago

> Standardized testing is the benchmark for efficient signaling. No, it's not. And your anecdote proves it. Or rather, standardized testing just show how well you do on an exam. That means it tests how smart you are, and how well you can study the material on the test. But (according to Caplan) that's barely the point in the kind of signalling education is good for. You could mostly just give people an IQ test as part of your hiring process. Education takes years doing mostly pointless tasks, so that you can signal your long term _conformity_. You might be able to see how an employer might value someone who can credibly show that they can be a good cog in the machine for a long time (even if they don't see the point of the machine).

rufus_foreman 3y ago
>> You could mostly just give people an IQ test as part of your hiring process Not allowed. Griggs v. Duke Power Co. "What Congress has forbidden is giving these devices and mechanisms controlling force unless they are demonstrably a reasonable measure of job performance. Congress has not commanded that the less qualified be preferred over the better qualified simply because of minority origins. Far from disparaging job qualifications as such, Congress has made such qualifications the controlling factor, so that race, religion, nationality, and sex become irrelevant. What Congress has commanded is that any tests used must measure the person for the job and not the person in the abstract." So I guess under that decision, they considered IQ tests as measuring the person in the abstract.
- eru 3y ago
  Eh, that decision hasn't hindered companies too much in practice. Eg software companies still give you 'programming problems' that are essentially badly designed IQ tests in disguise. If I remember right, Bryan Caplan addresses this court case in his book already.
  - rufus_foreman 3y ago
    >> software companies still give you 'programming problems' They've had to come up with all these ridiculous interview techniques when a simple "what's your IQ" would suffice.