3 ms·

And the paper specifically mentions that they correct for that by counting the scores higher for people who come from unprivileged backgrounds.

queuebert 3y ago
It would be interesting to regress out family income from scores and then correct the scores to the national median family income. I wonder if anyone does that.
- asoneth 3y ago
  I think the massive blowback the College Board got from "Adversity Scores" https://en.wikipedia.org/wiki/History_of_the_SAT#2019_introduction_and_abandonment_of_the_'Adversity_Score'_and_launching_of_'Landscape' https://en.wikipedia.org/wiki/History_of_the_SAT#2019_introd... would prevent them from doing that themselves. However they do offer a "Landscape" which provides similar details and many admissions programs use that to do something like what you're describing.
- jedberg 3y ago
  I'm pretty sure The College Board (makers of the SAT) have done that analysis. Not sure if they share it publicly, but I'll bet they share it with admissions officers.
webdood90 3y ago
I think you can argue that this still isn't fair.
- jedberg 3y ago
  Of course you can argue that. There is nothing that is perfectly fair. More fair would be to individually interview every college candidate with questions custom tailored their life experience via video chat where their face and voice is obscured as well as any accent and speech affectation they might have. But that certainly isn't scalable. I'd argue the SAT is one of the most fair things we have at scale. It's true that the SAT used to be biased (for example the word problems would use the word regatta which a wealthy east coast teen would know but a poor west coast teen would not) but from what I've seen they've done a good job removing that bias (as shown by this paper).
  - gnicholas 3y ago
    It would actually be really interesting if schools started using an AI-based live interview, where the interviewer (AI) was provided with their application file. Admissions officers could then watch the recording, or rely on the automated scoring that the AI platform spits out. Not everyone performs well on interviews, and I imagine performance would vary even more on interviews where you know your "interviewer" is an AI. But it could be really interesting, and a way to achieve the type of personalization and scale that you allude to.

queuebert 3y ago

It would be interesting to regress out family income from scores and then correct the scores to the national median family income. I wonder if anyone does that.

asoneth 3y ago
I think the massive blowback the College Board got from "Adversity Scores" https://en.wikipedia.org/wiki/History_of_the_SAT#2019_introduction_and_abandonment_of_the_'Adversity_Score'_and_launching_of_'Landscape' https://en.wikipedia.org/wiki/History_of_the_SAT#2019_introd... would prevent them from doing that themselves. However they do offer a "Landscape" which provides similar details and many admissions programs use that to do something like what you're describing.
jedberg 3y ago
I'm pretty sure The College Board (makers of the SAT) have done that analysis. Not sure if they share it publicly, but I'll bet they share it with admissions officers.

webdood90 3y ago

I think you can argue that this still isn't fair.

jedberg 3y ago
Of course you can argue that. There is nothing that is perfectly fair. More fair would be to individually interview every college candidate with questions custom tailored their life experience via video chat where their face and voice is obscured as well as any accent and speech affectation they might have. But that certainly isn't scalable. I'd argue the SAT is one of the most fair things we have at scale. It's true that the SAT used to be biased (for example the word problems would use the word regatta which a wealthy east coast teen would know but a poor west coast teen would not) but from what I've seen they've done a good job removing that bias (as shown by this paper).
- gnicholas 3y ago
  It would actually be really interesting if schools started using an AI-based live interview, where the interviewer (AI) was provided with their application file. Admissions officers could then watch the recording, or rely on the automated scoring that the AI platform spits out. Not everyone performs well on interviews, and I imagine performance would vary even more on interviews where you know your "interviewer" is an AI. But it could be really interesting, and a way to achieve the type of personalization and scale that you allude to.