5 ms·

Acceptance rates is a really poor choice of input for college ranking It's basically saying "other people seem to think this college is good, so we'll nudge th

Acceptance rates is a really poor choice of input for college ranking It's basically saying "other people seem to think this college is good, so we'll nudge them up a few points" In other words, it's attempting to measure public perception about the college instead of the actual results it produces. How the public _thinks_ a certain college is shouldn't factor into how good you _tell_ the public that that college performs

troupe 4y ago
Imagine college A that 100 apply for and all 100 get in vs. college B where 10,000 people applied and only 100 got in. If both colleges are choosing students based off of some measurable criteria, let's say the SAT score. Would you expect to have higher average scores at college A or college B? Obviously college B would have students with better SAT scores simply because they are so much more selective. Assuming that the college is selecting students for attributes that you are looking for in fellow students, then the college that denies more people will have students with more of the attributes you are wanting to be around. So it isn't just about perception. There is some math behind it as well.
- highwaylights 4y ago
  Even so, you don’t need to guess the reasons for acceptance on aggregate data - you can check if the SAT scores were higher. At least that would tell you for certain.
- ipaddr 4y ago
  It depends if school A costs 100,000 and school B almost free. I would put my money on the 100 who spent 100,000 over the free 10,000.
- HWR_14 4y ago
  > If both colleges are choosing students based off of some measurable criteria, let's say the SAT score. Would you expect to have higher average scores at college A or college B? Except "average SAT score of incoming class" is another field of the rankings. All the measurable criteria are handled as independent variables - there's no need for a poor proxy.
DenisM 4y ago
> It's basically saying "other people seem to think this college is good, so we'll nudge them up a few points" So does page rank or citation index - popularity begets visibility, begets popularity.
- simonh 4y ago
  The difference is that applicants by definition don’t have personal experience of how good the school is, it’s all perception. With page rank and citations the expectation is those linking to or citing something have read it and have reason to value it.
  - Spivak 4y ago
    Same with students applying. They’re betting that students aren’t applying completely randomly to schools and that the research they do beforehand isn’t entirely noise and has some signal that will show up in the average. I mean it’s a pretty darn conservative proposition to say “we think that averaged over the entire population of applicants that the schools that students spend their time applying to will trend, however small, to better schools. Where the flip side of “it’s all noise and it’s impossible to evaluate a school better than flipping a coin without actually attending” is an incredibly strong assertion.
    - simonh 4y ago
      I'm in the UK, but my eldest is starting Uni this September and rankings played a huge role in her selection of colleges. Yes of course she looked at their prospectuses and visited a few of them, but the shortlist of the ones to look at and visit itself was based on rankings. It's like the XKCD about Wikipedia entries being cited in articles, which get cited in papers, which then get cited in the same Wikipedia article.
- shreyshnaccount 4y ago
  I usually love this kind of feedback loops in systems but this one is kinda sad
Spooky23 4y ago
Perception is the result. Average SAT score relative to others tells you roughly average IQ difference for the student body. The rest is bunk. Undergraduates aren’t connecting with famous professors. A faculty team with deep expertise in a particular area is relevant but not in the rankings.
- naniwaduni 4y ago
  My impression has long been that the endowment per capita metric provides about as good a ranking as any of the third-party rankings within a category. Where it disagrees with US News, it usually matches my intuitions; and where it disagrees with my intuitions, I usually learn that the college in question has a better reputation than my impression. It doesn't do a great job comparing across categories, but then, neither does US News...
  - cma 4y ago
    Some top colleges don't have alumni/legacy preference so that endowment measure maybe gets skewed away from them. MIT and I think Caltech.
    - naniwaduni 4y ago
      They still do great on this metric, though! MIT comes out looking a bit short of Harvard, on par with Stanford. These schools ride a bit higher than the top tier of LACs (Williams/Amherst/Pomona/Swarthmore), which isn't necessarily the fairest comparison, but Caltech fits right in with those. Maybe they'd be richer if their admissions was more open about taking quasi-bribes, but I suspect you're dramatically underestimating the orders of magnitude of difference that appear on this metric. You find clear tier gaps. Don't get me wrong, this is a silly proxy metric about three degrees out from anything you'd actually want to know. But it gets surprisingly good results for how simple it is to tabulate.
Georgelemental 4y ago
Much (often most) of the value of a degree is in signaling. So public perception is very important
- bluGill 4y ago
  Not really. There are only a handful of schools where it matters, the rest nobody has even heard of. Odds are half the people you are competing with have a degree from a university I've never heard of, and the other half it is a local college that I know of but I have no idea how it stacks up compared to yours. The local school when you are local can be a negative though - I know the bad local schools. If you have the big name degree it might push me to interview you when otherwise I'm borderline about tossing your resume, but that is it. I need to know you are good not your school.

troupe 4y ago

Imagine college A that 100 apply for and all 100 get in vs. college B where 10,000 people applied and only 100 got in. If both colleges are choosing students based off of some measurable criteria, let's say the SAT score. Would you expect to have higher average scores at college A or college B? Obviously college B would have students with better SAT scores simply because they are so much more selective. Assuming that the college is selecting students for attributes that you are looking for in fellow students, then the college that denies more people will have students with more of the attributes you are wanting to be around. So it isn't just about perception. There is some math behind it as well.

highwaylights 4y ago
Even so, you don’t need to guess the reasons for acceptance on aggregate data - you can check if the SAT scores were higher. At least that would tell you for certain.
ipaddr 4y ago
It depends if school A costs 100,000 and school B almost free. I would put my money on the 100 who spent 100,000 over the free 10,000.
HWR_14 4y ago
> If both colleges are choosing students based off of some measurable criteria, let's say the SAT score. Would you expect to have higher average scores at college A or college B? Except "average SAT score of incoming class" is another field of the rankings. All the measurable criteria are handled as independent variables - there's no need for a poor proxy.

DenisM 4y ago

> It's basically saying "other people seem to think this college is good, so we'll nudge them up a few points" So does page rank or citation index - popularity begets visibility, begets popularity.

simonh 4y ago
The difference is that applicants by definition don’t have personal experience of how good the school is, it’s all perception. With page rank and citations the expectation is those linking to or citing something have read it and have reason to value it.
- Spivak 4y ago
  Same with students applying. They’re betting that students aren’t applying completely randomly to schools and that the research they do beforehand isn’t entirely noise and has some signal that will show up in the average. I mean it’s a pretty darn conservative proposition to say “we think that averaged over the entire population of applicants that the schools that students spend their time applying to will trend, however small, to better schools. Where the flip side of “it’s all noise and it’s impossible to evaluate a school better than flipping a coin without actually attending” is an incredibly strong assertion.
  - simonh 4y ago
    I'm in the UK, but my eldest is starting Uni this September and rankings played a huge role in her selection of colleges. Yes of course she looked at their prospectuses and visited a few of them, but the shortlist of the ones to look at and visit itself was based on rankings. It's like the XKCD about Wikipedia entries being cited in articles, which get cited in papers, which then get cited in the same Wikipedia article.
shreyshnaccount 4y ago
I usually love this kind of feedback loops in systems but this one is kinda sad

Spooky23 4y ago

Perception is the result. Average SAT score relative to others tells you roughly average IQ difference for the student body. The rest is bunk. Undergraduates aren’t connecting with famous professors. A faculty team with deep expertise in a particular area is relevant but not in the rankings.

naniwaduni 4y ago
My impression has long been that the endowment per capita metric provides about as good a ranking as any of the third-party rankings within a category. Where it disagrees with US News, it usually matches my intuitions; and where it disagrees with my intuitions, I usually learn that the college in question has a better reputation than my impression. It doesn't do a great job comparing across categories, but then, neither does US News...
- cma 4y ago
  Some top colleges don't have alumni/legacy preference so that endowment measure maybe gets skewed away from them. MIT and I think Caltech.
  - naniwaduni 4y ago
    They still do great on this metric, though! MIT comes out looking a bit short of Harvard, on par with Stanford. These schools ride a bit higher than the top tier of LACs (Williams/Amherst/Pomona/Swarthmore), which isn't necessarily the fairest comparison, but Caltech fits right in with those. Maybe they'd be richer if their admissions was more open about taking quasi-bribes, but I suspect you're dramatically underestimating the orders of magnitude of difference that appear on this metric. You find clear tier gaps. Don't get me wrong, this is a silly proxy metric about three degrees out from anything you'd actually want to know. But it gets surprisingly good results for how simple it is to tabulate.

Georgelemental 4y ago

Much (often most) of the value of a degree is in signaling. So public perception is very important

bluGill 4y ago
Not really. There are only a handful of schools where it matters, the rest nobody has even heard of. Odds are half the people you are competing with have a degree from a university I've never heard of, and the other half it is a local college that I know of but I have no idea how it stacks up compared to yours. The local school when you are local can be a negative though - I know the bad local schools. If you have the big name degree it might push me to interview you when otherwise I'm borderline about tossing your resume, but that is it. I need to know you are good not your school.