6 ms·

What college rankings really tell us

76 points by boh 14y ago

Alex3917 14y ago
What this article doesn't mention is how much effort the colleges at the top put into gaming the rankings. For example, Harvard spends millions of dollars every year sending direct mail to graduating seniors who they know have no chance of getting in, encouraging them to apply. Why? To increase their rejection rate. Similarly, Stanford was considered a relatively mediocre school until one year when they rejected every single valedictorian across the country as a marketing stunt, and suddenly jumped in the rankings after guidance counselors started encouraging the kids with the highest grades to apply there. Similarly, the admissions departments get an update twice a day telling them their projected US News rank based on the grades and SAT scores of the kids who they are admitting. Thus your chances of getting in can vary wildly based on whether your application is read in the morning or afternoon. If the projected rank drops too low then they stop even reading the essays and just admit solely based on grades and SATs until the rankings are back up.
- austenallred 14y ago
  Exactly. Brigham Young University (my university) is an absolute slave to the rankings. They intentionally limit the acceptance rates into certain programs even when there's open space for numbers' sake. If these rankings disappeared and universities operated only for the sake of its students, I think the overall quality of education would actually improve.
  - gliese1337 14y ago
    That just further enforces the point that you can't decide on what the best college for you would be by looking at any kind of ranking for the university as a whole. Different colleges and different departments within the university can be run so radically differently that it hardly matters that they're officially part of the same institution.
- harryh 14y ago
  Do you have a citation for the Stanford valedictorian thing? I can't find anything on that topic.
  - Alex3917 14y ago
    I heard it somewhere, I can't find a cite either. It's not the kind of thing they would ever admit to doing, so I'm not sure it would even really be possible to find a cite.
    - ebf 14y ago
      How would you know if it is actually true then and not just an urban legend? Marketing stunts don't make sense if no one knows about them.
    - gmrple 14y ago
      I just went looking for historical ranking data. What I found suggested that Stanford's Rankimg was higher in previous years, in 83, 84, and 85 they were number 1 and they have never dropped below 7. I can't verify my data, so please look for yourself, but unless this event was before then I doubt it happened.
      - nessus42 14y ago
        Yes, when I went to MIT in the 80s, Stanford was often considered to be the best overall university in the country, and MIT wanted to become more like Stanford. I don't know when it achieved such stature, but it also had been one of the Big 3 Computer Science graduate schools for quite some time. (I.e., MIT, Stanford, and CMU.)
        jcitme 14y ago
        No love for Berkeley? I would argue that if you were a perspective student reading USWNR, simply due to proximity to the silicon valley makes the average student get much better chances at jobs and experience in tech.
- ChuckMcM 14y ago
  Reed College (Portland) goes out of its way to not be ranked. Colin Diver, the outgoing President, suggested as others have, that chasing a rank doesn't do the students any good. [1] http://www.reed.edu/apply/news_and_articles/college_rankings.html http://www.reed.edu/apply/news_and_articles/college_rankings...
enraged_camel 14y ago
I graduated from University of Washington, a school that ranks 41st on that list. My roommate in my freshman year of college was Chinese. His dad, a college professor in Hong Kong, always gave him shit for not attending a university that placed higher in the rankings. This is despite the fact that my roommate was a computer science major, and UW has one of the best Comp-Sci programs in the nation. It always seemed to me that his dad cared more about the prestige factor than the actual quality of education his son received. It wouldn't surprise me to find out he has dreams about bragging to his friends that his son went to Harvard or Princeton. What a shame.
- mng2 14y ago
  It's just a Chinese Parent thing. I went to Berkeley (EECS) but my mom is always telling me about her friend's son who's doing a PhD at Harvard, or her friend's son who's a director at some chip firm, etc. You get used to it after a while.
  - SiVal 14y ago
    If it makes you feel any better: I get my hair cut at a place here in Silicon Valley that's full of Chinese tiger moms (incl. the staff). When I was sitting in the chair a few weeks ago, a tiger mom came in the shop with a book of 8x10 photos. She went from Chinese woman to Chinese woman, holding the book open and announcing, "This is my son. Here he is [pointing at the photos, and slowly turning the pages] at his graduation last week. He graduated from Berkeley in electrical engineering." Then, pausing for effect, turning her head and nodding slowly and meaningfully, added, "...with honors." She would then move down the line to the next woman and repeat the whole thing verbatim, down to the pause, slow meaningful nod, and "...with honors." Most women she spoke to tried to look unimpressed, which I thought was a bit rude. Some even sniffed and turned away. It was amazing. I was the lone white guy in a room full of Chinese women (and some kids) and apparently the only one who was happy for her and willing to look at her pictures. No luck. When she got to me, she sniffed at me and skipped past me to the next Chinese woman. I wasn't sure whether that was because my opinion was irrelevant to her sense of social status or, more simply, that she knew the Chinese women but didn't know me. I asked my hair stylist who, like most of the staff, has been a friend of mine for years. She told me that she had seen the woman a couple of times but that she didn't think anybody in the shop knew her by name. I (the only one in the room with a EE background and apparently the only one happy for her) was just not somebody whose opinion mattered to her. So it seems that your Berkeley EECS is plenty good enough to ruffle the feathers of other Chinese moms and to impress me--if that helps any. ;-)
prezjordan 14y ago
I attend a small tech school outside of NYC, and it's not terribly well-known, and not terribly well-ranked. I was a good student in high school, and got into schools ranked much higher than the one I attend now. Why? I love the location here, and I love the "personal" factor that comes with a small school. I've met lots of interesting people, I live in a beautiful town with beautiful views, and I really enjoy what I'm learning. It just "felt right" and it still does today. Some days I get a little upset that I don't attend a prestigious university, and I'm not really sure why. There's always grad school, though. Anyway, just a personal experience.
- kevhsu 14y ago
  RPI? RIT? Those are my guesses...
  - duaneb 14y ago
    IIRC RPI is nowhere near NYC.
    - endersshadow 14y ago
      RPI is in Renssellear, about 2 hours north of NYC. RIT is in Rochester, about 6 hours northwest of NYC--on Lake Ontario.
      - duaneb 14y ago
        For me, close to NYC means Poughkeepsie or further south.
      - waivej 14y ago
        RPI is actually in Troy, but the distance is correct.
        endersshadow 14y ago
        You're right, it is. I always thought that was just Rensselear, but I've never spent long enough to actually notice the town lines.
rickdale 14y ago
This article brings up some really good points. Ultimately schools are obsessed with rankings because alum and prospective students are also obsessed with rankings. The rankings do a really good job of enthralling the consumer and making it competitive to get into college. I graduated in 2009. In that same year we got a new president. He pledged to move the school up in the rankings from 22 in a welcome speech. The crowd gave a standing ovation.
bentlegen 14y ago
On the subjectiveness of reputation: I once met a recruiter for the University of Toronto (Canada) who focuses exclusively on visiting US high schools. She told me how U of T was behind other "top" Canadian schools in advertising to potential students in the US. She mentioned how McGill University (in Montreal) started going after US students aggressively beginning in the 70s, and as a result, has a far better reputation there today. Even The Simpsons once joked that McGill is "the harvard of Canada", even though most Canadians would probably say otherwise. Clip: http://www.youtube.com/watch?v=cc5vN2XReWs http://www.youtube.com/watch?v=cc5vN2XReWs
- tikhonj 14y ago
  I suspect that getting US students is important because they pay significantly more than students from Canada itself. I'm not sure exactly how it works in Canada, but at least at my US public university out-of-state and international students pay far more than instate students which certainly doesn't hurt the budget. Since the US is really close, has expensive education domestically and has a large population, it's probably relatively easy to convince high-paying US students to come to universities in Canada.
crusso 14y ago
If you're a slave to the school rankings in choosing a school, you get what you deserve.
bane 14y ago
I worked for a great guy who also happened to be on the board of a reasonably well respected tech university in the South Western U.S. Every year they have a meeting about doing the kinds of things that boost their rankings in US News and every year they've decided to not do those things because of how it would hurt what they feel to be their academic mission - educating students. He said that almost everything that you can do to boost your ratings are gimmick events like rejecting more applicants, or increasing spending on getting applicants. One particularly greusome idea was to hire national survey companies to hang outside of popular teen events and get students to fill out a preliminary application that they would then reject to get boost their rejection percent and appear more "competitive". They keep rejecting these kinds of meaningless shows of status. But the lure of prestige keeps this meeting on the agenda anyways.
tkahn6 14y ago
I predict the obsession with college and therefore collegiate rankings will dissipate in the next 50 years, maybe even by the close of this decade. Consider what a college experience offers more readily over what you can accomplish with a high-speed internet connection. - a community of people around the same age with no liabilities or responsibilities - guaranteed access to an expert on the subject who can answer direct questions - verification of knowledge Anything else? Why do we need thousands of college professors giving lectures on linear algebra each semester? We have the bandwidth to distribute a complete video lecture series from MIT to all college-going people in the US. The collegiate model is predicated on the inaccessibility and uneven distribution of information. That's why the top colleges have the biggest libraries. That's how information was primarily stored and accessed and catalogued. The developed world is a completely different place now.
- Locke1689 14y ago
  The importance of college won't change until hiring practices do. Most employers don't actually test most of their candidates on their skills. Until they do they need a way to distinguish their candidates. Hiring from Harvard is a good way to do it since you can outsource your requirements to the Harvard acceptance department.
  - kevhsu 14y ago
    Well at least in ECE/CS fields, we have the luxury of more liberal hiring practices in which personal projects on resumes and githubs can matter just as much as what school you went to if not more. It's much harder to measure the quality of candidate in other fields without the context of what school they went to and how well they did there. (Unless, of course, you have time for a full-on interview. I think parent is referring to the screening process though)
  - aswanson 14y ago
    CS and engineering jobs must exist in a different universe than the rest of the working world. I have yet to go through an interview process that did not require a verbal/written series of technical questions being answered.
kenjackson 14y ago
The USNWR rankings are especially insidious because they're also not objective. I wish I could find the citation for this, but I recall reading about how when it first started the person who created the ranking formula showed USNWR the list of top 10 schools and the Ivy League schools didn't do as well as expected. If HYP weren't at the top there was a flaw with the methodology, so they had to go back and get a new methodology that would conform with conventional wisdom. Also back in '99 Caltech took the top spot. Again this was viewed with some disdain from the East Coast elite. They changed their methodology and added/emphasized a metric they call "value add", which is the difference between actual and expected graduation rates. Because CalTech is difficult to graduate from, they were seriously penalized. IOW, rankings are often simply a way to perpetuate the status quo, but make it look objective.
- _delirium 14y ago
  Some mathematicians did a reanalysis a few years ago where they investigated how sensitive the rankings were to the particular choice of weightings. They found that the answer is "very sensitive", to the extent that you can produce a wide range of desired outcomes by carefully choosing the weights to assign to each criterion: http://www.sciencenews.org/view/generic/id/37207/title/Math_Trek__Rating_the_rankings http://www.sciencenews.org/view/generic/id/37207/title/Math_...
- ajross 14y ago
  I think your broad point about "elite" schools being a matter of reputation rather than objective measurement is basically right. But some of your analysis seems weird. On the graduation rate: why shouldn't a school be penalized for a low graduation rate? The point of those rankings (beyond selling magazines, of course) is to give prospective students an idea of how each school will help them in life. If I have a choice of going to school A from which I'm 90% likely to graduate and an otherwise equally ranked school B which is 50% likely to kick me out without a diploma, which am I going to choose? And the bit about CalTech seems a little like sour grapes. I was at one of those "East Coast elite" schools in the 90's and don't remember anyone thinking anything but good things about CalTech (Pasadena, on the other hand...). Basically within its fairly narrow realm of focus CT is absolutely one of the "elite" schools everyone looks to.
api 14y ago
I love how everyone responsible for ranking colleges lives along the North-Eastern seaboard. That area is such a bubble of regional self-importance.
- jmduke 14y ago
  True, but yesterday a post titled 'How Coffee Meetings Power Silicon Valley' hit the front page. Every region has people who are self-important and people who are not.
mullr 14y ago
I wonder where the data for these rankings comes from. Is it publicly available? It would be nice to have a dynamic ranker which prioritizes the things you care about.
jmduke 14y ago
http://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_the_United_States_by_endowment http://en.wikipedia.org/wiki/List_of_colleges_and_universiti... Sort by total endowment per student. Cross-reference this list with the USNWR rankings. The overlaps are striking.
justaddwater57 14y ago
Interesting that Tulane is called out here as an example: "One common statistic used to evaluate colleges, for example, is called “graduation rate performance,” which compares a school’s actual graduation rate with its predicted graduation rate given the socioeconomic status and the test scores of its incoming freshman class.... Tulane, given the qualifications of the students that it admits, ought to have a graduation rate of eighty-seven per cent; its actual 2009 graduation rate was seventy-three per cent. That shortfall suggests that something is amiss at Tulane." Anyone else think that those numbers for the 2009 graduating class has anything to do with the fact that Hurricane Katrina hit during the first few weeks of '09's freshman year, forcing students at Tulane (which is located in New Orleans) to take leaves of absences or transfers? Just goes to show, in addition to inherent biases, rankings also don't capture extenuating circumstances that can have drastic short term effects that go unnoticed.
- sfny 14y ago
  Have you read Gladwell's book "Outliers"? Basically deals with the situational factors that create outliers like Bill Gates. Circumstances have incredible life-determining effect that go totally unnoticed too.
- RedwoodCity 14y ago
  Hurricane, or perhaps Tulane is a party school.
geebee 14y ago
The Washington Monthly had an interesting angle on college rankings, and how different their results are from US News: http://www.washingtonmonthly.com/magazine/septemberoctober_2011/features/introduction_a_different_kind031630.php http://www.washingtonmonthly.com/magazine/septemberoctober_2...
mnemonicsloth 14y ago
(sort-by (fn [u] (* (:age u) (:tuition u) (if (rhymes-with (:name u) 'Warvard) 1 (+ 0.9 (rand 0.1)))) univs)
appleflaxen 14y ago
I liked malcolm gladwell a lot more before I found out he is a big tobacco shill. http://exiledonline.com/malcolm-gladwell-tobacco-industry-shill/ http://exiledonline.com/malcolm-gladwell-tobacco-industry-sh...
sigil 14y ago
Watch for recommender systems to disrupt and replace many of these rating systems. As Gladwell pointed out in the Car & Driver example, the "scores" assigned in these rating systems are arbitrary linear combinations of factors chosen by "experts." You the car buyer will personally care more or less about certain factors than they do, but they've already chosen the coefficients for you, so too bad. Same for the college ratings assigned by the USNWR. I think we're asking the wrong questions with college rankings. Prospective students should be asking, what education will maximize the expected value to me personally in the future? It's not a heterogenous and inexplicable "score" they're looking for, it's the probability that the match between student and college will be a productive one. Given data about individual students, colleges, and historical outcomes, surely a better predictive model is possible.
RedwoodCity 14y ago
Thank You to who ever posted this article. It points out how stupid college/car/etc... rankings are. I think people have a natural tendency to want to rank things, in the end it just makes us unhappy.

76 points by boh 14y ago

What this article doesn't mention is how much effort the colleges at the top put into gaming the rankings. For example, Harvard spends millions of dollars every year sending direct mail to graduating seniors who they know have no chance of getting in, encouraging them to apply. Why? To increase their rejection rate. Similarly, Stanford was considered a relatively mediocre school until one year when they rejected every single valedictorian across the country as a marketing stunt, and suddenly jumped in the rankings after guidance counselors started encouraging the kids with the highest grades to apply there. Similarly, the admissions departments get an update twice a day telling them their projected US News rank based on the grades and SAT scores of the kids who they are admitting. Thus your chances of getting in can vary wildly based on whether your application is read in the morning or afternoon. If the projected rank drops too low then they stop even reading the essays and just admit solely based on grades and SATs until the rankings are back up.

austenallred 14y ago
Exactly. Brigham Young University (my university) is an absolute slave to the rankings. They intentionally limit the acceptance rates into certain programs even when there's open space for numbers' sake. If these rankings disappeared and universities operated only for the sake of its students, I think the overall quality of education would actually improve.
- gliese1337 14y ago
  That just further enforces the point that you can't decide on what the best college for you would be by looking at any kind of ranking for the university as a whole. Different colleges and different departments within the university can be run so radically differently that it hardly matters that they're officially part of the same institution.
harryh 14y ago
Do you have a citation for the Stanford valedictorian thing? I can't find anything on that topic.
- Alex3917 14y ago
  I heard it somewhere, I can't find a cite either. It's not the kind of thing they would ever admit to doing, so I'm not sure it would even really be possible to find a cite.
  - ebf 14y ago
    How would you know if it is actually true then and not just an urban legend? Marketing stunts don't make sense if no one knows about them.
  - gmrple 14y ago
    I just went looking for historical ranking data. What I found suggested that Stanford's Rankimg was higher in previous years, in 83, 84, and 85 they were number 1 and they have never dropped below 7. I can't verify my data, so please look for yourself, but unless this event was before then I doubt it happened.
    - nessus42 14y ago
      Yes, when I went to MIT in the 80s, Stanford was often considered to be the best overall university in the country, and MIT wanted to become more like Stanford. I don't know when it achieved such stature, but it also had been one of the Big 3 Computer Science graduate schools for quite some time. (I.e., MIT, Stanford, and CMU.)
      - jcitme 14y ago
        No love for Berkeley? I would argue that if you were a perspective student reading USWNR, simply due to proximity to the silicon valley makes the average student get much better chances at jobs and experience in tech.
ChuckMcM 14y ago
Reed College (Portland) goes out of its way to not be ranked. Colin Diver, the outgoing President, suggested as others have, that chasing a rank doesn't do the students any good. [1] http://www.reed.edu/apply/news_and_articles/college_rankings.html http://www.reed.edu/apply/news_and_articles/college_rankings...

enraged_camel 14y ago

I graduated from University of Washington, a school that ranks 41st on that list. My roommate in my freshman year of college was Chinese. His dad, a college professor in Hong Kong, always gave him shit for not attending a university that placed higher in the rankings. This is despite the fact that my roommate was a computer science major, and UW has one of the best Comp-Sci programs in the nation. It always seemed to me that his dad cared more about the prestige factor than the actual quality of education his son received. It wouldn't surprise me to find out he has dreams about bragging to his friends that his son went to Harvard or Princeton. What a shame.

mng2 14y ago
It's just a Chinese Parent thing. I went to Berkeley (EECS) but my mom is always telling me about her friend's son who's doing a PhD at Harvard, or her friend's son who's a director at some chip firm, etc. You get used to it after a while.
- SiVal 14y ago
  If it makes you feel any better: I get my hair cut at a place here in Silicon Valley that's full of Chinese tiger moms (incl. the staff). When I was sitting in the chair a few weeks ago, a tiger mom came in the shop with a book of 8x10 photos. She went from Chinese woman to Chinese woman, holding the book open and announcing, "This is my son. Here he is [pointing at the photos, and slowly turning the pages] at his graduation last week. He graduated from Berkeley in electrical engineering." Then, pausing for effect, turning her head and nodding slowly and meaningfully, added, "...with honors." She would then move down the line to the next woman and repeat the whole thing verbatim, down to the pause, slow meaningful nod, and "...with honors." Most women she spoke to tried to look unimpressed, which I thought was a bit rude. Some even sniffed and turned away. It was amazing. I was the lone white guy in a room full of Chinese women (and some kids) and apparently the only one who was happy for her and willing to look at her pictures. No luck. When she got to me, she sniffed at me and skipped past me to the next Chinese woman. I wasn't sure whether that was because my opinion was irrelevant to her sense of social status or, more simply, that she knew the Chinese women but didn't know me. I asked my hair stylist who, like most of the staff, has been a friend of mine for years. She told me that she had seen the woman a couple of times but that she didn't think anybody in the shop knew her by name. I (the only one in the room with a EE background and apparently the only one happy for her) was just not somebody whose opinion mattered to her. So it seems that your Berkeley EECS is plenty good enough to ruffle the feathers of other Chinese moms and to impress me--if that helps any. ;-)

prezjordan 14y ago

I attend a small tech school outside of NYC, and it's not terribly well-known, and not terribly well-ranked. I was a good student in high school, and got into schools ranked much higher than the one I attend now. Why? I love the location here, and I love the "personal" factor that comes with a small school. I've met lots of interesting people, I live in a beautiful town with beautiful views, and I really enjoy what I'm learning. It just "felt right" and it still does today. Some days I get a little upset that I don't attend a prestigious university, and I'm not really sure why. There's always grad school, though. Anyway, just a personal experience.

kevhsu 14y ago
RPI? RIT? Those are my guesses...
- duaneb 14y ago
  IIRC RPI is nowhere near NYC.
  - endersshadow 14y ago
    RPI is in Renssellear, about 2 hours north of NYC. RIT is in Rochester, about 6 hours northwest of NYC--on Lake Ontario.
    - duaneb 14y ago
      For me, close to NYC means Poughkeepsie or further south.
    - waivej 14y ago
      RPI is actually in Troy, but the distance is correct.
      - endersshadow 14y ago
        You're right, it is. I always thought that was just Rensselear, but I've never spent long enough to actually notice the town lines.

rickdale 14y ago

This article brings up some really good points. Ultimately schools are obsessed with rankings because alum and prospective students are also obsessed with rankings. The rankings do a really good job of enthralling the consumer and making it competitive to get into college. I graduated in 2009. In that same year we got a new president. He pledged to move the school up in the rankings from 22 in a welcome speech. The crowd gave a standing ovation.

bentlegen 14y ago

On the subjectiveness of reputation: I once met a recruiter for the University of Toronto (Canada) who focuses exclusively on visiting US high schools. She told me how U of T was behind other "top" Canadian schools in advertising to potential students in the US. She mentioned how McGill University (in Montreal) started going after US students aggressively beginning in the 70s, and as a result, has a far better reputation there today. Even The Simpsons once joked that McGill is "the harvard of Canada", even though most Canadians would probably say otherwise. Clip: http://www.youtube.com/watch?v=cc5vN2XReWs http://www.youtube.com/watch?v=cc5vN2XReWs

tikhonj 14y ago
I suspect that getting US students is important because they pay significantly more than students from Canada itself. I'm not sure exactly how it works in Canada, but at least at my US public university out-of-state and international students pay far more than instate students which certainly doesn't hurt the budget. Since the US is really close, has expensive education domestically and has a large population, it's probably relatively easy to convince high-paying US students to come to universities in Canada.

crusso 14y ago

If you're a slave to the school rankings in choosing a school, you get what you deserve.

bane 14y ago

I worked for a great guy who also happened to be on the board of a reasonably well respected tech university in the South Western U.S. Every year they have a meeting about doing the kinds of things that boost their rankings in US News and every year they've decided to not do those things because of how it would hurt what they feel to be their academic mission - educating students. He said that almost everything that you can do to boost your ratings are gimmick events like rejecting more applicants, or increasing spending on getting applicants. One particularly greusome idea was to hire national survey companies to hang outside of popular teen events and get students to fill out a preliminary application that they would then reject to get boost their rejection percent and appear more "competitive". They keep rejecting these kinds of meaningless shows of status. But the lure of prestige keeps this meeting on the agenda anyways.

tkahn6 14y ago

I predict the obsession with college and therefore collegiate rankings will dissipate in the next 50 years, maybe even by the close of this decade. Consider what a college experience offers more readily over what you can accomplish with a high-speed internet connection. - a community of people around the same age with no liabilities or responsibilities - guaranteed access to an expert on the subject who can answer direct questions - verification of knowledge Anything else? Why do we need thousands of college professors giving lectures on linear algebra each semester? We have the bandwidth to distribute a complete video lecture series from MIT to all college-going people in the US. The collegiate model is predicated on the inaccessibility and uneven distribution of information. That's why the top colleges have the biggest libraries. That's how information was primarily stored and accessed and catalogued. The developed world is a completely different place now.

Locke1689 14y ago
The importance of college won't change until hiring practices do. Most employers don't actually test most of their candidates on their skills. Until they do they need a way to distinguish their candidates. Hiring from Harvard is a good way to do it since you can outsource your requirements to the Harvard acceptance department.
- kevhsu 14y ago
  Well at least in ECE/CS fields, we have the luxury of more liberal hiring practices in which personal projects on resumes and githubs can matter just as much as what school you went to if not more. It's much harder to measure the quality of candidate in other fields without the context of what school they went to and how well they did there. (Unless, of course, you have time for a full-on interview. I think parent is referring to the screening process though)
- aswanson 14y ago
  CS and engineering jobs must exist in a different universe than the rest of the working world. I have yet to go through an interview process that did not require a verbal/written series of technical questions being answered.

kenjackson 14y ago

The USNWR rankings are especially insidious because they're also not objective. I wish I could find the citation for this, but I recall reading about how when it first started the person who created the ranking formula showed USNWR the list of top 10 schools and the Ivy League schools didn't do as well as expected. If HYP weren't at the top there was a flaw with the methodology, so they had to go back and get a new methodology that would conform with conventional wisdom. Also back in '99 Caltech took the top spot. Again this was viewed with some disdain from the East Coast elite. They changed their methodology and added/emphasized a metric they call "value add", which is the difference between actual and expected graduation rates. Because CalTech is difficult to graduate from, they were seriously penalized. IOW, rankings are often simply a way to perpetuate the status quo, but make it look objective.

_delirium 14y ago
Some mathematicians did a reanalysis a few years ago where they investigated how sensitive the rankings were to the particular choice of weightings. They found that the answer is "very sensitive", to the extent that you can produce a wide range of desired outcomes by carefully choosing the weights to assign to each criterion: http://www.sciencenews.org/view/generic/id/37207/title/Math_Trek__Rating_the_rankings http://www.sciencenews.org/view/generic/id/37207/title/Math_...
ajross 14y ago
I think your broad point about "elite" schools being a matter of reputation rather than objective measurement is basically right. But some of your analysis seems weird. On the graduation rate: why shouldn't a school be penalized for a low graduation rate? The point of those rankings (beyond selling magazines, of course) is to give prospective students an idea of how each school will help them in life. If I have a choice of going to school A from which I'm 90% likely to graduate and an otherwise equally ranked school B which is 50% likely to kick me out without a diploma, which am I going to choose? And the bit about CalTech seems a little like sour grapes. I was at one of those "East Coast elite" schools in the 90's and don't remember anyone thinking anything but good things about CalTech (Pasadena, on the other hand...). Basically within its fairly narrow realm of focus CT is absolutely one of the "elite" schools everyone looks to.

api 14y ago

I love how everyone responsible for ranking colleges lives along the North-Eastern seaboard. That area is such a bubble of regional self-importance.

jmduke 14y ago
True, but yesterday a post titled 'How Coffee Meetings Power Silicon Valley' hit the front page. Every region has people who are self-important and people who are not.

mullr 14y ago

I wonder where the data for these rankings comes from. Is it publicly available? It would be nice to have a dynamic ranker which prioritizes the things you care about.

jmduke 14y ago

http://en.wikipedia.org/wiki/List_of_colleges_and_universities_in_the_United_States_by_endowment http://en.wikipedia.org/wiki/List_of_colleges_and_universiti... Sort by total endowment per student. Cross-reference this list with the USNWR rankings. The overlaps are striking.

justaddwater57 14y ago

Interesting that Tulane is called out here as an example: "One common statistic used to evaluate colleges, for example, is called “graduation rate performance,” which compares a school’s actual graduation rate with its predicted graduation rate given the socioeconomic status and the test scores of its incoming freshman class.... Tulane, given the qualifications of the students that it admits, ought to have a graduation rate of eighty-seven per cent; its actual 2009 graduation rate was seventy-three per cent. That shortfall suggests that something is amiss at Tulane." Anyone else think that those numbers for the 2009 graduating class has anything to do with the fact that Hurricane Katrina hit during the first few weeks of '09's freshman year, forcing students at Tulane (which is located in New Orleans) to take leaves of absences or transfers? Just goes to show, in addition to inherent biases, rankings also don't capture extenuating circumstances that can have drastic short term effects that go unnoticed.

sfny 14y ago
Have you read Gladwell's book "Outliers"? Basically deals with the situational factors that create outliers like Bill Gates. Circumstances have incredible life-determining effect that go totally unnoticed too.
RedwoodCity 14y ago
Hurricane, or perhaps Tulane is a party school.

geebee 14y ago

The Washington Monthly had an interesting angle on college rankings, and how different their results are from US News: http://www.washingtonmonthly.com/magazine/septemberoctober_2011/features/introduction_a_different_kind031630.php http://www.washingtonmonthly.com/magazine/septemberoctober_2...

mnemonicsloth 14y ago

(sort-by (fn [u] (* (:age u) (:tuition u) (if (rhymes-with (:name u) 'Warvard) 1 (+ 0.9 (rand 0.1)))) univs)

appleflaxen 14y ago

I liked malcolm gladwell a lot more before I found out he is a big tobacco shill. http://exiledonline.com/malcolm-gladwell-tobacco-industry-shill/ http://exiledonline.com/malcolm-gladwell-tobacco-industry-sh...

sigil 14y ago

Watch for recommender systems to disrupt and replace many of these rating systems. As Gladwell pointed out in the Car & Driver example, the "scores" assigned in these rating systems are arbitrary linear combinations of factors chosen by "experts." You the car buyer will personally care more or less about certain factors than they do, but they've already chosen the coefficients for you, so too bad. Same for the college ratings assigned by the USNWR. I think we're asking the wrong questions with college rankings. Prospective students should be asking, what education will maximize the expected value to me personally in the future? It's not a heterogenous and inexplicable "score" they're looking for, it's the probability that the match between student and college will be a productive one. Given data about individual students, colleges, and historical outcomes, surely a better predictive model is possible.

RedwoodCity 14y ago

Thank You to who ever posted this article. It points out how stupid college/car/etc... rankings are. I think people have a natural tendency to want to rank things, in the end it just makes us unhappy.