9 ms·

The Economist's US college rankings

115 points by mrsaint 11y ago

hacknat 11y ago
It would be very cool if you could sort by percentage differential between the over/under and expected earnings. I think sorting by that parameter would produce some really interesting results.
- ryandrake 11y ago
  Agreed. Ranking by raw dollar values means that schools in very expensive areas (or that feed into very expensive areas) will have exaggerated scores and end up either really highly ranked or really low. Another interesting study would be to see whether a child's college attendance correlates with a better or worse standard of living from their parents'. Maybe look at Father's salary at age 30 and child's salary at age 30, and see if the attended college is significant. The question being: If you're born rich, you tend to stay rich, and if you're born poor, you tent to stay poor. Does college choice matter?
cassieramen 11y ago
Does anyone know of a good diy college ranking system? One where you can pick that factors that actually matter to you. I'm always happy to see options other than U.S. News but I'd love to be able to tweak the algorithm myself. I think these one size fits all rankings are all flawed by their inherent nature.
- harmegido 11y ago
  The economist article linked here references (and uses in their rankings) data from https://collegescorecard.ed.gov/ https://collegescorecard.ed.gov/, which is put out by the US Dept of Ed. Looks like there's some additional metrics in there that may be useful.
- brensudol 11y ago
  I made a simple DIY ranking system using the same College Scorecard data last week. Here you go: http://bsudol.com/1PUPohk http://bsudol.com/1PUPohk You can select which variables are important and how much, and it generates a top/bottom 50 list. This was inspired by this article from NPR (http://www.npr.org/sections/ed/2015/09/21/441417608/the-new-college-scorecard-npr-does-some-math http://www.npr.org/sections/ed/2015/09/21/441417608/the-new-...)
  - cassieramen 11y ago
    This is awesome! I'm happy the college score card data is being used for great project like this.
  - vpalan2 11y ago
    This is great. As a next step, perhaps make the sliders more mobile friendly?
  - Amorymeltzer 11y ago
    Submitted: https://news.ycombinator.com/item?id=10480851 https://news.ycombinator.com/item?id=10480851
6stringmerc 11y ago
The concept of studying in Higher Education for more opportunities, job stability, and income during one's lifetime is not to be dismissed. That said, if the predominant value of education is in the pursuit of money, then I think using the Brookings Institute model (which The Economist references) that includes two-year and vocational education entities is a much more pragmatic and practical approach. That is, if money is the goal, only focus on money as the outcome. Conflating money with prestige is, to me, foolish. A University is a prestige degree, insofar as there's a pursuit of knowledge, in theory, to produce a well-rounded, educated person...in theory. I say this as a graduate of two generally highly ranked Universities, who occasionally gets the feeling that I'd be making more money if I'd taken my education budget, got trained in HVAC service/repair/sales, and started my own company.
- btilly 11y ago
  Even if the predominant value of education is the pursuit of money, the Economist still has the more useful model. It allows you to differentiate between schools whose graduates make money because they are full of people who are going to try to make money, and schools that make money because the school brings something to the table that helps its graduates above other options. Look at CalTech for example. It does well in the Brookings model because it attracts a lot of people who want to be engineers. It does poorly in The Economist's model because its engineers are not making as much in 10 years as would be expected given a variety of factors that they control for. (Now that said, CalTech may be hurt because they graduate a lot of people who take side tracks through grad school and presumably would make more 15-20 years out. But you have to work with the data you have, and all they had was 10 years.) Now if I'm a student planning to go into engineering, the Brookings model merely confirms that I'm making an excellent choice. The Economist's model suggests that if I'm capable of going to CalTech, maybe that is not the best school for me.
  - 6stringmerc 11y ago
    After reading through your response, I believe you missed my point. What I was referring to is that for a large population of potential students - those who wish to simply make more than if they do not go to a higher education program - going to a two year or vocational path school is exceptionally smart. Leaving that out of consideration is...not smart...highly biased...as in, how many readers of The Economist would consider sending their kid to become a diesel mechanic? From the research I've absorbed over the years, students that can afford to go to prestige universities often come from well-to-do, college educated families - these are high indicators of future success. For a much larger swath of the population that is trying to advance their stake in life, they are first-generation students, and often not well-served by taking on debt and attending a large University environment. Thus, many Universities are, practically speaking, very bad at ROI for a large contingent of students who simply want to make more money - vocational training? An excellent prospect. What remains to be seen is how much more earning potential there is in professions that require vocational training to be licensed or successful (i.e. welding, machining, air traffic control, plumbing, electrical contracting) as the generation currently working retires / dies from the workforce, thereby spiking demand, and in fields where a "traditional" University degree are, well, useless.
adenverd 11y ago
This tool would be a lot more useful if it allowed for filtering by chosen course of study. For example, University of Washington has a median earnings that is slightly below expected, but I can pretty much guarantee that their computer science graduates are earning well above median.
- aidenn0 11y ago
  It assumes you select a school that is focused on your field of choice, so if you pick an art school, your preference is for art, and if you pick an engineering school, your preference is for engineering.
  - rifung 11y ago
    The issue is that you can't select the University of Washington's engineering school. You can only see University of Washington which has both an engineering and an art school, and so it doesn't tell you what the data will be if you only look at engineering students.
- digikata 11y ago
  That was my thought too, to make a simplistic example if a university had both say an engineering school and an art school, it might presumably do worse than a university with only an engineering school. So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries...
  - _delirium 11y ago
    > So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries... I don't think that part's necessarily true. If a school focuses on an area with high median salaries, the model will take that into account in the predicted salaries, so the school will have to have even higher actual salaries than typical for the field (and its input demographics, SAT scores, etc.) to get a positive value-add. See Caltech for an example of a STEM-focused school that does badly by this measure: from its SAT scores, demographics, and heavy concentration of STEM majors, the regression analysis predicts that it should produce graduates with a median salary of $82k. But the actual median is $74k, so its value-add is taken to be -$8k. Some of the schools that do well are in areas with poor salaries, but score highly because they do better than you'd expect (or than the model would expect, anyway) for that area and student demographics. Otis College of Art and Design has a predicted salary of $29k from the regression analysis, but actual median is $42k, so implied value-add +$13k.
hackuser 11y ago
The Brookings study [1] mentioned by the the Economist offers this interesting analysis: 1. Graduates of some colleges enjoy much more economic success than their characteristics at time of admission would suggest. Colleges with high value-added in terms of alumni earnings are often focused on training for high-paying careers in technical subjects. ... 2. Four college quality factors are strongly associated with higher earnings for alumni: Curriculum value: The amount earned by people in the workforce who hold degrees in a field of study offered by the college, averaged across all the degrees the college awards; STEM orientation: The share of graduates prepared to work in STEM occupations; Completion rates: The percentage of students finishing their award within at least 1.5 times the normal time (three years for a two-year college, six years for a four-year college); Faculty salaries: The average monthly compensation of all teaching staff 3. Value-added measures are fairly reliable over time. ... (Personally, I think the most valuable things gained in college or in any education don't happen to have much monetary value.) [1] http://www.brookings.edu/research/reports2/2015/10/29-earnings-data-college-scorecard-rothwell http://www.brookings.edu/research/reports2/2015/10/29-earnin...
- jldugger 11y ago
  > (Personally, I think the most valuable things gained in college or in any education don't happen to have much monetary value.) Well, someone does, because they're charging for it.
  - hackuser 11y ago
    > Well, someone does, because they're charging for it. Good point; I should have said, I don't think they are things that earn you money.
impendia 11y ago
I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provided. I suspect that many Caltech undergraduates decide to pursue grad school and academia, which is not an especially lucrative career, and which is probably not highly corrolated with political leftism or "reefer madness". I can also tell you that my friends at Rice who were looking to make a lot of money after they graduated, by and large succeeded. In summary: Bullshit.
- armaansarkar 11y ago
  Didn't they kinda address this? "The bar is set extremely high for universities like Caltech, which are selective, close to prosperous cities and teach mainly lucrative subjects. If their students didn’t go on to extremely high-paying careers, the college would probably be doing something gravely wrong."
  - impendia 11y ago
    I dunno, how about pursuing careers as scientists rather than as hacks for the finance industry? Is that "gravely wrong"? And as far as being close to a prosperous city -- that is because Caltech is near the world hub of the entertainment industry. A Caltech education is mostly irrelevant to finding employment there.
  - slg 11y ago
    Isn't that an inherit flaw in the system? It is also based on the assumption that everyone prioritizes money over everything else. If I worked in the Yale marketing department I could easily spin this as a being caused by the high character of Yale graduates who are more willing to dedicate their life to lower paying but higher impact jobs like academics or public service while those greedy kids from Harvard only ever chase money.
    - 11y ago
hackuser 11y ago
Income is a poor measure of college education. The problem is that the valuable benefits of college are difficult to quantify; income has importance and is quantifiable, so I understand the temptation to use it as a metric, but it doesn't represent of the values of a college. It's like using the number of lines of code as a metric to represent the value of a piece of code - it's quantifiable, significant in some ways, but not representative of the code's value. I much prefer the Times Higher Education model, especially their reputation survey. They survey 10,000 tenured and published academics worldwide, using what looks like well-designed methodology [1]. These are people in a position to have expert knowledge about the qualities of various universities. Yes it's imperfect and those people have bias too, but I can't think of a better model: https://www.timeshighereducation.com/world-university-rankings/2015/reputation-ranking https://www.timeshighereducation.com/world-university-rankin... One excellent alternative is Washington Monthly's rankings. Their approach: We rate schools based on their contribution to the public good in three broad categories: Social Mobility (recruiting and graduating low-income students), Research (producing cutting-edge scholarship and PhDs), and Service (encouraging students to give something back to their country). More here: http://www.washingtonmonthly.com/college_guide/rankings-2015/national-universities-rank.php http://www.washingtonmonthly.com/college_guide/rankings-2015... ---- [1] https://www.timeshighereducation.com/world-reputation-rankings-2015-methodology https://www.timeshighereducation.com/world-reputation-rankin...
fiatmoney 11y ago
There seems to be a major issue with cost-of-living adjustments as well. Several rather obscure California colleges are highly ranked, and I assume this is because both wages and costs are higher in California, where many of their graduates stay after college.
- wtvanhest 11y ago
  Absolutely. Cost of living in an area will have a massive impact on exit salaries. The economist did not realize that the South and Mid-west have the lowest cost of living when they wrote the following: > Its upper tiers are dominated by colleges that emphasise engineering (such as Worcester Polytechnic) and attract students with high SAT scores (like Stanford). The lower extreme is populated by religious and art-focused colleges, particularly those in the south and Midwest. This number represents the benchmark against which we subsequently compare each college’s alumni earnings to produce the rankings.
- dj_axl 11y ago
  > There seems to be a major issue with cost-of-living adjustments Yep yep. Otis College of Art and Design "overperforming" for $42k salary is ridiculous. The title Junior Graphic Designer pays around $40k* in Los Angeles, where Otis is located. Therefore, the average for graduates is as-expected. *Source: Glassdoor.
- rconti 11y ago
  I'm not so sure. My generally-highly-regarded college in California (bay area, no less!) is almost bang-on average ($-33) despite being in a very high cost of living area.
  - rconti 11y ago
    To reply to myself, the comments include this mention from "DR" of The Economist: ----- Geography (both city and state) are variables included in our model. Colleges in the Bay Area are not rewarded for the high salaries available there--they have to surpass a higher "bar" of expected earnings. That is why our eighth-ranked college is in West Virginia, and the ninth in Laredo, Texas. ------ Now, I don't know how they compensate for this. Cost of living at the place where the student ends up? Or where they study? If you study in a high cost of living area but move to Podunk, Iowa, your return will be far lower than expected, if they're only controlling for CoL at the college's geographical location. If they control for CoL at the student's new home, then okay.
leroy_masochist 11y ago
So basically, the methodology is that they do a very complex analysis of the student body to predict what their average earnings would be if they went to "college in general" and then compare that with what the graduates actually make. There's one thing that jumps out at me when I see the actual rankings: At least three of the eight schools that received a perfect 100 score -- W&L, Babson, and Bentley -- have a very, very high number of students who go to work for lucrative family businesses immediately upon graduation. In the case of W&L (which provides a great liberal arts education mind you) it's mostly southern good ol' boy/girl types. At Babson / Bentley (which provide great undergrad business educations), a huge chunk of the student body are the scions of the economic elite in developing countries, who are getting schooled up so they can be ready to be put in charge of something at a young age. My hunch is that the Economist's "expected earnings" methodology wasn't granular enough to take these idiosyncratic attributes into account, and that its r-squared of .85 might not be a rigorous number.
- beachstartup 11y ago
  > My hunch is that the Economist's "expected earnings" methodology wasn't granular enough to take these idiosyncratic attributes into account or, given the economist's core demographic (rich borderless global elites) this is exactly what they optimized for.
- btilly 11y ago
  Note that the data set they had only includes students who got federal loans. I strongly doubt that students who go to work for lucrative family businesses immediately upon graduation will be in the data set. But having classmates who look like that seems to be a really good economic choice! Which is a factor that is not apparent in normal college rankings.
mynameishere 11y ago
Well, I'm glad my shitty school beat Yale (almost last) so handily. Really, I don't think they could have come up with a worse metric. If you simply took each school's equivalent of this: https://en.wikipedia.org/wiki/List_of_Yale_University_people https://en.wikipedia.org/wiki/List_of_Yale_University_people ...and just sorted it by # of entries, you'd have a far superior list.
jpatel3 11y ago
Would love to get hold of csv or api to see state wide university data.
seibelj 11y ago
If a college has a huge communications program and a smaller engineering program, it would be ranked less because of pay disparities. This methodology is simply inaccurate
et2o 11y ago
This is hilariously bad. Schools that produce academics as opposed to those who go into high-paying professions are penalized, when in fact (at many of the top institutions) producing academics is one of the main goals.
- stagger87 11y ago
  This is not bad. It is simply one of many possible ways to rank colleges.
  - BookmarkSaver 11y ago
    But it is a serious flaw in the methodology. It says nothing about the quality of education or the boost in earnings that a college will provide, if you are specifically going to a school to do so. At the very least they need to somehow account for individuals that go to the top schools but aren't looking to go into lucrative careers. CalTech is a common example being cited here. It is basically impossible to deny realistically that going to CalTech is a great way to make a lot of money. But it is in the bottom percentages of these rankings, likely because so many CalTech alumns go into academia. The metric is supposedly showing which colleges can potentially increase your earnings the most, but it just isn't doing that. It's a great idea, it just hasn't been implemented fully.
    - selimthegrim 11y ago
      I think the money thing is paralleling something else entirely - given the supposed caliber of student that comes into Caltech, is Caltech really making them better scientists than if they had gone elsewhere (or at least better scientists than they were turning out in the 1970s). That to me is a open question.
      - BookmarkSaver 11y ago
        Yes, but Caltech students have some of the highest rates of going into academia. Which pays well on a general scale, but compared to where engineers and developers go onto it drags the stats down. They do admit that the metric only gauges financial success and is imperfect when it comes to accounting for alternative priorities. But do you seriously believe that CalTech provides one of the worst "financial bumps" even accounting for the quality of their admissions? I find it incredibly hard to believe.
ronyeh 11y ago
The data makes more sense if you sort by expected earnings or median earnings and then compare within localized clusters of universities. For example, MIT and Caltech have similar expected earnings, but MIT wins on median earnings.
deleted 11y ago
[deleted]
obastani 11y ago
This analysis doesn't make any sense to me. Even by their own intent (i.e., best added value based on salary), their methodology is nonsensical. For example: Why should CalTech get hurt for being near L.A.? They're basically arguing that you're better off going to a school in the middle of nowhere because "hey, for being in such a crappy location, you did pretty well!". In an absolute sense you are better off going to CalTech, it's just that they might not leverage their advantage as well as some other schools. Not that they even show the last point -- it seems unlikely that the true model is linear (I'm guessing they used linear regression). For example, if the true model is closer to a sigmoid, then schools at the high end suddenly get unfairly penalized and schools near the low end get unfairly boosted. Finally, the statistical indicators are equally misleading. I can obtain an R^2 of 1.0 just by including indicators I[is COLLEGE_NAME] for each college. While that might not give you significance, the point is that getting good prediction is meaningless. I think what they really want is to restrict to predictors about the students. So, given that you're a straight A student with a 2400 SAT score, what would you expect to make coming out of each school? This at least tells me something about the added value to me of going to a certain school. (This approach is still prone to bias, but in the opposite direction -- there's a chance that the straight A student with a 2400 SAT score going to community college may have been smart but unmotivated, which might correlate with lower salary.) Edit: Here's another concern. They're claiming to have a model for "expected" earnings: earnings = A * (college covariates) + b + error but they can't distinguish between model error (i.e., error because their model is misspecified) vs. the school variation that they are trying to capture.
- krstck 11y ago
  > They're basically arguing that you're better off going to a school in the middle of nowhere because "hey, for being in such a crappy location, you did pretty well!". Well, no, not exactly. It's a subtle distinction, but what it's actually ranking is how well that school exceeds expectations, not best outcomes. This is not necessarily a list that will give a student the best school to go to, but rather (what it says on the tin) a scorecard for how well those schools are doing, given their resources.
  - 11y ago
thefastlane 11y ago
i tried to look up several four-year U.S. institutions just now, including one major research university, and none were in the list. from the article, apparently they excluded universities when the scorecard dataset was missing one or more factors; it would be useful if the tool at least showed the university anyway as a placeholder and indicated what data was missing.
cgearhart 11y ago
>"The government generated the numbers by matching individuals’ student-loan applications to their subsequent tax returns, making it possible to compare pupils’ qualifications and demographic characteristics when they entered college with their salaries ten years later." So the dataset excludes students who do not apply for loans? (i.e., this analysis penalizes schools who admit the folks most likely to make lots of money, and the schools that have the lowest expected student contribution.) >"...based on a simple, if debatable, premise: the economic value of a university is equal to the gap between how much money its graduates earn, and how much they might have made had they studied elsewhere." If we are only to look at financial incentives, a more reasonable analysis would be to compare the expected future earnings _distribution_ as opposed to just a central tendency statistic like the median.
- dragonwriter 11y ago
  > So the dataset excludes students who do not apply for loans? (i.e., this analysis penalizes schools who admit the folks most likely to make lots of money, and the schools that have the lowest expected student contribution.) It penalizes (or at a minimum, undersamples in a nonrepresentative way) schools with their own, non-loan, aid programs. This probably very badly hurts schools like Caltech, since it means that there (unless Caltech no longer has the essentially full-coverage need-based aid they had when I went -- unsuccessfully, I graduated elsewhere) they are only counting people who are making the unwise choice of taking loans when they have no need given their current resources. That this will skew the data badly should be obvious.
  - selimthegrim 11y ago
    Axlines were suspended some time ago, I think they are still proactive with need based aid but I think with their tuition skyrocketing they can't possibly be as gracious as they once were.
kelukelugames 11y ago
Going to plug Professor Lauren Rivera's book again. [1] It stresses how school ranking are part of the system reducing economic mobility. 1. http://www.amazon.com/gp/product/0691155623/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0691155623&linkCode=as2&tag=wwwocfberkele-20&linkId=3TSKTACWO3KICFOR http://www.amazon.com/gp/product/0691155623/ref=as_li_tl?ie=...
TACIXAT 11y ago
My university asked for my earnings only one time. The categories maxed out at 70k+. They might be higher on that list if they took a more precise survey.
- tpudlik 11y ago
  The data used here does not come from the universities, but from the government. Specifically, from matching federal student loan applications (from the Department of Education) to tax returns (from the IRS).
nilkn 11y ago
If I were a prospective student interested in making a lot of money out of college, surely I'd still just follow this very simple model: Take all the schools I could get into, look up their actual median earnings, and go to the one which is highest (let's ignore issues of cost of attendance). If it turns out that the one which is highest isn't ranked high on this particular list, I don't see why that would suggest I still shouldn't pick that school. Imagine these are my options, for instance: (A) A school with expected earnings of $90k and actual earnings of $75k. (B) A school with expected earnings of $50k and actual earnings of $55k. (B) will rank vastly higher than (A) in this study, but I'd pick (A) over (B) every time if I just wanted money. In short, I don't actually see the value add here from this list. How am I supposed to act on these rankings? How are these rankings supposed to change any idea I might have about which school I should attend? It seems if you want to know which school to attend based on earnings we already have much more reliable data for that: actual earnings data.
- TheCoelacanth 11y ago
  The problem with your method (which this analysis is attempting to although may not actually correct) is what if the difference in the actual earnings between the schools is due to quality of the students admitted into the school rather than the increase in the students earnings that the school causes. For example, if your options are (A) A school that admits 1000 people with IQ of 200 who go on to earn an average of $100,000 and 100 people with IQ of 50 who go on to earn an average of $1,000 (B) A school that admits 100 people with IQ of 200 who go on to earn an average of $1,000,000 and 1000 people with IQ of 50 who go on to earn an average of $10,000 (A) has median earnings of $100,000 while (B) has median earnings of $10,000, so by your criteria (A) is much better. However, (B) has much better outcomes for both groups of students.
noelsusman 11y ago
The methodology feels odd. The results are essentially just errors in their prediction model. They're assuming that the model errors represent the school's contribution to median income. They're also assuming that those contributions are normally distributed around zero with constant variance. I understand why they made those assumptions but they're almost certainly not true, which isn't exactly ideal. Then again I can't quickly think of a better way to do it without more granular data. Maybe use a mixed effects model with multiple years of data from each school....
alistairSH 11y ago
Something feels "off" about a methodology that basically ranks all of the "top" state schools as poor values. Take the top-3 state schools in Virginia (by most other rankings, that's UVA, W&M, and VT). All three are nationally recognized. And all three are competitive that you need very good grades and a solid set of extra-curricular activities to be accepted. UVA ranks the lowest of the three, yet has the highest actual earnings. VT is ranked significantly higher with the middle earnings value. What does a student do with this? Cost of attendance at the three is similar. Should they attend VT with it's higher ranking, despite lower average earnings? Similar comments can be made about UNC-CH, UW-Seattle, UT-Austin. Also lacking from this analysis seems to be the loan burden borne by students. Georgetown and Villanova both rank very high in this list. But, both are insanely expensive to attend. Even with high actual earnings, it could take a decade or more for many students to pay off a potential six-figure loan.
- tpudlik 11y ago
  I don't think the Economist's ranking can be used for choosing which school to attend. It tells you how good a job schools do at boosting the earnings of the people who attend there---conditioned on who attends. In other words, they tell you how much the school does for a typical member of its student body. But if you're not representative of that student body (and you certainly won't be representative at many of the schools, especially the outliers!), this will not tell you how much the school would do for you.
  - alistairSH 11y ago
    I guess I just find it extremely hard to believe almost all of the nations "top" state schools have a negative impact (actual earnings less than expected) on the students that attend. Maybe I need to dig into the source of the expected earning figures. I'm definitely biased to some degree, being a graduate of one of the state schools with a poor ranking - I'm sitting here wondering where else I could have gone to get a better value.
- ThrustVectoring 11y ago
  The methodology is trying to remove the admissions department's effect on student success, leaving behind everything else (faculty, peers, course structure, university prestige, money vs prestige focus, etc).

115 points by mrsaint 11y ago

It would be very cool if you could sort by percentage differential between the over/under and expected earnings. I think sorting by that parameter would produce some really interesting results.

ryandrake 11y ago
Agreed. Ranking by raw dollar values means that schools in very expensive areas (or that feed into very expensive areas) will have exaggerated scores and end up either really highly ranked or really low. Another interesting study would be to see whether a child's college attendance correlates with a better or worse standard of living from their parents'. Maybe look at Father's salary at age 30 and child's salary at age 30, and see if the attended college is significant. The question being: If you're born rich, you tend to stay rich, and if you're born poor, you tent to stay poor. Does college choice matter?

cassieramen 11y ago

Does anyone know of a good diy college ranking system? One where you can pick that factors that actually matter to you. I'm always happy to see options other than U.S. News but I'd love to be able to tweak the algorithm myself. I think these one size fits all rankings are all flawed by their inherent nature.

harmegido 11y ago
The economist article linked here references (and uses in their rankings) data from https://collegescorecard.ed.gov/ https://collegescorecard.ed.gov/, which is put out by the US Dept of Ed. Looks like there's some additional metrics in there that may be useful.
brensudol 11y ago
I made a simple DIY ranking system using the same College Scorecard data last week. Here you go: http://bsudol.com/1PUPohk http://bsudol.com/1PUPohk You can select which variables are important and how much, and it generates a top/bottom 50 list. This was inspired by this article from NPR (http://www.npr.org/sections/ed/2015/09/21/441417608/the-new-college-scorecard-npr-does-some-math http://www.npr.org/sections/ed/2015/09/21/441417608/the-new-...)
- cassieramen 11y ago
  This is awesome! I'm happy the college score card data is being used for great project like this.
- vpalan2 11y ago
  This is great. As a next step, perhaps make the sliders more mobile friendly?
- Amorymeltzer 11y ago
  Submitted: https://news.ycombinator.com/item?id=10480851 https://news.ycombinator.com/item?id=10480851

6stringmerc 11y ago

The concept of studying in Higher Education for more opportunities, job stability, and income during one's lifetime is not to be dismissed. That said, if the predominant value of education is in the pursuit of money, then I think using the Brookings Institute model (which The Economist references) that includes two-year and vocational education entities is a much more pragmatic and practical approach. That is, if money is the goal, only focus on money as the outcome. Conflating money with prestige is, to me, foolish. A University is a prestige degree, insofar as there's a pursuit of knowledge, in theory, to produce a well-rounded, educated person...in theory. I say this as a graduate of two generally highly ranked Universities, who occasionally gets the feeling that I'd be making more money if I'd taken my education budget, got trained in HVAC service/repair/sales, and started my own company.

btilly 11y ago
Even if the predominant value of education is the pursuit of money, the Economist still has the more useful model. It allows you to differentiate between schools whose graduates make money because they are full of people who are going to try to make money, and schools that make money because the school brings something to the table that helps its graduates above other options. Look at CalTech for example. It does well in the Brookings model because it attracts a lot of people who want to be engineers. It does poorly in The Economist's model because its engineers are not making as much in 10 years as would be expected given a variety of factors that they control for. (Now that said, CalTech may be hurt because they graduate a lot of people who take side tracks through grad school and presumably would make more 15-20 years out. But you have to work with the data you have, and all they had was 10 years.) Now if I'm a student planning to go into engineering, the Brookings model merely confirms that I'm making an excellent choice. The Economist's model suggests that if I'm capable of going to CalTech, maybe that is not the best school for me.
- 6stringmerc 11y ago
  After reading through your response, I believe you missed my point. What I was referring to is that for a large population of potential students - those who wish to simply make more than if they do not go to a higher education program - going to a two year or vocational path school is exceptionally smart. Leaving that out of consideration is...not smart...highly biased...as in, how many readers of The Economist would consider sending their kid to become a diesel mechanic? From the research I've absorbed over the years, students that can afford to go to prestige universities often come from well-to-do, college educated families - these are high indicators of future success. For a much larger swath of the population that is trying to advance their stake in life, they are first-generation students, and often not well-served by taking on debt and attending a large University environment. Thus, many Universities are, practically speaking, very bad at ROI for a large contingent of students who simply want to make more money - vocational training? An excellent prospect. What remains to be seen is how much more earning potential there is in professions that require vocational training to be licensed or successful (i.e. welding, machining, air traffic control, plumbing, electrical contracting) as the generation currently working retires / dies from the workforce, thereby spiking demand, and in fields where a "traditional" University degree are, well, useless.

adenverd 11y ago

This tool would be a lot more useful if it allowed for filtering by chosen course of study. For example, University of Washington has a median earnings that is slightly below expected, but I can pretty much guarantee that their computer science graduates are earning well above median.

aidenn0 11y ago
It assumes you select a school that is focused on your field of choice, so if you pick an art school, your preference is for art, and if you pick an engineering school, your preference is for engineering.
- rifung 11y ago
  The issue is that you can't select the University of Washington's engineering school. You can only see University of Washington which has both an engineering and an art school, and so it doesn't tell you what the data will be if you only look at engineering students.
digikata 11y ago
That was my thought too, to make a simplistic example if a university had both say an engineering school and an art school, it might presumably do worse than a university with only an engineering school. So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries...
- _delirium 11y ago
  > So this metric might favor smaller, focused schools which happen to concentrate on education areas with high median salaries... I don't think that part's necessarily true. If a school focuses on an area with high median salaries, the model will take that into account in the predicted salaries, so the school will have to have even higher actual salaries than typical for the field (and its input demographics, SAT scores, etc.) to get a positive value-add. See Caltech for an example of a STEM-focused school that does badly by this measure: from its SAT scores, demographics, and heavy concentration of STEM majors, the regression analysis predicts that it should produce graduates with a median salary of $82k. But the actual median is $74k, so its value-add is taken to be -$8k. Some of the schools that do well are in areas with poor salaries, but score highly because they do better than you'd expect (or than the model would expect, anyway) for that area and student demographics. Otis College of Art and Design has a predicted salary of $29k from the regression analysis, but actual median is $42k, so implied value-add +$13k.

hackuser 11y ago

The Brookings study [1] mentioned by the the Economist offers this interesting analysis: 1. Graduates of some colleges enjoy much more economic success than their characteristics at time of admission would suggest. Colleges with high value-added in terms of alumni earnings are often focused on training for high-paying careers in technical subjects. ... 2. Four college quality factors are strongly associated with higher earnings for alumni: Curriculum value: The amount earned by people in the workforce who hold degrees in a field of study offered by the college, averaged across all the degrees the college awards; STEM orientation: The share of graduates prepared to work in STEM occupations; Completion rates: The percentage of students finishing their award within at least 1.5 times the normal time (three years for a two-year college, six years for a four-year college); Faculty salaries: The average monthly compensation of all teaching staff 3. Value-added measures are fairly reliable over time. ... (Personally, I think the most valuable things gained in college or in any education don't happen to have much monetary value.) [1] http://www.brookings.edu/research/reports2/2015/10/29-earnings-data-college-scorecard-rothwell http://www.brookings.edu/research/reports2/2015/10/29-earnin...

jldugger 11y ago
> (Personally, I think the most valuable things gained in college or in any education don't happen to have much monetary value.) Well, someone does, because they're charging for it.
- hackuser 11y ago
  > Well, someone does, because they're charging for it. Good point; I should have said, I don't think they are things that earn you money.

impendia 11y ago

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech. This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provided. I suspect that many Caltech undergraduates decide to pursue grad school and academia, which is not an especially lucrative career, and which is probably not highly corrolated with political leftism or "reefer madness". I can also tell you that my friends at Rice who were looking to make a lot of money after they graduated, by and large succeeded. In summary: Bullshit.

armaansarkar 11y ago
Didn't they kinda address this? "The bar is set extremely high for universities like Caltech, which are selective, close to prosperous cities and teach mainly lucrative subjects. If their students didn’t go on to extremely high-paying careers, the college would probably be doing something gravely wrong."
- impendia 11y ago
  I dunno, how about pursuing careers as scientists rather than as hacks for the finance industry? Is that "gravely wrong"? And as far as being close to a prosperous city -- that is because Caltech is near the world hub of the entertainment industry. A Caltech education is mostly irrelevant to finding employment there.
- slg 11y ago
  Isn't that an inherit flaw in the system? It is also based on the assumption that everyone prioritizes money over everything else. If I worked in the Yale marketing department I could easily spin this as a being caused by the high character of Yale graduates who are more willing to dedicate their life to lower paying but higher impact jobs like academics or public service while those greedy kids from Harvard only ever chase money.
  - 11y ago

hackuser 11y ago

Income is a poor measure of college education. The problem is that the valuable benefits of college are difficult to quantify; income has importance and is quantifiable, so I understand the temptation to use it as a metric, but it doesn't represent of the values of a college. It's like using the number of lines of code as a metric to represent the value of a piece of code - it's quantifiable, significant in some ways, but not representative of the code's value. I much prefer the Times Higher Education model, especially their reputation survey. They survey 10,000 tenured and published academics worldwide, using what looks like well-designed methodology [1]. These are people in a position to have expert knowledge about the qualities of various universities. Yes it's imperfect and those people have bias too, but I can't think of a better model: https://www.timeshighereducation.com/world-university-rankings/2015/reputation-ranking https://www.timeshighereducation.com/world-university-rankin... One excellent alternative is Washington Monthly's rankings. Their approach: We rate schools based on their contribution to the public good in three broad categories: Social Mobility (recruiting and graduating low-income students), Research (producing cutting-edge scholarship and PhDs), and Service (encouraging students to give something back to their country). More here: http://www.washingtonmonthly.com/college_guide/rankings-2015/national-universities-rank.php http://www.washingtonmonthly.com/college_guide/rankings-2015... ---- [1] https://www.timeshighereducation.com/world-reputation-rankings-2015-methodology https://www.timeshighereducation.com/world-reputation-rankin...

fiatmoney 11y ago

There seems to be a major issue with cost-of-living adjustments as well. Several rather obscure California colleges are highly ranked, and I assume this is because both wages and costs are higher in California, where many of their graduates stay after college.

wtvanhest 11y ago
Absolutely. Cost of living in an area will have a massive impact on exit salaries. The economist did not realize that the South and Mid-west have the lowest cost of living when they wrote the following: > Its upper tiers are dominated by colleges that emphasise engineering (such as Worcester Polytechnic) and attract students with high SAT scores (like Stanford). The lower extreme is populated by religious and art-focused colleges, particularly those in the south and Midwest. This number represents the benchmark against which we subsequently compare each college’s alumni earnings to produce the rankings.
dj_axl 11y ago
> There seems to be a major issue with cost-of-living adjustments Yep yep. Otis College of Art and Design "overperforming" for $42k salary is ridiculous. The title Junior Graphic Designer pays around $40k* in Los Angeles, where Otis is located. Therefore, the average for graduates is as-expected. *Source: Glassdoor.
rconti 11y ago
I'm not so sure. My generally-highly-regarded college in California (bay area, no less!) is almost bang-on average ($-33) despite being in a very high cost of living area.
- rconti 11y ago
  To reply to myself, the comments include this mention from "DR" of The Economist: ----- Geography (both city and state) are variables included in our model. Colleges in the Bay Area are not rewarded for the high salaries available there--they have to surpass a higher "bar" of expected earnings. That is why our eighth-ranked college is in West Virginia, and the ninth in Laredo, Texas. ------ Now, I don't know how they compensate for this. Cost of living at the place where the student ends up? Or where they study? If you study in a high cost of living area but move to Podunk, Iowa, your return will be far lower than expected, if they're only controlling for CoL at the college's geographical location. If they control for CoL at the student's new home, then okay.

leroy_masochist 11y ago

So basically, the methodology is that they do a very complex analysis of the student body to predict what their average earnings would be if they went to "college in general" and then compare that with what the graduates actually make. There's one thing that jumps out at me when I see the actual rankings: At least three of the eight schools that received a perfect 100 score -- W&L, Babson, and Bentley -- have a very, very high number of students who go to work for lucrative family businesses immediately upon graduation. In the case of W&L (which provides a great liberal arts education mind you) it's mostly southern good ol' boy/girl types. At Babson / Bentley (which provide great undergrad business educations), a huge chunk of the student body are the scions of the economic elite in developing countries, who are getting schooled up so they can be ready to be put in charge of something at a young age. My hunch is that the Economist's "expected earnings" methodology wasn't granular enough to take these idiosyncratic attributes into account, and that its r-squared of .85 might not be a rigorous number.

beachstartup 11y ago
> My hunch is that the Economist's "expected earnings" methodology wasn't granular enough to take these idiosyncratic attributes into account or, given the economist's core demographic (rich borderless global elites) this is exactly what they optimized for.
btilly 11y ago
Note that the data set they had only includes students who got federal loans. I strongly doubt that students who go to work for lucrative family businesses immediately upon graduation will be in the data set. But having classmates who look like that seems to be a really good economic choice! Which is a factor that is not apparent in normal college rankings.

mynameishere 11y ago

Well, I'm glad my shitty school beat Yale (almost last) so handily. Really, I don't think they could have come up with a worse metric. If you simply took each school's equivalent of this: https://en.wikipedia.org/wiki/List_of_Yale_University_people https://en.wikipedia.org/wiki/List_of_Yale_University_people ...and just sorted it by # of entries, you'd have a far superior list.

jpatel3 11y ago

Would love to get hold of csv or api to see state wide university data.

seibelj 11y ago

If a college has a huge communications program and a smaller engineering program, it would be ranked less because of pay disparities. This methodology is simply inaccurate

et2o 11y ago

This is hilariously bad. Schools that produce academics as opposed to those who go into high-paying professions are penalized, when in fact (at many of the top institutions) producing academics is one of the main goals.

stagger87 11y ago
This is not bad. It is simply one of many possible ways to rank colleges.
- BookmarkSaver 11y ago
  But it is a serious flaw in the methodology. It says nothing about the quality of education or the boost in earnings that a college will provide, if you are specifically going to a school to do so. At the very least they need to somehow account for individuals that go to the top schools but aren't looking to go into lucrative careers. CalTech is a common example being cited here. It is basically impossible to deny realistically that going to CalTech is a great way to make a lot of money. But it is in the bottom percentages of these rankings, likely because so many CalTech alumns go into academia. The metric is supposedly showing which colleges can potentially increase your earnings the most, but it just isn't doing that. It's a great idea, it just hasn't been implemented fully.
  - selimthegrim 11y ago
    I think the money thing is paralleling something else entirely - given the supposed caliber of student that comes into Caltech, is Caltech really making them better scientists than if they had gone elsewhere (or at least better scientists than they were turning out in the 1970s). That to me is a open question.
    - BookmarkSaver 11y ago
      Yes, but Caltech students have some of the highest rates of going into academia. Which pays well on a general scale, but compared to where engineers and developers go onto it drags the stats down. They do admit that the metric only gauges financial success and is imperfect when it comes to accounting for alternative priorities. But do you seriously believe that CalTech provides one of the worst "financial bumps" even accounting for the quality of their admissions? I find it incredibly hard to believe.

ronyeh 11y ago

The data makes more sense if you sort by expected earnings or median earnings and then compare within localized clusters of universities. For example, MIT and Caltech have similar expected earnings, but MIT wins on median earnings.

deleted 11y ago

[deleted]

obastani 11y ago

This analysis doesn't make any sense to me. Even by their own intent (i.e., best added value based on salary), their methodology is nonsensical. For example: Why should CalTech get hurt for being near L.A.? They're basically arguing that you're better off going to a school in the middle of nowhere because "hey, for being in such a crappy location, you did pretty well!". In an absolute sense you are better off going to CalTech, it's just that they might not leverage their advantage as well as some other schools. Not that they even show the last point -- it seems unlikely that the true model is linear (I'm guessing they used linear regression). For example, if the true model is closer to a sigmoid, then schools at the high end suddenly get unfairly penalized and schools near the low end get unfairly boosted. Finally, the statistical indicators are equally misleading. I can obtain an R^2 of 1.0 just by including indicators I[is COLLEGE_NAME] for each college. While that might not give you significance, the point is that getting good prediction is meaningless. I think what they really want is to restrict to predictors about the students. So, given that you're a straight A student with a 2400 SAT score, what would you expect to make coming out of each school? This at least tells me something about the added value to me of going to a certain school. (This approach is still prone to bias, but in the opposite direction -- there's a chance that the straight A student with a 2400 SAT score going to community college may have been smart but unmotivated, which might correlate with lower salary.) Edit: Here's another concern. They're claiming to have a model for "expected" earnings: earnings = A * (college covariates) + b + error but they can't distinguish between model error (i.e., error because their model is misspecified) vs. the school variation that they are trying to capture.

krstck 11y ago
> They're basically arguing that you're better off going to a school in the middle of nowhere because "hey, for being in such a crappy location, you did pretty well!". Well, no, not exactly. It's a subtle distinction, but what it's actually ranking is how well that school exceeds expectations, not best outcomes. This is not necessarily a list that will give a student the best school to go to, but rather (what it says on the tin) a scorecard for how well those schools are doing, given their resources.
- 11y ago

thefastlane 11y ago

i tried to look up several four-year U.S. institutions just now, including one major research university, and none were in the list. from the article, apparently they excluded universities when the scorecard dataset was missing one or more factors; it would be useful if the tool at least showed the university anyway as a placeholder and indicated what data was missing.

cgearhart 11y ago

>"The government generated the numbers by matching individuals’ student-loan applications to their subsequent tax returns, making it possible to compare pupils’ qualifications and demographic characteristics when they entered college with their salaries ten years later." So the dataset excludes students who do not apply for loans? (i.e., this analysis penalizes schools who admit the folks most likely to make lots of money, and the schools that have the lowest expected student contribution.) >"...based on a simple, if debatable, premise: the economic value of a university is equal to the gap between how much money its graduates earn, and how much they might have made had they studied elsewhere." If we are only to look at financial incentives, a more reasonable analysis would be to compare the expected future earnings _distribution_ as opposed to just a central tendency statistic like the median.

dragonwriter 11y ago
> So the dataset excludes students who do not apply for loans? (i.e., this analysis penalizes schools who admit the folks most likely to make lots of money, and the schools that have the lowest expected student contribution.) It penalizes (or at a minimum, undersamples in a nonrepresentative way) schools with their own, non-loan, aid programs. This probably very badly hurts schools like Caltech, since it means that there (unless Caltech no longer has the essentially full-coverage need-based aid they had when I went -- unsuccessfully, I graduated elsewhere) they are only counting people who are making the unwise choice of taking loans when they have no need given their current resources. That this will skew the data badly should be obvious.
- selimthegrim 11y ago
  Axlines were suspended some time ago, I think they are still proactive with need based aid but I think with their tuition skyrocketing they can't possibly be as gracious as they once were.

kelukelugames 11y ago

Going to plug Professor Lauren Rivera's book again. [1] It stresses how school ranking are part of the system reducing economic mobility. 1. http://www.amazon.com/gp/product/0691155623/ref=as_li_tl?ie=UTF8&camp=1789&creative=9325&creativeASIN=0691155623&linkCode=as2&tag=wwwocfberkele-20&linkId=3TSKTACWO3KICFOR http://www.amazon.com/gp/product/0691155623/ref=as_li_tl?ie=...

TACIXAT 11y ago

My university asked for my earnings only one time. The categories maxed out at 70k+. They might be higher on that list if they took a more precise survey.

tpudlik 11y ago
The data used here does not come from the universities, but from the government. Specifically, from matching federal student loan applications (from the Department of Education) to tax returns (from the IRS).

nilkn 11y ago

If I were a prospective student interested in making a lot of money out of college, surely I'd still just follow this very simple model: Take all the schools I could get into, look up their actual median earnings, and go to the one which is highest (let's ignore issues of cost of attendance). If it turns out that the one which is highest isn't ranked high on this particular list, I don't see why that would suggest I still shouldn't pick that school. Imagine these are my options, for instance: (A) A school with expected earnings of $90k and actual earnings of $75k. (B) A school with expected earnings of $50k and actual earnings of $55k. (B) will rank vastly higher than (A) in this study, but I'd pick (A) over (B) every time if I just wanted money. In short, I don't actually see the value add here from this list. How am I supposed to act on these rankings? How are these rankings supposed to change any idea I might have about which school I should attend? It seems if you want to know which school to attend based on earnings we already have much more reliable data for that: actual earnings data.

TheCoelacanth 11y ago
The problem with your method (which this analysis is attempting to although may not actually correct) is what if the difference in the actual earnings between the schools is due to quality of the students admitted into the school rather than the increase in the students earnings that the school causes. For example, if your options are (A) A school that admits 1000 people with IQ of 200 who go on to earn an average of $100,000 and 100 people with IQ of 50 who go on to earn an average of $1,000 (B) A school that admits 100 people with IQ of 200 who go on to earn an average of $1,000,000 and 1000 people with IQ of 50 who go on to earn an average of $10,000 (A) has median earnings of $100,000 while (B) has median earnings of $10,000, so by your criteria (A) is much better. However, (B) has much better outcomes for both groups of students.

noelsusman 11y ago

The methodology feels odd. The results are essentially just errors in their prediction model. They're assuming that the model errors represent the school's contribution to median income. They're also assuming that those contributions are normally distributed around zero with constant variance. I understand why they made those assumptions but they're almost certainly not true, which isn't exactly ideal. Then again I can't quickly think of a better way to do it without more granular data. Maybe use a mixed effects model with multiple years of data from each school....

alistairSH 11y ago

Something feels "off" about a methodology that basically ranks all of the "top" state schools as poor values. Take the top-3 state schools in Virginia (by most other rankings, that's UVA, W&M, and VT). All three are nationally recognized. And all three are competitive that you need very good grades and a solid set of extra-curricular activities to be accepted. UVA ranks the lowest of the three, yet has the highest actual earnings. VT is ranked significantly higher with the middle earnings value. What does a student do with this? Cost of attendance at the three is similar. Should they attend VT with it's higher ranking, despite lower average earnings? Similar comments can be made about UNC-CH, UW-Seattle, UT-Austin. Also lacking from this analysis seems to be the loan burden borne by students. Georgetown and Villanova both rank very high in this list. But, both are insanely expensive to attend. Even with high actual earnings, it could take a decade or more for many students to pay off a potential six-figure loan.

tpudlik 11y ago
I don't think the Economist's ranking can be used for choosing which school to attend. It tells you how good a job schools do at boosting the earnings of the people who attend there---conditioned on who attends. In other words, they tell you how much the school does for a typical member of its student body. But if you're not representative of that student body (and you certainly won't be representative at many of the schools, especially the outliers!), this will not tell you how much the school would do for you.
- alistairSH 11y ago
  I guess I just find it extremely hard to believe almost all of the nations "top" state schools have a negative impact (actual earnings less than expected) on the students that attend. Maybe I need to dig into the source of the expected earning figures. I'm definitely biased to some degree, being a graduate of one of the state schools with a poor ranking - I'm sitting here wondering where else I could have gone to get a better value.
ThrustVectoring 11y ago
The methodology is trying to remove the admissions department's effect on student success, leaving behind everything else (faculty, peers, course structure, university prestige, money vs prestige focus, etc).