3 ms·
A bit off topic, but honestly meeting people at top schools and mid/low ranked schools, I don't see much of a difference except in the opportunities these peopl
by godelski 3y ago
A bit off topic, but honestly meeting people at top schools and mid/low ranked schools, I don't see much of a difference except in the opportunities these people have. Whether it is physics or CS, I don't see a meaningful quality difference in their work, only that the Stanford/MIT/Berkeley kid gets more internships and has an easier opportunity getting a new job if they mess up at their old one. I routinely see people from very prestigious universities with PhDs fail on rather simple concepts that are within their domain (or answered by a google search). E.g. I've had someone from a top 5 CS school, with a PhD in ML, ask on my github to solve a tensor mismatch for them when my research code uses pytorch's nn.Embeddings and we direct them to the correct checkpoint to use (but they don't). (not our code so we don't directly provide the embedding checkpoint, but we indirectly do)
A bit on topic, if this trend is as strong as it appears to be (many people I talk to see the same thing) then it really shows an over valuing of elite schools and frames this entire argument a bit differently. It would make sense that this should happen too, as teaching positions are highly competitive so there's a strong trickle down effect. You routinely see at low ranked schools professors from high ranked schools (often in a soft retire, for location, or in a junior position). It would make sense that quality levelizes when a single top ranked school produces more PhDs than there are positions to fill nation wide. And on top of the general _trend_ that you can only work at a university lower in ranking than the one you attended (immediately post graduation).
- goalonetwo 3y agoI will echo this. The only things top school really do is gate-keep who is allowed to get in, providing as a consequence some sense of exclusivity. The quality of education does not seem to be that different. And even if it was, it doesn't matter that much in a field like CS that is mainly self-learned and gets reinvented every decade.
- rented_mule 3y agoI had an interesting experience a decade ago when I spent a week recruiting at UT Austin and the next week recruiting at MIT, both for software engineering jobs. UT Austin is no slouch, but it doesn't have the reputation of MIT for software engineering related majors. We collected several hundred resumes from each. Only the best resumes from UT Austin were as good as the worst resumes from MIT. But, talking to the students and interviewing them revealed that the quality distribution, in terms of what we wanted to see in interns and new grads, was much harder to tell apart. My conclusion was that the MIT admissions process did a great job of selecting for people who present themselves well on paper, probably because admission is so competitive. Both schools offer a great education for those who take full advantage of it. And we wanted to hire from that pool of people, regardless of school.
- godelski 3y agoI think part of this too is that the top universities are really good at metric hacking. I do agree that the biases are baked in, but the other aspect is the compounding effects. E.g. MIT students have a better "foot in the door" for internships, research opportunities (especially at undergrad), and so on. Not only does their name do lifting, but they get more recruiters and the universities themselves do a lot of lifting. We know many internships are provided not through blind applications but connections (often grad students get internships via their advisors). These will definitely make the resumes look far better, but don't necessarily correlate with skills. It gets worse because this metric hacking is everywhere. As a researcher I honestly believe that were I at a higher ranked university more of my papers would be published in conferences (instead of arxiv or dead), I'd have to refine less to get published, and I'd have double the citations (university evangelism. Just being a MIT student will get you more Twitter followers than being a UT Austin student, so more people see when you post about your work). None of these things have to do with the actual quality of the work. Though of course, they can also tend to do "better" work in the sense that they have more resources. In a world where we have ill-defined goals and metrics compute becomes a large edge because instead of performing nuanced analysis you instead perform a more exhaustive analysis (the sledgehammer approach). Which is not necessarily better, though can (often) have lower entropy. As I posted in a sibling comment, measuring and evaluating things is insanely hard. Idk why as our world is increasing in complexity we've adopted to simplifying things more and more. Maybe the double edge sword of access to knowledge is feeling more knowledgeable than we are.
- rented_mule 3y agoYeah, it certainly can be a self-fulfilling prophecy. One of the MIT students we hired eventually left us to create a startup when they were accepted into a Y Combinator class. When it came time to pitch for their series A at Andreessen Horowitz, their YC advisor told them to wear their college sweatshirts to the pitch (MIT and an ivy). Apparently, the business plan / product hardly even came up in the pitch - according to one of them, it was just a "bro-down about our schools." They got the money.
- 3y ago
- andromeduck 3y agoSame experience herw - I've met Stanford/MIT grads and PhDs that struggled with the simplest things the brand has entirely worn off. I think I'd have an easier time trusting a tier 2 grad who "made it" than tier 1s in general, at least as far as technical competency is concerned.
- m463 3y agoI've known both Stanford and MIT grads who were top notch. That said, they worked at top firms, so there's some selection going on.
- andromeduck 3y agoOh yeah of course many of them are top notch - but it feels like ivy's produce a lot more duds too.
- tester756 3y agoThat's why I wish there were exams just like high-school ending exams. This way it aint as relevent where you went, but what your score was, so people from the "bottom" can bootstrap themselves easier. In India there's such a thing >Graduate Aptitude Test in Engineering (GATE) is an entrance examination conducted in India that primarily tests the comprehensive understanding of undergraduate subjects in engineering and sciences for admission into postgraduate programs.
- godelski 3y ago> That's why I wish there were exams just like high-school ending exams. I'm not sure exams are good at measuring actual abilities. Measuring things is an insanely difficult problem that isn't given enough credit. All too often people think it is just to perform an experiment and measure the outcome but this is really only a tractable when we have a well defined mathematical theory to test our results against. Most often in the real world this is done empirically and in these cases there are loads of assumptions that are being made and often not realized being made. Assumptions that are highly impactful. They come from the data that is being measured as well as the metrics being used to measure them. Remember, metrics are models too, and like all models are "wrong." I'll give two points of evidence to why having a testing system is likely ineffective: 1) These elite schools already have exams that correlate strongly with who is admitted. This gives evidence that exams like SAT and ACT or even the GRE do not do a good job at filtering the issue given the downstream observation. 2) You mention India and the testing structure. Several other countries have similar structures. Personal beliefs aside[0], the fact that India and other countries do this already results in a baked in effect into the US job market. Presuming the effect that I and others are observing is real, then these too look to be ineffective metrics. Since these metrics are likely going to correlate strongly with university rankings as well. [0] Side note, I do not think these systems are good for countries. They appear to be highly ineffective and just promote gate-keeping. They additionally create a lot of strain and pressure on the society because they result in career bottlenecks. Without these systems people have larger opportunities to change later in life. People make highly effective career changes even into their late 30's but these tests are usually performed on people under 18. [1] Other note: meritocracies are flawed at the core due to many effects and this can be quite counterintuitive. First off is the well known Goodhart's Law, where systems optimize the cost function rather than the intended goal of the cost function. This is seen in India's famous Cobra effect, reinforcement (and other ML) learning models, bureaucracies, and so on. This is because the metric is not perfectly aligned with the intention behind the metric, which is often ill-defined and static in nature compared to the dynamic solution space. Aligning metrics is nearly impossible and has only been achieved in the most simple of cases. To learn more take any optimization class, learn about optimal transport and non-convex solutions. Metric hacking is quite common in the wild and we should be really aware of it. This even happens in the precise example we're talking about. I'm sure you've known people who are really good at taking tests but are not so good at generalizing their knowledge (using it in the real world or adapting to new situations). It is because they overfit the cost function. Remember, costs functions are models too. So the question here is about how to properly evaluate things (distinguishing from measuring). Evaluation requires nuance and processing the data and metrics. Most importantly interpreting them (this is where bureaucracies fail and why everyone hates them. Lack of nuance). The truth is that this is always fuzzy and inexact. The only real way to perform this properly is to incorporate noise and uncertainty into your evaluation method. Not to remove it completely, but to acknowledge it and have it play a role in your model. You should of course remove as much noise as possible (reduce), but you can never reduce it to zero and if you remove it from your model you won't be able to capture your uncertainties. These problems only get harder as we move into multi-dimensional data (things we try to measure). High dimensional spaces are highly non-intuitive and the geodesics are not going to be what you expect. This is actually a relatively new area of mathematics and it would be extremely naive to think we have a good understanding of this. (not to knock on the great work that people have done in these areas, but these are fucking hard problems and we haven't had as much time)
- solatic 3y agoI think there's a classic fallacy here of confusing signals with proof. Admittance to an elite program signals (with higher confidence) higher competency, but it can never prove it, largely because part of what determines competency is emotional engagement, which can naturally change long after admission to the elite program. You would think that there would be arbitrage here on the part of employers - find more people from non-elite backgrounds who are actually competent. And surely they exist, but they don't have a way to signal it. Application queues (for jobs, grants, etc.) are already incredibly long. Good luck finding the competent people in the pile who can't signal it.
- godelski 3y agoSure, that's possible. But at least with my observations the likelihood of the samples doesn't line up. It isn't that there's just some duds, it is that there are some duds and nearly all other samples are on par with those of others. Which puts the sampling pretty consistent with a random effect rather than a correlation with school ranking. But such a thing would be incredibly difficult to prove one way or another (that competency strongly correlates with ranking or no effect). But I did give a mechanism for a plausible explanation of no effect. I'll add that at least with csrankings.com if you do a regression analysis there's a huge correlation with ranking and number of faculty, but that should also make sense if you just look at the metric used to rank those schools.