13 ms·

> So much for that common, popular notion that standardized tests do not predict anything of value. To be fair, I don't think the debate was ever about the qua

• gameswithgo 5y ago
  Is the current push back against coding tests in job hiring perhaps similar to this push back against the SAT?
  • yeahwhatever10 5y ago
    I think that is more about the disconnect between coding tests and the actual day to day work and skills required to do the job. Example: I could have a high level of competency in software engineering and also not care how a mouse gets out of a bucket.
    • alecbz 5y ago
      Those "mouse getting out of a blender" brain-teasers or whatever are pretty unheard of at this point I think. Most people complain about coding questions, generally leetcode-style questions I think.
  • artful-hacker 5y ago
    It's similar but it also brings in a challenging problem: coding tests costs candidates far more than it costs employers in terms of time. I am currently interviewing and two of the companies I am otherwise excited for sent take-home tests that just exhaust me, especially after a long day of otherwise productive work. I've got 12 years of experience under my belt but somehow great references and a killer resume aren't enough to convince them I can find a security vulnerability.
    • geraldwhen 5y ago
      I do coding tests for the first interview. Nothing hard, just enough to do basic data modeling and writing a unit test. I also time cap to under an hour, and the internet is available as a resource. This filters out most people.
    • smilekzs 5y ago
      I respectfully disagree. Any decent interviewer spends the same man-hour as the candidate does.
  • adfgadfgaery 5y ago
    The SAT has been demonstrated to be effective at predicting success in university. We have almost no evidence about the computer industry's hiring practices. It is completely unscientific. Interviews operate on folklore, not statistics.
    • bluGill 5y ago
      This is something your HR department should be very concerned about. If the questions you ask during your interview are not useful in finding a good candidate why are you asking. This isn't just about time either, interviews have some strong laws around them so asking the wrong question could get you in court. I know when we wanted to do a coding test they told use we need to spend 6 months of giving everyone a coding test, have it independently graded by someone not involved in the hiring process. Then after people have worked here for 6 months we examine our actual results from those we hired and see if the tests at all predicted something useful. (or something like that - there is room in the scientific process for some variation)
      • alecbz 5y ago
        The bar below which HR has to be worried is not "we've scientifically determined that our interview questions lead to good on-the-job performance". There has to be some reasonable sense in which you could argue the interview filters for good candidates, but no one is requiring you run studies. Google once did a retrospective study and found that interview scores for people we ended up hiring were not correlated at all with people's on-the-job performance. I'm pretty sure nothing really changed as a result of this. I think it's a combination of the industry, especially FAANG, being kind of "stuck" on these kinds of interviews, and a lack of clearly better alternatives (I think there are better alternatives but it's not like I can point to studies backing me up). > I know when we wanted to do a coding test they told use we need to spend 6 months of giving everyone a coding test, have it independently graded by someone not involved in the hiring process. Then after people have worked here for 6 months we examine our actual results from those we hired and see if the tests at all predicted something useful. This is interesting but also way heavier weight than anything I've ever heard of. OOC where do you work? (Like vague description of kind of company, if you're not comfortable sharing the specific name).
  • hiq 5y ago
    > Is the current push back against coding tests in job hiring Is there such a pushback? As in, is the percentage of the workforce refusing to take such tests increasing?
    • alecbz 5y ago
      I don't know if there's a _rising_ pushback but you definitely do hear a not-small amount of complaining about coding interviews on HN.
      • hiq 5y ago
        Sure, but all in all, you just need 100 frequent HN users hating on these interviews to fill all related threads with such complaints, and there are dozens of millions of SWE worldwide. Big tech companies still hire mostly based on this type of interviews and they've been growing a lot lately, meaning that enough people apply to them. Leetcode and the likes have also democratized the process for many candidates. If anything, I'd say that the proportion of the workforce willing to submit themselves to these tests is increasing, but that's just a guess.
    • neon_electro 5y ago
      It very much depends on the style of coding test - personally, I'm more than happy to do take-home style tests where I prepare something in a matter of a few hours, but I can't stand "leetcode" interviews or anything where I'm pressured to produce in 30 minutes or less; perhaps that's because that's typically not how I work in the real world and in my experience, they do a really poor job of demonstrating my skill set and experience. I have terminated interviews before they even got started because of poor interview loop design from employers.
      • isbvhodnvemrwvn 5y ago
        From the other part of the table - we'd lose more candidates if we did take-homes. People in general prefer to study once and use that knowledge for multiple companies at once, you can't optimize take-homes like that.
  • nradov 5y ago
    It's more about labor market dynamics and supply versus demand. If there are plenty of developers available to hire then employers will insert extra hurdles in the process to filter out weak candidates (with the understanding that there will be some "false negatives"). But when the labor market is tight then employers will take a chance on any candidate who seems minimally competent because they need to fill the req.
• dijit 5y ago
  Unsure if I buy this. I'm definitely not the person you describe, but the idea of standardised testing being equivalent across all factors just strikes me as being fundamentally untrue. Personally I am very lucky to test well; and I definitely buy the notion that people who test well in SATs may go on to do better in University, but the reasons are probably the same: freedom from worry about financial circumstances will affect grades. 10 times in every 10.
  • Psyladine 5y ago
    >but the idea of standardised testing being equivalent across all factors just strikes me as being fundamentally untrue. What's your take on MIT's stance? our ability to accurately predict student academic success at MIT 02 Our research shows this predictive validity holds even when you control for socioeconomic factors that correlate with testing.
    • dijit 5y ago
      My take is exactly what I said. The same factors that lead to success for SATs can lead to further academic success. I believe that MIT is probably right, in fact, I'm quite certain of it. Many people will drop out of university or perform poorly than their peers for socio-economic reasons, the person working while studying will probably do worse than the person who just studies. MIT wants the most graduates and especially the most successful graduates, so the institution is right to do this, but I do still think it's more inhumane than I'm personally comfortable with -- but this is part of why I live in Europe where university students in general are seen as an investment by the state and not so much a business to be optimised.
      • chernevik 5y ago
        What's "inhumane" about trying to select those who will benefit most from your program?
      • tsimionescu 5y ago
        Doesn't most of Europe also rely on standardized testing for university admissions? My country definitely does so, and has for decades, both ore and post communist times. I also know France has the famous Bacalaureat at the end of high school.
• adfgadfgaery 5y ago
  The debate is about the quality and predictive value of the tests. Opponents claimed that the tests had a cultural bias so students from some backgrounds would do better than others, that students who had a good education before university would be better prepared, and that studying for tests or taking tests repeatedly has been shown to improve scores but is only accessible to people who can afford it. These are all claims that the tests are not good at predicting aptitude. The arguments against these tests are, of course, awful. Objective tests are the best way we know of to remove human bias. Aptitude tests (basically IQ tests) are the best way we know of to measure someone's natural ability (determined in early childhood) with little influence from their experience. Since their arguments make so little sense, it is reasonable to wonder about the psychology of opponents of standardized testing. But their arguments are, at least on the surface, about predictive value.
  • hkt 5y ago
    > it is reasonable to wonder about the psychology of opponents of standardized testing It is, at its core, a fear that testing largely reproduces the status quo. If one accepts the idea that there is an intellectual elite who constitute the highest strata of society, and that their gifts are innate and heritable rather than trained, it follows that social mobility is pretty much dead. It is a bleak vision. Personally I think there are different problems that are much bigger and woollier which keep people from non-elite backgrounds down, regardless of test outcomes. The structure of the education sector and employment more widely. Expectations about life and the distribution of rewards etc. We rarely have good quality, nonpartisan discussions about these things which I think pushes people to take views which are instrumental rather than informed.
    • adfgadfgaery 5y ago
      >it follows that social mobility is pretty much dead. It is a bleak vision. I have always found the idea of social mobility depressing. It assumes that we will always have a hierarchy, with some people who are powerful and prestigious and others who are poor and always feel inadequate. It assumes that we will always have an underclass but at least people can leave it.
• paulpauper 5y ago
  >To be fair, I don't think the debate was ever about the quality or predictive value of the tests. It is. The common argument is that GPAs are as predictive as SATs. MIT says it is not. I think the problem is you only need average ability to a good GPA, but a top 1-5% SAT score confers a higher ceiling of ability. MIT wants to admit exceptional students, not just average or above average ones.
• danShumway 5y ago
  I've pushed back against standardized testing at certain points of my life, and I don't think this comment even remotely summarizes my views. If anything, I would say that my views are the opposite -- homogenization creates a lack of differentiation around skill and aptitude based on questionable science (and sometimes outright pseudoscience) and often leads to an oversimplification of human intelligence in general. It always feels very strange to me that people trying to compress aptitude into a single number say that they're defending differentiation or diversity of talent. MIT's findings here don't really change my view of the value of SATs, although the findings are interesting and I think they're worth looking into further. I'm not sure "they're more predictive than GPAs" is the glowing recommendation that SAT proponents think it is. You can agree or disagree with me on that point, I'm not here to debate the entire idea of testing or IQ or whatever -- I just want to point out the above comment is a pretty big oversimplification and (in my mind) a borderline complete misrepresentation (I assume unintentionally) of what people like me believe. I can only speak for myself though, maybe there are people out there who do hate the idea of excellence.
  • zaidf 5y ago
    What's a better alternative in your view?
    • danShumway 5y ago
      I'm not completely sure. I think MIT's conclusions might be correct, they might be preferable to GPAs. I also think there might be other alternatives that aren't easy to implement, that require either a restructuring of how we do school or a better distribution of resources than we currently have. One conclusion that MIT hints at (although it doesn't say it outright) is that SATs might be a better indicator of success across economic levels in part because it's harder to buy a better SAT score with money. Looking at things like extracurricular activity runs into many of the same problems as looking at Github repos during hiring processes -- a lot of people don't have time to do a bunch of extracurricular activities, and access to those extracurricular activities is likely highly correlated with socioeconomic status. It might be difficult to move in that direction when access to school resources varies so much between areas. I do think the SAT could be improved -- I think one really easy way would be to change how it's administered so that it optimizes less for formal test-taking skill. The really good thing about the SAT is that it's a less school-specific measure than GPA. So a better alternative might be a version of the SAT that kept a standardized metric but that either widened its scope significantly or was administered differently. I also want to put forward the idea that admissions might just be really hard, period, and there might not be an easy way to assess potential, and trying to figure out the easiest way to do it might be like asking, "what's the best way to teach a child to play an instrument in a single day?" ---- One really important point that I want to get across: there is a difference between a measure being good and a measure being "the least terrible option we have at the moment" -- and confusing the two can cause real harm. At the top of this thread I see the quote, "so much for that common, popular notion that standardized tests do not predict anything of value." And if that's somebody's attitude, then they're never going to find a better option because the whole thing is being approached through the lens of "see, we were right, this is a good metric." I think a lot of criticism of standardized testing, IQ, coding tests for hiring, etc... is not necessarily trying to destroy everything, it's just trying to point out that many of these measures are really bad and they shouldn't be treated with the respect they're often given. I think that someone can very easily both have the position, "yeah, MIT probably should use SAT scores alongside GPAs" and the position, "people place way too much confidence in these things as an indicator of success."
• sgustard 5y ago
  The MIT article literally says "diversity" is one of their goals. Seems like you're arguing against yourself.
  • tharne 5y ago
    > The MIT article literally says "diversity" is one of their goals. Seems like you're arguing against yourself. Except no one goes to MIT because it's "diverse", whatever that even means anymore. They go there because it is one of the best schools in the world.
    • crackercrews 5y ago
      It is actually possible that some people go to MIT because it has more diversity [1] than its very close competitor, Caltech. [2] But it's true that the first filter for these students is undoubtedly world-leading-technical-program. 1: https://www.collegefactual.com/colleges/massachusetts-institute-of-technology/student-life/diversity/chart-ethnic-diversity.html https://www.collegefactual.com/colleges/massachusetts-instit... 2: https://www.collegefactual.com/colleges/california-institute-of-technology/student-life/diversity/chart-undergraduate-racial-ethnic-diversity.html https://www.collegefactual.com/colleges/california-institute...
    • sgustard 5y ago
      We're talking about the school's goals in forming a class, not the applicants' goals. Most schools that are among the "best in the world" find they can weigh multiple factors to decide who to admit, and there's no single magic number that does that job for them.