5 ms·

It's still a positive predictor in general. It just means that, beyond a certain point, it's a negative predictor. Just like a GPA of 3.5+ is probably positivel

by oskarth 12y ago

It's still a positive predictor in general. It just means that, beyond a certain point, it's a negative predictor. Just like a GPA of 3.5+ is probably positively correlated with getting tenure, but maybe getting 4.0 is worse than getting 3.9. Watch the actual video to get the real take away: I think what it meant was that, everybody who gets hired at Google is pretty good. If I just had to pick someone off the street, I really want that contest winner, I got to take him every time. Or her. But if it's somebody who passed the hiring bar, then they are all pretty much at the same level. And maybe the ones used to winning the contest, they are really used to going really, really fast and cranking the answer out and moving on to the next thing. And you perform better on the job if you are a little more reflective. And go slowly and make sure you get things right. -- Peter Norvig.

gwern 12y ago
Yeah, this might be an instance of http://en.wikipedia.org/wiki/Berkson%27s_paradox http://en.wikipedia.org/wiki/Berkson%27s_paradox - correlations after selection/conditioning may be different than before. Not well known, but relevant to a lot of the stuff about Google's hiring in particular (eg I wasted a lot of time explaining to people 2 years ago or so that Google's HR guy saying that intelligence didn't matter to their hiring decisions != intelligence doesn't matter to job performance (it does, as has been demonstrated many times), because Google was already selecting for <1% of the population.)
- stretchwithme 12y ago
  Intelligence SHOULD have mattered when that HR guy was hired. Perhaps he means it in the Carol Dweck sense, that how we view our failures is what determines our success. IQ tests only measure what we've learned already, not our attitude towards learning.
  - diminoten 12y ago
    No, you're missing the whole point. Intelligence doesn't matter because everyone is smart who applies to Google.
blazespin 12y ago
So, what he's saying, if you have nothing else that shines on your resume - highlight your programming contest wins. After you have some more stuff under your belt (patents, project wins), than stop emphasizing the contests. Indeed, what Norvig might be missing, is that the more methodical folks might also be contest winners but they chose not to highlight that because they don't think it's worth talking about compared to their other accomplishments. You can only put so much on a resume.
joe_the_user 12y ago
I don't think that should take away from the argument of the article, however: programming contest winners are used to cranking solutions out fast and that you performed better at the job if you were more reflective and went slowly and made sure things were right. The idea that a programmer can be very good at one level and have that goodness fail at other levels is important to consider. It is important to consider that a combination of skills, from cranking code to considering the impact of that code, matter. This consideration can lead to an entire team being valued rather than, say, the glorification of the "rock star" programmer.
- Ruberik 12y ago
  I think it's fair to say that programming contest winners are exceptionally good at cranking solutions out fast; but they could be just as good at working reflectively in a team as everyone else, or even better. The author of this article appears to think that he or she has provided evidence for the sentence you quoted, but a number of comments in this thread have pointed out that isn't what the evidence says; so the quoted sentence is simply speculation.
- Gibbon1 12y ago
  One of the things I noticed in school and later working is a lot of people excel when the problem is spoon fed to them. When the constraints and and desired answer are very mechanical. As it gets fuzzier, a lot of people even 'smart' people start having real issues. And you see the usual cognitive blockers kick in. 'frustration, fear, anger' Meaning, the usual academic problem where you are given the inputs, and only the inputs you need, and a desired solution, doesn't actually match up that well with real world problems. The other thing I've seen is a _lot_ of formally trained engineers are motivated by puzzles. Once they've solved it, all the other things that apply to the real world, like deadlines, shipping projects, making customers happy, and making money for the company are not the least amount interesting to them.
sytelus 12y ago
Being a champion at something requires excruciating narrow focus on something for unusually long time. If you are getting GPA of 4.0 or play Rachmaninoff's Piano Concerto No 3 or deadlift 400 pounds or in top 1000 chess players - you probably have to work on it for hours a day for years while ignoring everything else (of course unless you are one of those 1 in million polymath). People getting in to final rounds of programming contests are usually not the best programmers in the world in terms of breadth, maturity and exposure to other aspects of programming. They typically have put enormous effort on super optimizing their mental structure to do nothing but learn to master every little trick necessary to win the contest. This typically means memorizing implementation of pretty much all popular solutions in these contests, mastering odd tricks while ignoring general insights, less attention on elegance and maintainability, less incentivized to work in group, more tuned to getting any solution as quickly as possible instead of striving to get best possible solution, ignoring any areas that are not popular in such contests etc. This kind of brain training obviously has its negative. However on average, winners from contests will usually out perform your average programmer any day, they just won't outperform our super stars who would probably disqualify well before semi-finals because they were busy in learning other things that are not useful at competitions. My observations from dealing with one such person (I know, a great statistical sample size) very closely was that they are extremely fast learners and usually gravitate toward working on small very well defined problem that they already know how to solve but they suck at getting big picture, learning something large and complex and understanding the intricate interdependencies involved in complex systems.
- oskarth 12y ago
  In general these things tend to follow a logistic curve. It's a useful meta-skill to know when investing more time has marginal utility. I think your examples are great, if you just change deadlift 400 pounds to maybe ~500-600 pound, (or roughly 2.5-3x bodyweight for adult men) [1]. Most adult men can reach a 400 pound deadlift in less than 2-3 years of lifting (that's <5h/week). I mention this only because a lot of people here on HN are probably lacking physical activity, compared to what human were evolutionary selected for, and it's useful to know what can be expected. It's generally more than most people think. 1: See for example http://chrissalvato.com/2009/12/skill-guidelines-for-building-strong-useful-adaptable-athletes/ http://chrissalvato.com/2009/12/skill-guidelines-for-buildin...

by oskarth 12y ago

gwern 12y ago

Yeah, this might be an instance of http://en.wikipedia.org/wiki/Berkson%27s_paradox http://en.wikipedia.org/wiki/Berkson%27s_paradox - correlations after selection/conditioning may be different than before. Not well known, but relevant to a lot of the stuff about Google's hiring in particular (eg I wasted a lot of time explaining to people 2 years ago or so that Google's HR guy saying that intelligence didn't matter to their hiring decisions != intelligence doesn't matter to job performance (it does, as has been demonstrated many times), because Google was already selecting for <1% of the population.)

stretchwithme 12y ago
Intelligence SHOULD have mattered when that HR guy was hired. Perhaps he means it in the Carol Dweck sense, that how we view our failures is what determines our success. IQ tests only measure what we've learned already, not our attitude towards learning.
- diminoten 12y ago
  No, you're missing the whole point. Intelligence doesn't matter because everyone is smart who applies to Google.

blazespin 12y ago

So, what he's saying, if you have nothing else that shines on your resume - highlight your programming contest wins. After you have some more stuff under your belt (patents, project wins), than stop emphasizing the contests. Indeed, what Norvig might be missing, is that the more methodical folks might also be contest winners but they chose not to highlight that because they don't think it's worth talking about compared to their other accomplishments. You can only put so much on a resume.

joe_the_user 12y ago

I don't think that should take away from the argument of the article, however: programming contest winners are used to cranking solutions out fast and that you performed better at the job if you were more reflective and went slowly and made sure things were right. The idea that a programmer can be very good at one level and have that goodness fail at other levels is important to consider. It is important to consider that a combination of skills, from cranking code to considering the impact of that code, matter. This consideration can lead to an entire team being valued rather than, say, the glorification of the "rock star" programmer.

Ruberik 12y ago
I think it's fair to say that programming contest winners are exceptionally good at cranking solutions out fast; but they could be just as good at working reflectively in a team as everyone else, or even better. The author of this article appears to think that he or she has provided evidence for the sentence you quoted, but a number of comments in this thread have pointed out that isn't what the evidence says; so the quoted sentence is simply speculation.
Gibbon1 12y ago
One of the things I noticed in school and later working is a lot of people excel when the problem is spoon fed to them. When the constraints and and desired answer are very mechanical. As it gets fuzzier, a lot of people even 'smart' people start having real issues. And you see the usual cognitive blockers kick in. 'frustration, fear, anger' Meaning, the usual academic problem where you are given the inputs, and only the inputs you need, and a desired solution, doesn't actually match up that well with real world problems. The other thing I've seen is a _lot_ of formally trained engineers are motivated by puzzles. Once they've solved it, all the other things that apply to the real world, like deadlines, shipping projects, making customers happy, and making money for the company are not the least amount interesting to them.

sytelus 12y ago

Being a champion at something requires excruciating narrow focus on something for unusually long time. If you are getting GPA of 4.0 or play Rachmaninoff's Piano Concerto No 3 or deadlift 400 pounds or in top 1000 chess players - you probably have to work on it for hours a day for years while ignoring everything else (of course unless you are one of those 1 in million polymath). People getting in to final rounds of programming contests are usually not the best programmers in the world in terms of breadth, maturity and exposure to other aspects of programming. They typically have put enormous effort on super optimizing their mental structure to do nothing but learn to master every little trick necessary to win the contest. This typically means memorizing implementation of pretty much all popular solutions in these contests, mastering odd tricks while ignoring general insights, less attention on elegance and maintainability, less incentivized to work in group, more tuned to getting any solution as quickly as possible instead of striving to get best possible solution, ignoring any areas that are not popular in such contests etc. This kind of brain training obviously has its negative. However on average, winners from contests will usually out perform your average programmer any day, they just won't outperform our super stars who would probably disqualify well before semi-finals because they were busy in learning other things that are not useful at competitions. My observations from dealing with one such person (I know, a great statistical sample size) very closely was that they are extremely fast learners and usually gravitate toward working on small very well defined problem that they already know how to solve but they suck at getting big picture, learning something large and complex and understanding the intricate interdependencies involved in complex systems.

oskarth 12y ago
In general these things tend to follow a logistic curve. It's a useful meta-skill to know when investing more time has marginal utility. I think your examples are great, if you just change deadlift 400 pounds to maybe ~500-600 pound, (or roughly 2.5-3x bodyweight for adult men) [1]. Most adult men can reach a 400 pound deadlift in less than 2-3 years of lifting (that's <5h/week). I mention this only because a lot of people here on HN are probably lacking physical activity, compared to what human were evolutionary selected for, and it's useful to know what can be expected. It's generally more than most people think. 1: See for example http://chrissalvato.com/2009/12/skill-guidelines-for-building-strong-useful-adaptable-athletes/ http://chrissalvato.com/2009/12/skill-guidelines-for-buildin...