24 ms·

Good at programming competitions does not equal good on the job [video]

372 points by jpn 9y ago

sghiassy 9y ago
Question? Does being good at programming competitions make you good at interviewing for programming jobs? Obviously there's some irony in that question. But I also think there's some truth to it as well.
- msteffen 9y ago
  As a former ICPC world finalist who later worked at Google, I would say that practicing for programming contests is almost like cheating as far as Google interviews are concerned. The Google interview format is almost identical to the ICPC practice sessions I did in college. I've seen this video before, and IMO the extreme similarity between programming contest questions and Google interview questions could explain the negative correlation. Specifically, borderline engineers with programming contest experience are more likely to get hired than borderline engineers without programming contest experience, and therefore the set of Google engineers with programming contest experience includes more borderline people than the set of Google engineers without programming contest experience. Thus the slight negative correlation.
  - cbhl 9y ago
    When I was in high school, being good at programming contests meant picking up a bunch of habits that you'd never use at work -- memorizing the same twenty #include lines that include every conceivable STL data struct you'd need; using single letter variable names; no comments whatsoever. If you and the next person come up with the solution at the same time, you might lose simply because the other person could type faster. You have to un-learn these habits for industry. Just about anyone can get exposure to a set of representative coding questions (see: the USACO training robot, or Cracking the Coding Interview), but training for these contests means spending XX hours a year under a time limit trying to write code from memory (because you don't have time to look things up in the manual).
  - opportune 9y ago
    I expect this to be the true root cause of the negative correlation as well. Technical questions are good at rooting out those not good at data structures and algorithms, and good at promoting those that do. However, you can be excellent at solving ICPC questions and not know anything about documentation, object oriented design, OS, databases, anything web related, etc. It's a very specific skill and practicing programming contests optimizes for it.
  - christophilus 9y ago
    This is the best explanation of the phenomenon I've read. I think you're exactly right.
- xyzzyz 9y ago
  In my opinion, based on my and my friends' experience, being good at programming competitions clearly makes you very good at interviewing. I don't think that being good at contests makes you bad at the job, though. Consider the following scenario: assume that skill of being good at contests is completely orthogonal to being good at the job (so the real correlation coefficient is 0, instead of negative, observed by Norvig). Then, since it's easier to to get hired if you're good at contests, the observed correlation coefficient inside a single company will be negative, due to selection bias. Depending on the effect sizes, the selection bias might even make observed correlation coefficient negative even if it's positive in reality (which I think it is). For a more intuitive explanation, consider a used car market buyer who has preference for black cars (this corresponds to algorithm-based interview process of companies like Google or Facebook). Even though the color of the car should have no correlation with its quality, the black cars owned by that buyer will more likely be worse quality than non-black ones.
  - thaumasiotes 9y ago
    > Depending on the effect sizes, the selection bias might even make observed correlation coefficient negative even if it's positive in reality (which I think it is). This is well known in SAT scores. SAT math and verbal scores have a very strong positive correlation in reality; inside almost all colleges they are negatively correlated, because colleges impose threshold effects below and above. (That is, colleges admit students in a narrow band of SAT scores, rather than admitting students who are above a minimum threshold.) You can stylize that into the idea that total SAT score is constant for the students in any given college, which easily explains why component scores would be negatively correlated within colleges.
williamsmj 9y ago
The title of the post understates the claim in the link, which is that, "being good at programming competitions correlates negatively with being good on the job".
- williamsmj 9y ago
  I tweeted this link this morning (apparently the first time it's been posted to Twitter, so perhaps how it ended up here on HN). I did so in response to the CTO of Kaggle tweeting "Super confused why we still use resumes. Get 100x the signal from domain profiles (GitHub, StackOverflow, Kaggle, etc.) & real work samples", which ... where to start [https://twitter.com/benhamner/status/883137638084956160 https://twitter.com/benhamner/status/883137638084956160].
- rquant 9y ago
  no the statement is ""being good at programming competitions correlates negatively with being good on the job, conditional on passing Google interviews" which is completely different statement. "being good at programming competitions" hugely positively correlated with "begin good on the programming job" unconditionally
  - sbierwagen 9y ago
    Do you have... any kind of evidence to support that claim?
    - Klockan 9y ago
      The overwhelming majority of people who can't code would fail horribly both at jobs and competitions, this effect will drown out everything else no matter how the distribution looks for the few percent who can code.
purpleidea 9y ago
Completely agree with Peter. I'm shit at those competitions (maybe not the worst ever) but I think I'm pretty good at my job. But I think there are also some who are good at both.
bluetwo 9y ago
Overfitting?
AndrewStephens 9y ago
I don't find this surprising. I played around on Top Coder enough to reach the first division but the problems presented were in no way related to my day to day work. While the people that did very well (I never got anywhere once I hit the top grade) might have had an excellent command of very specific data structures and techniques, non of the code would have been very useful in the real world. If I was interviewing someone, a career as a competitive programmer would not be a detriment but it would not count for very much overall. We are looking for creative thinkers that can work as a team.
NTDF9 9y ago
I have had a bad experience with an algorthmic topcoder. The guy has a brilliant mind. But he doesn't understand the big picture or why his code would fail when integrated with big codebases. He thinks his work is "done" when he has written his tiny little code with some queues and trees. Then he leaves it for someone else to integrate and thus really solve the problem. The best engineers I found were the ones who took ownership of their projects and had the ethic to dig deep. Not the ones who could solve toy algo problems.
CalChris 9y ago
The market is pretty good at sorting this out and I'm sure programming competition winners are fairly valued. I've seen more Berkeley, Stanford, MIT, CMU and IIT degrees rise to the top than other schools. I haven't seen any top coders. Maybe they're there and I haven't noticed. But I haven't seen them on resumes that I've culled through. It's probably something you might list on a first job resume but further on down the road, you'd just list your work and your education.
deleted 9y ago
[deleted]
agounaris 9y ago
Winning the dunk competition does not make your team be an NBA champion...
- rdiddly 9y ago
  Another sports metaphor: Being a good sprinter doesn't make you a good marathoner.
- BoiledCabbage 9y ago
  But winning the 3pt shooting contest is a pretty good proxy for shooting ability.
  - NTDF9 9y ago
    And I'm so glad that Golden State Warriors didn't just Hire Kevin Durant for his 3pt shooting abilities. He'd lose to a lot of people who are better 3pt shooters.
    - BoiledCabbage 9y ago
      True, but they do already have the prior two years winners in Curry and Klay. In fact 5 of the past 7 years 3pt shooting contents winners were present in the NBA finals this year. James Jones, Kevin Love, Kyrie Irving, Marco Belinelli, Steph Curry, Klay Thompson, Eric Gordon That's probably not a coincidence.
    - sjg007 9y ago
      So what happened to teamwork? We see dominant teams last 4-5 years and then fizzle. Wooden had 10 championships with 7 in a row. Celtics had 8 in a row of 10 championships.
      - NTDF9 9y ago
        Exactly! I'd hire a Durant (a high performing team player) over a sharp-shooting 3 pointer genius (fast algo coder)
paulcole 9y ago
Also important to remember that good at programming does not necessarily equal good at job. Excelling as a communicator, being an empathetic person, and having great interpersonal skills are just as (perhaps more) important than how well you can code.
potatolicious 9y ago
It continues to surprise and frustrate me that as an industry we continue to highly prize proxy signals for engineering skill, when engineering skill is so directly measurable. Why even bother with measuring things that are N-degrees removed from actual engineering, when you can just get people to engineer things? I know others on HN have been hammering this point home for years, but until something changes it deserves to be repeated ad nauseum: work samples work samples work samples work samples. Stop with the trivia questions. Stop with the contrived algorithms questions. Ask people to design systems, ask people to defend their designs, ask people to write real runnable code that directly relates to the work they will be doing at your job, ask people to review a real piece of code written at your company, ask people to critique real design produced at your company. Anything but what we're doing right now.
- forhayley 9y ago
  Nobody good has time to do work samples. Not to mention in terms of time invested they are always high asymmetric. Do you ever stop to consider why most top successful companies ask algorithm questions? Are they all really just so stupid?
- williamsmj 9y ago
  It's not that simple. Many good engineers, especially those who aren't young men, don't have existing work samples they are able to share. Perhaps, by choice or otherwise, they have a personal life that does not allow them to write code for free. And their employers won't allow them to share work code. And many good engineers are not willing to spend an unpaid weekend writing code auditioning for a job they aren't yet sure they want. And, as you know if you've ever worked with a disruptive colleague, or had customers, engineering in a team for a product is much more than just writing code.
  - softawre 9y ago
    You are allowed to pay somebody for a work sample.
  - angersock 9y ago
fatjokes 9y ago
I'm also a former ICPC world finalist and I'm willing to believe this is the case in a BigCo because: 1) Top ICPC competitors tend to be extremely socially awkward, and may not work well in groups. 2) Vast majority of engineering work is very algorithmically simple, thus these folks don't get a chance to "shine". However, I've noticed that they are popular in prop trading firms, where work tends to be in very small teams or individual. I don't know how their performance correlates to fund performance. If I were hiring, I'd still prefer to hire at least some top ICPC performers. The hard algorithms are rare---but can make or break your product. I also think the knowledge learned from programming contests is invaluable. I'd like to be able to discuss bipartite matching or min-cut with my colleagues without eliciting a blank look.
- lordnacho 9y ago
  Aren't contests normally under huge time pressure? I would think that a lot of people could solve even quite hard problems given ordinary conditions like having lots of time, colleagues, and reference materials available.
  - jmcgough 9y ago
    At the same time, if you've never seen a solution that uses something like DP, you're not going to see a DP problem and think "oh, I can use DP for this". Part of it is ensuring that programmers have awareness of the tools at their disposal, so they can pull them out when they're under a deadline to get something done on time that's performant
- joenot443 9y ago
  I agree for the most part, but I think it really depends on the product your team is building. I can think of at least a couple companies I've worked at where the hard algorithms weren't rare, they were actually non-existent. The product's success relied on user experience, a solid design, and exhaustively tested and maintained code. The reality with many companies these days is that the algorithmically hard problems are solved in the frameworks and libraries they use, it's simply not necessary for most engineers to understand their inner workings. I'm curious, would you prefer an engineer with solid software engineering and design knowledge, or one with minimal experience building real software, but very in-depth algorithm and CS theory knowledge?
FTA 9y ago
Look at programming competitions from an assessment perspective: what are they measuring and is what's being measured good or bad for a job? Ability to work under short time constraints (probably good), to hack out some solution that will work temporarily (good) but probably not solid (bad), to forego much time to consider the implications of design and implementation choices (bad), to develop without communication if solo competition (bad) or without communication outside of the small group of core developers (bad), to build a solution without getting feedback and refinements from stakeholders (bad), and so on.
0x4d464d48 9y ago
I haven't done a code competition before but my understanding is that code competitions reward cowboy coding over good engineering practice, i.e. implementing features while paying no heed to maintainability. When you have that mind set and start dealing with a codebase as monstrous as Google's it sounds like a recipe for some serious technical debt.
- forhayley 9y ago
  And yet Google absolutely loves hiring competitive programmers. Your point is somewhat alike to saying elite sprinters will not do well in marathons, however the point is elite sprinters will still be far better than any random person off the street.
pklausler 9y ago
As a long-time interviewer, I've learned that a candidate being good at programming competitions means that they're probably good at programming competitions. It's a weak signal either way for success or failure at interviewing and being able to do the job. Part of the problem, as we've discussed on HN so often recently, is that a programming interview has to waste time with FizzBuzz-style questions just to flag the candidates with great resumes, transcripts, and phone screenings who still actually can't program a computer to solve even a trivial problem.
- kafkaesq 9y ago
  who still actually can't program a computer to solve even a trivial problem. Or who get nervous, and freeze up.
pitt1980 9y ago
I'm not able to watch the video at my current computer but its actually really typical that when something becomes an advantage for being selected to a certain pool the success of the those in the pool after the selection will negatively correlate with that thing --------- the really obvious example of this is the hockey birthday thing from Malcolm Gladwell's Outliers people with the earlier birthdays were more likely to make it past each selection stage in becoming an NHL player but those with the later birthdays who were able to be selected in spite of their later birthdays, were typically more successful after the selection --------- The authors contend that the strategy might actually work against a team's success because they found that players born later in the year and drafted later actually had more productive hockey careers. Deaner said the study showed that men drafted in the second half of the year were about twice as likely to have successful careers in the NHL ??? reaching benchmarks like 400 games played or 200 points scored ??? than those born earlier in the year. "If the team wasn't making this mistake, they probably would have been more successful," he said. "The guys born in the first part of the year are much more likely to be busts." https://www.nhl.com/news/study-suggests-nhl-has-bias-in-favour-of-players-born-earlier-in-the-year/c-657724 https://www.nhl.com/news/study-suggests-nhl-has-bias-in-favo...
- chongli 9y ago
  I favour the simpler explanation: Campbell's law. [0] If programming contests are favoured, then people will optimize for them. Simplify/generalize even further and you get Goodhart's law. [1] [0] https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law [1] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- lordnacho 9y ago
  > the success of the those in the pool after the selection will negatively correlate with that thing Could be relevant: https://en.wikipedia.org/wiki/Berkson%27s_paradox https://en.wikipedia.org/wiki/Berkson%27s_paradox
  - pitt1980 9y ago
    Yeah, exactly
js8 9y ago
I would tend to agree, and while I was never very good at programming competitions, when I saw some of the winning code on Topcoder, I somehow lost interest. If that's the code I would write if I learned to be better at competing, then I should rather spend time learning something else. On the other hand, something like Project Euler or even Advent of Code is very nice, if you do it at your own pace, or for learning a new language.
- shortleon 9y ago
  Although, I do think that competitive programmers can become great programmers more often than not. They practice writing correct for-loops, branches, recursive functions, efficient graph searches, input-output, data structure uses etc. in a very fast pace. To be able to do that requires you to chunk a significant amount of information. I found that doing stuff very fast correctly makes you learn and internalize concepts quickly and more deeply. This is all unfortunately anecdotal, I'm not sure how to google for such research (not just in cp but in "learning to do stuff fast improves learning rate"). Yet every instrument I've played and tried to learn, the moment I tried to play stuff fast but correctly (be it drums or piano), I had to improve my technique, had to internalize the rhythm patterns (more complex rhythms are insanely difficult to do fast), memory etc. With it came a significant amount of progress. Same thing happened to me with language. I was speaking English for 20 years but still had trouble with fluent pronunciation (despite my writing feeling natural), I knew what I had to say but somehow my tongue got all tangled. Yet, when I tried learning some rap songs that use insanely fast diction, my speaking improved up to a point where it felt normal.
  - deleted 9y ago
    [deleted]
  - deleted 9y ago
    [deleted]
kinkrtyavimoodh 9y ago
Adding to the good points here, I think coding competitions superstars are also potentially more likely to have a diva-complex, while others are more likely to have an impostor complex when they get into a company like Google. The right amount of impostor complex (where it enables you but does not cripple you) is actually very helpful as it helps you learn and better yourself.
kazinator 9y ago
People who enter programming competitions are looking for some sort of glory: to be stars. When they don't get it from the job, they get bored. I suspect that the people good at programming competitions could easily perform well on the job, if the motivation were there. I don't think it necessarily has anything to do with short-term versus long-term problem-solving focus. There are plenty of short-term problems that you have to solve on the job to be effective. You're not doing them in competition against anyone such that if you procrastinate, you will lose, so the motivation vaporizes. Also, since job interviews are like programming competitions, people who are good at programming competitions figure they can easily get a job anywhere, and to do that repeatedly. They are not motivated into working hard by job security concerns.
- deleted 9y ago
  [deleted]
bit_logic 9y ago
Here's a list of companies that don't do this https://github.com/poteto/hiring-without-whiteboards https://github.com/poteto/hiring-without-whiteboards I hope it's the beginning of an industry wide trend away from these types of interviews.
rkunnamp 9y ago
Just like 'There is no such person as Good CEO', but only 'A Good CEO for a particular company, at a particular point of time', there is no such a person as 'Good Programmer', there is only 'A Good Programmer for a particular job, at a particular point of time'. Context matters a lot. One does not use an AK-47 to kill a mosquito. It is terrible for that job. But that does not make AK-47 a terrible weapon. Good at programming competitions does not equal good at any programming job, is a more appropriate sentence.
- dsfyu404ed 9y ago
  Have you tried killing mosquitos on the back of a deer[1] standing behind a tree at 400yd. Even the AK with it's heavy cartridge isn't enough. You want a full power rifle, especially if the tree is a hardwood. /s
- kpil 9y ago
  I think that there are personal traits that make people perform good in most situations, but perhaps not all extremes. As continuity is normally good, you try to find those, and change only if you really have to. A programming competition evaluates almost none of the traits thats doing a marathon run as a highly preforming team.
mcv 9y ago
I once had an online discussion about the difference between competitive programmers and professional programmers, and which were better. Someone argued that competitive programmers were better, because they had to perform in more extreme circumstances. As he put it: they were sent into the forest with a knife to kill a lion. Everybody else ran with that metaphor. Someone asked what a lion was doing in a forest; don't they live on the savannah? I asked whether he was sure there was a lion; plenty of times I've been sent to kill a lion and ended up having to kill a goat or an elephant instead. Are we even sure it needs killing? I think that's the difference between a competitive programmer and a professional programmer. The competitive programmer will be much faster with a solution to the given problem, but the professional programmer will solve a better problem.
- dsacco 9y ago
  That conclusion seems just as arbitrary as the one you're contesting.
- wlesieutre 9y ago
  Mountain lions like forests just fine! Unfortunately for them, we chopped most of the forests down. Assuming you're in the US and somebody tells you there's a lion around, pretty good odds it's not one of the big orange ones. EDIT: Bay area published yesterday http://www.nbcbayarea.com/news/local/Several-Mountain-Lion-Sightings-Shut-Down-Trail-at-Alum-Rock-Park-432968923.html http://www.nbcbayarea.com/news/local/Several-Mountain-Lion-S...
  - mcv 9y ago
    A mountain lion is a very different animal from a lion. Different subfamily and genus, even. I don't live in the US, but when someone tells me "lion", I tend to assume they mean a lion, and not a puma. Though if there's any doubt, that's absolutely something to ask questions about.
lr4444lr 9y ago
In the chapter of his interview in Coders at Work[0], Norvig found that the strongest correlate in the interview process with success at Google was paradoxically to have been given the lowest possible score by one of the interviewers. He surmised that this was because in order for such a person to have even gotten hired at the end of the process, someone trustworthy must have seen so much potential in the prospective hire that he strongly advocated for the person to be offered a position which worked in spite of that other low rating. Kind of ironic for a company whose product values are so tightly tied to quantitive data. [0]http://www.apress.com/us/book/9781430219484 http://www.apress.com/us/book/9781430219484
- andrewla 9y ago
  A qualitative assessment of that factor is hard because Google doesn't hire the people that it doesn't hire. Another assessment could be that a divergence of opinion among interviewers is itself a positive sign -- programmers with strong controversial opinions who are willing to hold to them even in an interview setting might be better programmers for that. A less sanguine assessment is that "success at Google" correlates with people who generate controversy around themselves, simply because that is something that creates visibility.
- aoeusnth1 9y ago
  This isn't surprising - in fact, it would be surprising if it wasn't the case. This fact says nothing about the ability of Google's hiring bar to distinguish between good and bad hires, except that it is not a perfect signal.
  - lr4444lr 9y ago
    Totally agree. It just serves as a useful reminder though not to fall prey to the fallacy of deliberately chasing the correlational measure as a matter of policy.
- ubernostrum 9y ago
  More recently, people who have and are willing to analyze data across many interviewees and interviewers at multiple companies (so they can see ongoing performance of a candidate a company passed on!) have pointed out: http://blog.interviewing.io/you-cant-fix-diversity-in-tech-without-fixing-the-technical-interview/ http://blog.interviewing.io/you-cant-fix-diversity-in-tech-w... After looking at thousands of interviews on the platform, we’ve discovered something alarming: interviewee performance from interview to interview varied quite a bit, even for people with a high average performance ... roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview.
jpn 9y ago
This: https://news.ycombinator.com/item?id=13739329 https://news.ycombinator.com/item?id=13739329
bjacokes 9y ago
Here are some of the positive qualities I would expect from a competitive programmer compared to the general CS population early in their career: - knowledgeable at algorithms and data structures - good at analyzing correctness and edge cases, even on simple non-algorithmic problems (e.g. FizzBuzz) - accustomed to working hard and learning new concepts. this attribute is not specific to competitive programmer – for example, I'd expect similarly from an open source contributor – but it's higher than for a typical college student. Some negatives I'd expect, which are fixable over the course of the course of their career: - over-confidence in code, under-testing - less skilled in OOP, coding style, version control systems, as well as web development or systems code (unless they have specific previous work in these areas) - sometimes looking down on gruntwork/rote as beneath them (like the view that pure mathematicians have towards applied math or statistics) I think that list of positive attributes often outweighs the potential negatives, especially during an internship or in the first year or two of someone's career. After that, I would expect many non-competitive programmers to have picked up on some of those advantages (code correctness, learning new concepts). I've tried to steer my own interviews away from algorithms (especially DP) and focusing more on giving problems that are relatively straightforward, while still being complicated enough that someone has to write precise code and identify/fix a few edge cases.
WalterBright 9y ago
All job interviews rely on proxies for whether someone will be good on the job or not, because the only way to know is to actually hire them. All proxies are inaccurate.
- softawre 9y ago
  Yeah. The best proxy is just to hire somebody as a temp contractor for a month or so if both parties can afford it.
- blacksmythe 9y ago
  Also, everyone is convinced that their proxy is the best achievable, although of course they test it only against false positives, not false negatives.
  - WalterBright 9y ago
    > everyone is convinced that their proxy is the best achievable In my experience, people merely hoped their process was good enough. And it usually was. The "we only hire the best" is marketing propaganda and they know it.
deleted 9y ago
[deleted]
bshanks 9y ago
I see a lot of comments speculating that there may be something wrong with programming competition winners, but this result might be a statistical curiousity unique to Google. It's possible that for most companies, good at programming competitions could still be positively correlated with job performance. For example, what if most people whom Google hires could have won some programming competitions if they had wanted to (a situation that most other companies are probably not in)? In this case, 'won programming competitions' could be data that is a lousy filter for Google, yet a good filter for other companies.
- closed 9y ago
  Good point, in essence it sounds like one theory people have is that competition winners have over-specialized, but your theory is that there could be floor effects in the range of ability at Google (i.e. the floor is high, many people at Google could win competitions).
samlittlewood 9y ago
When I am interviewing: Candidate having competed in programming competitions - big green flag. But, if success in competitions is, in their view, their biggest asset - amber flag. I would extend this to most competitive endeavours.
hajderr 9y ago
well if you're not good at programming contests would that entail a contemplating/slow programmer? I'd rather pick an algorithmist and teach him/her to reflect than a so called "reflectionist" / "slow coder" and teach him/her how to solve algorithmic problems. I'd be interested in knowing the guy's (catonmat) own reflections and experiences too.
adavidoaiei 9y ago
They are two types of business, first type business to business, software made to run internal in a company, the business run on this software, generally in this kind of software you use enterprise frameworks, the second type are business to consumer where anyone can made an account and use application, some software as a service are like that, they are too other types of web applications business to consumer. When you allow that anyone to make an account and use your application sometimes you couldn't rely on enterprise technology and it's better to write everything from scratch or to customize open source solutions. What they are testing in algorithms contests: 1.) the algorithm is correct with various test sets 2.) the performance of the algorithm, running time in milliseconds, there you should to know how to measure time complexity of algorithm with O(N) 3.) memory consumed, there you should know how much memory you are allocating for variables, for example a variable of type byte consume 8 bits I think that programmers which has prizes in algorithms contests are suitable for business to consumer applications because there they will find a challenge, this kind of programmers will be bored to dig in enterprise frameworks.
twii 9y ago
Depends also on what job I guess?
x1798DE 9y ago
In general, I think there are a lot of things like whiteboard interviews and coding competitions where employers would prefer "good at X means good on the job", but are reasonably happy to settle for "bad at X means bad on the job."
qdev 9y ago
My viewpoint is taken from the context as someone who is a reasonably seasoned developer returning to the job market. I never did programming competitions in university, though I surely was someone who fit the profile (computer science guy with a discrete math bent). As part of my recent prep for a job search, I joined one of the programming competition websites and did a couple of contests. I posit that one of the dangers of spending a lot of time doing programming competitions and becoming very proficient at them is that, perhaps, you can come to believe that "true" programming, some sort of Platonic ideal of programming, is about coming up with the clever insight that solves an algorithmic puzzle. But, in fact, a fair bit of _commercial_ programming is down and dirty, with databases, and user interfaces, and a lot of the time is really just shuffling data from one place to another, maybe filtering it or combining it with another set of data. And that's just at the beginning of your career. Later on in your career, success means being able to work at larger scales in a team. That means organizing the code in a way that supports the efficient development of the codebase by individuals like yourself, by your team, by the development group as a whole... And at the architect level, you perhaps are looking at designing the system to support the efficient operation of the entire organization. So I can easily believe that success at a programming competition does not correlate with long-term success as a software engineer in commercial software development. The two are really very different. (Btw, I actually found the competitions that I did to be fun, but mentally exhausting. I'd say go ahead and do them, especially if you have an inclination for those types of problems. Just be prepared to use a different mindset for commercial software development.)
aqp 9y ago
http://ruberik.blogspot.com/2017/07/no-programming-competitions-dont.html http://ruberik.blogspot.com/2017/07/no-programming-competiti... It looks like it wasn't "being good at programming competitions" that was negatively correlated with job performance. It was "participated in programming competitions". And there are some more "how to interpret machine learning models" caveats in that blog post. It seems to me the biggest factor in explaining this is that the people who are just below the hiring line but participate in competitions get a bump over the line. Since there are more people just below the line than above it, the "participates" group is bottom-heavy, producing the correlation. I do a lot of interviews, and it seems to me that lots of people with experience perform below how they "should", because they're not practiced at solving problems from scratch, they work all day on modifying larger systems. Programming competitions would fix that for them, as would most open-source hobby projects.

372 points by jpn 9y ago

Question? Does being good at programming competitions make you good at interviewing for programming jobs? Obviously there's some irony in that question. But I also think there's some truth to it as well.

msteffen 9y ago
As a former ICPC world finalist who later worked at Google, I would say that practicing for programming contests is almost like cheating as far as Google interviews are concerned. The Google interview format is almost identical to the ICPC practice sessions I did in college. I've seen this video before, and IMO the extreme similarity between programming contest questions and Google interview questions could explain the negative correlation. Specifically, borderline engineers with programming contest experience are more likely to get hired than borderline engineers without programming contest experience, and therefore the set of Google engineers with programming contest experience includes more borderline people than the set of Google engineers without programming contest experience. Thus the slight negative correlation.
- cbhl 9y ago
  When I was in high school, being good at programming contests meant picking up a bunch of habits that you'd never use at work -- memorizing the same twenty #include lines that include every conceivable STL data struct you'd need; using single letter variable names; no comments whatsoever. If you and the next person come up with the solution at the same time, you might lose simply because the other person could type faster. You have to un-learn these habits for industry. Just about anyone can get exposure to a set of representative coding questions (see: the USACO training robot, or Cracking the Coding Interview), but training for these contests means spending XX hours a year under a time limit trying to write code from memory (because you don't have time to look things up in the manual).
- opportune 9y ago
  I expect this to be the true root cause of the negative correlation as well. Technical questions are good at rooting out those not good at data structures and algorithms, and good at promoting those that do. However, you can be excellent at solving ICPC questions and not know anything about documentation, object oriented design, OS, databases, anything web related, etc. It's a very specific skill and practicing programming contests optimizes for it.
- christophilus 9y ago
  This is the best explanation of the phenomenon I've read. I think you're exactly right.
xyzzyz 9y ago
In my opinion, based on my and my friends' experience, being good at programming competitions clearly makes you very good at interviewing. I don't think that being good at contests makes you bad at the job, though. Consider the following scenario: assume that skill of being good at contests is completely orthogonal to being good at the job (so the real correlation coefficient is 0, instead of negative, observed by Norvig). Then, since it's easier to to get hired if you're good at contests, the observed correlation coefficient inside a single company will be negative, due to selection bias. Depending on the effect sizes, the selection bias might even make observed correlation coefficient negative even if it's positive in reality (which I think it is). For a more intuitive explanation, consider a used car market buyer who has preference for black cars (this corresponds to algorithm-based interview process of companies like Google or Facebook). Even though the color of the car should have no correlation with its quality, the black cars owned by that buyer will more likely be worse quality than non-black ones.
- thaumasiotes 9y ago
  > Depending on the effect sizes, the selection bias might even make observed correlation coefficient negative even if it's positive in reality (which I think it is). This is well known in SAT scores. SAT math and verbal scores have a very strong positive correlation in reality; inside almost all colleges they are negatively correlated, because colleges impose threshold effects below and above. (That is, colleges admit students in a narrow band of SAT scores, rather than admitting students who are above a minimum threshold.) You can stylize that into the idea that total SAT score is constant for the students in any given college, which easily explains why component scores would be negatively correlated within colleges.

williamsmj 9y ago

The title of the post understates the claim in the link, which is that, "being good at programming competitions correlates negatively with being good on the job".

williamsmj 9y ago
I tweeted this link this morning (apparently the first time it's been posted to Twitter, so perhaps how it ended up here on HN). I did so in response to the CTO of Kaggle tweeting "Super confused why we still use resumes. Get 100x the signal from domain profiles (GitHub, StackOverflow, Kaggle, etc.) & real work samples", which ... where to start [https://twitter.com/benhamner/status/883137638084956160 https://twitter.com/benhamner/status/883137638084956160].
rquant 9y ago
no the statement is ""being good at programming competitions correlates negatively with being good on the job, conditional on passing Google interviews" which is completely different statement. "being good at programming competitions" hugely positively correlated with "begin good on the programming job" unconditionally
- sbierwagen 9y ago
  Do you have... any kind of evidence to support that claim?
  - Klockan 9y ago
    The overwhelming majority of people who can't code would fail horribly both at jobs and competitions, this effect will drown out everything else no matter how the distribution looks for the few percent who can code.

purpleidea 9y ago

Completely agree with Peter. I'm shit at those competitions (maybe not the worst ever) but I think I'm pretty good at my job. But I think there are also some who are good at both.

bluetwo 9y ago

Overfitting?

AndrewStephens 9y ago

I don't find this surprising. I played around on Top Coder enough to reach the first division but the problems presented were in no way related to my day to day work. While the people that did very well (I never got anywhere once I hit the top grade) might have had an excellent command of very specific data structures and techniques, non of the code would have been very useful in the real world. If I was interviewing someone, a career as a competitive programmer would not be a detriment but it would not count for very much overall. We are looking for creative thinkers that can work as a team.

NTDF9 9y ago

I have had a bad experience with an algorthmic topcoder. The guy has a brilliant mind. But he doesn't understand the big picture or why his code would fail when integrated with big codebases. He thinks his work is "done" when he has written his tiny little code with some queues and trees. Then he leaves it for someone else to integrate and thus really solve the problem. The best engineers I found were the ones who took ownership of their projects and had the ethic to dig deep. Not the ones who could solve toy algo problems.

CalChris 9y ago

The market is pretty good at sorting this out and I'm sure programming competition winners are fairly valued. I've seen more Berkeley, Stanford, MIT, CMU and IIT degrees rise to the top than other schools. I haven't seen any top coders. Maybe they're there and I haven't noticed. But I haven't seen them on resumes that I've culled through. It's probably something you might list on a first job resume but further on down the road, you'd just list your work and your education.

deleted 9y ago

[deleted]

agounaris 9y ago

Winning the dunk competition does not make your team be an NBA champion...

rdiddly 9y ago
Another sports metaphor: Being a good sprinter doesn't make you a good marathoner.
BoiledCabbage 9y ago
But winning the 3pt shooting contest is a pretty good proxy for shooting ability.
- NTDF9 9y ago
  And I'm so glad that Golden State Warriors didn't just Hire Kevin Durant for his 3pt shooting abilities. He'd lose to a lot of people who are better 3pt shooters.
  - BoiledCabbage 9y ago
    True, but they do already have the prior two years winners in Curry and Klay. In fact 5 of the past 7 years 3pt shooting contents winners were present in the NBA finals this year. James Jones, Kevin Love, Kyrie Irving, Marco Belinelli, Steph Curry, Klay Thompson, Eric Gordon That's probably not a coincidence.
  - sjg007 9y ago
    So what happened to teamwork? We see dominant teams last 4-5 years and then fizzle. Wooden had 10 championships with 7 in a row. Celtics had 8 in a row of 10 championships.
    - NTDF9 9y ago
      Exactly! I'd hire a Durant (a high performing team player) over a sharp-shooting 3 pointer genius (fast algo coder)

paulcole 9y ago

Also important to remember that good at programming does not necessarily equal good at job. Excelling as a communicator, being an empathetic person, and having great interpersonal skills are just as (perhaps more) important than how well you can code.

potatolicious 9y ago

It continues to surprise and frustrate me that as an industry we continue to highly prize proxy signals for engineering skill, when engineering skill is so directly measurable. Why even bother with measuring things that are N-degrees removed from actual engineering, when you can just get people to engineer things? I know others on HN have been hammering this point home for years, but until something changes it deserves to be repeated ad nauseum: work samples work samples work samples work samples. Stop with the trivia questions. Stop with the contrived algorithms questions. Ask people to design systems, ask people to defend their designs, ask people to write real runnable code that directly relates to the work they will be doing at your job, ask people to review a real piece of code written at your company, ask people to critique real design produced at your company. Anything but what we're doing right now.

forhayley 9y ago
Nobody good has time to do work samples. Not to mention in terms of time invested they are always high asymmetric. Do you ever stop to consider why most top successful companies ask algorithm questions? Are they all really just so stupid?
williamsmj 9y ago
It's not that simple. Many good engineers, especially those who aren't young men, don't have existing work samples they are able to share. Perhaps, by choice or otherwise, they have a personal life that does not allow them to write code for free. And their employers won't allow them to share work code. And many good engineers are not willing to spend an unpaid weekend writing code auditioning for a job they aren't yet sure they want. And, as you know if you've ever worked with a disruptive colleague, or had customers, engineering in a team for a product is much more than just writing code.
- softawre 9y ago
  You are allowed to pay somebody for a work sample.
- angersock 9y ago

fatjokes 9y ago

I'm also a former ICPC world finalist and I'm willing to believe this is the case in a BigCo because: 1) Top ICPC competitors tend to be extremely socially awkward, and may not work well in groups. 2) Vast majority of engineering work is very algorithmically simple, thus these folks don't get a chance to "shine". However, I've noticed that they are popular in prop trading firms, where work tends to be in very small teams or individual. I don't know how their performance correlates to fund performance. If I were hiring, I'd still prefer to hire at least some top ICPC performers. The hard algorithms are rare---but can make or break your product. I also think the knowledge learned from programming contests is invaluable. I'd like to be able to discuss bipartite matching or min-cut with my colleagues without eliciting a blank look.

lordnacho 9y ago
Aren't contests normally under huge time pressure? I would think that a lot of people could solve even quite hard problems given ordinary conditions like having lots of time, colleagues, and reference materials available.
- jmcgough 9y ago
  At the same time, if you've never seen a solution that uses something like DP, you're not going to see a DP problem and think "oh, I can use DP for this". Part of it is ensuring that programmers have awareness of the tools at their disposal, so they can pull them out when they're under a deadline to get something done on time that's performant
joenot443 9y ago
I agree for the most part, but I think it really depends on the product your team is building. I can think of at least a couple companies I've worked at where the hard algorithms weren't rare, they were actually non-existent. The product's success relied on user experience, a solid design, and exhaustively tested and maintained code. The reality with many companies these days is that the algorithmically hard problems are solved in the frameworks and libraries they use, it's simply not necessary for most engineers to understand their inner workings. I'm curious, would you prefer an engineer with solid software engineering and design knowledge, or one with minimal experience building real software, but very in-depth algorithm and CS theory knowledge?

FTA 9y ago

Look at programming competitions from an assessment perspective: what are they measuring and is what's being measured good or bad for a job? Ability to work under short time constraints (probably good), to hack out some solution that will work temporarily (good) but probably not solid (bad), to forego much time to consider the implications of design and implementation choices (bad), to develop without communication if solo competition (bad) or without communication outside of the small group of core developers (bad), to build a solution without getting feedback and refinements from stakeholders (bad), and so on.

0x4d464d48 9y ago

I haven't done a code competition before but my understanding is that code competitions reward cowboy coding over good engineering practice, i.e. implementing features while paying no heed to maintainability. When you have that mind set and start dealing with a codebase as monstrous as Google's it sounds like a recipe for some serious technical debt.

forhayley 9y ago
And yet Google absolutely loves hiring competitive programmers. Your point is somewhat alike to saying elite sprinters will not do well in marathons, however the point is elite sprinters will still be far better than any random person off the street.

pklausler 9y ago

As a long-time interviewer, I've learned that a candidate being good at programming competitions means that they're probably good at programming competitions. It's a weak signal either way for success or failure at interviewing and being able to do the job. Part of the problem, as we've discussed on HN so often recently, is that a programming interview has to waste time with FizzBuzz-style questions just to flag the candidates with great resumes, transcripts, and phone screenings who still actually can't program a computer to solve even a trivial problem.

kafkaesq 9y ago
who still actually can't program a computer to solve even a trivial problem. Or who get nervous, and freeze up.

pitt1980 9y ago

I'm not able to watch the video at my current computer but its actually really typical that when something becomes an advantage for being selected to a certain pool the success of the those in the pool after the selection will negatively correlate with that thing --------- the really obvious example of this is the hockey birthday thing from Malcolm Gladwell's Outliers people with the earlier birthdays were more likely to make it past each selection stage in becoming an NHL player but those with the later birthdays who were able to be selected in spite of their later birthdays, were typically more successful after the selection --------- The authors contend that the strategy might actually work against a team's success because they found that players born later in the year and drafted later actually had more productive hockey careers. Deaner said the study showed that men drafted in the second half of the year were about twice as likely to have successful careers in the NHL ??? reaching benchmarks like 400 games played or 200 points scored ??? than those born earlier in the year. "If the team wasn't making this mistake, they probably would have been more successful," he said. "The guys born in the first part of the year are much more likely to be busts." https://www.nhl.com/news/study-suggests-nhl-has-bias-in-favour-of-players-born-earlier-in-the-year/c-657724 https://www.nhl.com/news/study-suggests-nhl-has-bias-in-favo...

chongli 9y ago
I favour the simpler explanation: Campbell's law. [0] If programming contests are favoured, then people will optimize for them. Simplify/generalize even further and you get Goodhart's law. [1] [0] https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law [1] https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
lordnacho 9y ago
> the success of the those in the pool after the selection will negatively correlate with that thing Could be relevant: https://en.wikipedia.org/wiki/Berkson%27s_paradox https://en.wikipedia.org/wiki/Berkson%27s_paradox
- pitt1980 9y ago
  Yeah, exactly

js8 9y ago

I would tend to agree, and while I was never very good at programming competitions, when I saw some of the winning code on Topcoder, I somehow lost interest. If that's the code I would write if I learned to be better at competing, then I should rather spend time learning something else. On the other hand, something like Project Euler or even Advent of Code is very nice, if you do it at your own pace, or for learning a new language.

shortleon 9y ago
Although, I do think that competitive programmers can become great programmers more often than not. They practice writing correct for-loops, branches, recursive functions, efficient graph searches, input-output, data structure uses etc. in a very fast pace. To be able to do that requires you to chunk a significant amount of information. I found that doing stuff very fast correctly makes you learn and internalize concepts quickly and more deeply. This is all unfortunately anecdotal, I'm not sure how to google for such research (not just in cp but in "learning to do stuff fast improves learning rate"). Yet every instrument I've played and tried to learn, the moment I tried to play stuff fast but correctly (be it drums or piano), I had to improve my technique, had to internalize the rhythm patterns (more complex rhythms are insanely difficult to do fast), memory etc. With it came a significant amount of progress. Same thing happened to me with language. I was speaking English for 20 years but still had trouble with fluent pronunciation (despite my writing feeling natural), I knew what I had to say but somehow my tongue got all tangled. Yet, when I tried learning some rap songs that use insanely fast diction, my speaking improved up to a point where it felt normal.
- deleted 9y ago
  [deleted]
- deleted 9y ago
  [deleted]

kinkrtyavimoodh 9y ago

Adding to the good points here, I think coding competitions superstars are also potentially more likely to have a diva-complex, while others are more likely to have an impostor complex when they get into a company like Google. The right amount of impostor complex (where it enables you but does not cripple you) is actually very helpful as it helps you learn and better yourself.

kazinator 9y ago

People who enter programming competitions are looking for some sort of glory: to be stars. When they don't get it from the job, they get bored. I suspect that the people good at programming competitions could easily perform well on the job, if the motivation were there. I don't think it necessarily has anything to do with short-term versus long-term problem-solving focus. There are plenty of short-term problems that you have to solve on the job to be effective. You're not doing them in competition against anyone such that if you procrastinate, you will lose, so the motivation vaporizes. Also, since job interviews are like programming competitions, people who are good at programming competitions figure they can easily get a job anywhere, and to do that repeatedly. They are not motivated into working hard by job security concerns.

deleted 9y ago
[deleted]

bit_logic 9y ago

Here's a list of companies that don't do this https://github.com/poteto/hiring-without-whiteboards https://github.com/poteto/hiring-without-whiteboards I hope it's the beginning of an industry wide trend away from these types of interviews.

rkunnamp 9y ago

Just like 'There is no such person as Good CEO', but only 'A Good CEO for a particular company, at a particular point of time', there is no such a person as 'Good Programmer', there is only 'A Good Programmer for a particular job, at a particular point of time'. Context matters a lot. One does not use an AK-47 to kill a mosquito. It is terrible for that job. But that does not make AK-47 a terrible weapon. Good at programming competitions does not equal good at any programming job, is a more appropriate sentence.

dsfyu404ed 9y ago
Have you tried killing mosquitos on the back of a deer[1] standing behind a tree at 400yd. Even the AK with it's heavy cartridge isn't enough. You want a full power rifle, especially if the tree is a hardwood. /s
kpil 9y ago
I think that there are personal traits that make people perform good in most situations, but perhaps not all extremes. As continuity is normally good, you try to find those, and change only if you really have to. A programming competition evaluates almost none of the traits thats doing a marathon run as a highly preforming team.

mcv 9y ago

I once had an online discussion about the difference between competitive programmers and professional programmers, and which were better. Someone argued that competitive programmers were better, because they had to perform in more extreme circumstances. As he put it: they were sent into the forest with a knife to kill a lion. Everybody else ran with that metaphor. Someone asked what a lion was doing in a forest; don't they live on the savannah? I asked whether he was sure there was a lion; plenty of times I've been sent to kill a lion and ended up having to kill a goat or an elephant instead. Are we even sure it needs killing? I think that's the difference between a competitive programmer and a professional programmer. The competitive programmer will be much faster with a solution to the given problem, but the professional programmer will solve a better problem.

dsacco 9y ago
That conclusion seems just as arbitrary as the one you're contesting.
wlesieutre 9y ago
Mountain lions like forests just fine! Unfortunately for them, we chopped most of the forests down. Assuming you're in the US and somebody tells you there's a lion around, pretty good odds it's not one of the big orange ones. EDIT: Bay area published yesterday http://www.nbcbayarea.com/news/local/Several-Mountain-Lion-Sightings-Shut-Down-Trail-at-Alum-Rock-Park-432968923.html http://www.nbcbayarea.com/news/local/Several-Mountain-Lion-S...
- mcv 9y ago
  A mountain lion is a very different animal from a lion. Different subfamily and genus, even. I don't live in the US, but when someone tells me "lion", I tend to assume they mean a lion, and not a puma. Though if there's any doubt, that's absolutely something to ask questions about.

lr4444lr 9y ago

In the chapter of his interview in Coders at Work[0], Norvig found that the strongest correlate in the interview process with success at Google was paradoxically to have been given the lowest possible score by one of the interviewers. He surmised that this was because in order for such a person to have even gotten hired at the end of the process, someone trustworthy must have seen so much potential in the prospective hire that he strongly advocated for the person to be offered a position which worked in spite of that other low rating. Kind of ironic for a company whose product values are so tightly tied to quantitive data. [0]http://www.apress.com/us/book/9781430219484 http://www.apress.com/us/book/9781430219484

andrewla 9y ago
A qualitative assessment of that factor is hard because Google doesn't hire the people that it doesn't hire. Another assessment could be that a divergence of opinion among interviewers is itself a positive sign -- programmers with strong controversial opinions who are willing to hold to them even in an interview setting might be better programmers for that. A less sanguine assessment is that "success at Google" correlates with people who generate controversy around themselves, simply because that is something that creates visibility.
aoeusnth1 9y ago
This isn't surprising - in fact, it would be surprising if it wasn't the case. This fact says nothing about the ability of Google's hiring bar to distinguish between good and bad hires, except that it is not a perfect signal.
- lr4444lr 9y ago
  Totally agree. It just serves as a useful reminder though not to fall prey to the fallacy of deliberately chasing the correlational measure as a matter of policy.
ubernostrum 9y ago
More recently, people who have and are willing to analyze data across many interviewees and interviewers at multiple companies (so they can see ongoing performance of a candidate a company passed on!) have pointed out: http://blog.interviewing.io/you-cant-fix-diversity-in-tech-without-fixing-the-technical-interview/ http://blog.interviewing.io/you-cant-fix-diversity-in-tech-w... After looking at thousands of interviews on the platform, we’ve discovered something alarming: interviewee performance from interview to interview varied quite a bit, even for people with a high average performance ... roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview.

jpn 9y ago

This: https://news.ycombinator.com/item?id=13739329 https://news.ycombinator.com/item?id=13739329

bjacokes 9y ago

Here are some of the positive qualities I would expect from a competitive programmer compared to the general CS population early in their career: - knowledgeable at algorithms and data structures - good at analyzing correctness and edge cases, even on simple non-algorithmic problems (e.g. FizzBuzz) - accustomed to working hard and learning new concepts. this attribute is not specific to competitive programmer – for example, I'd expect similarly from an open source contributor – but it's higher than for a typical college student. Some negatives I'd expect, which are fixable over the course of the course of their career: - over-confidence in code, under-testing - less skilled in OOP, coding style, version control systems, as well as web development or systems code (unless they have specific previous work in these areas) - sometimes looking down on gruntwork/rote as beneath them (like the view that pure mathematicians have towards applied math or statistics) I think that list of positive attributes often outweighs the potential negatives, especially during an internship or in the first year or two of someone's career. After that, I would expect many non-competitive programmers to have picked up on some of those advantages (code correctness, learning new concepts). I've tried to steer my own interviews away from algorithms (especially DP) and focusing more on giving problems that are relatively straightforward, while still being complicated enough that someone has to write precise code and identify/fix a few edge cases.

WalterBright 9y ago

All job interviews rely on proxies for whether someone will be good on the job or not, because the only way to know is to actually hire them. All proxies are inaccurate.

softawre 9y ago
Yeah. The best proxy is just to hire somebody as a temp contractor for a month or so if both parties can afford it.
blacksmythe 9y ago
Also, everyone is convinced that their proxy is the best achievable, although of course they test it only against false positives, not false negatives.
- WalterBright 9y ago
  > everyone is convinced that their proxy is the best achievable In my experience, people merely hoped their process was good enough. And it usually was. The "we only hire the best" is marketing propaganda and they know it.

deleted 9y ago

[deleted]

bshanks 9y ago

I see a lot of comments speculating that there may be something wrong with programming competition winners, but this result might be a statistical curiousity unique to Google. It's possible that for most companies, good at programming competitions could still be positively correlated with job performance. For example, what if most people whom Google hires could have won some programming competitions if they had wanted to (a situation that most other companies are probably not in)? In this case, 'won programming competitions' could be data that is a lousy filter for Google, yet a good filter for other companies.

closed 9y ago
Good point, in essence it sounds like one theory people have is that competition winners have over-specialized, but your theory is that there could be floor effects in the range of ability at Google (i.e. the floor is high, many people at Google could win competitions).

samlittlewood 9y ago

When I am interviewing: Candidate having competed in programming competitions - big green flag. But, if success in competitions is, in their view, their biggest asset - amber flag. I would extend this to most competitive endeavours.

hajderr 9y ago

well if you're not good at programming contests would that entail a contemplating/slow programmer? I'd rather pick an algorithmist and teach him/her to reflect than a so called "reflectionist" / "slow coder" and teach him/her how to solve algorithmic problems. I'd be interested in knowing the guy's (catonmat) own reflections and experiences too.

adavidoaiei 9y ago

They are two types of business, first type business to business, software made to run internal in a company, the business run on this software, generally in this kind of software you use enterprise frameworks, the second type are business to consumer where anyone can made an account and use application, some software as a service are like that, they are too other types of web applications business to consumer. When you allow that anyone to make an account and use your application sometimes you couldn't rely on enterprise technology and it's better to write everything from scratch or to customize open source solutions. What they are testing in algorithms contests: 1.) the algorithm is correct with various test sets 2.) the performance of the algorithm, running time in milliseconds, there you should to know how to measure time complexity of algorithm with O(N) 3.) memory consumed, there you should know how much memory you are allocating for variables, for example a variable of type byte consume 8 bits I think that programmers which has prizes in algorithms contests are suitable for business to consumer applications because there they will find a challenge, this kind of programmers will be bored to dig in enterprise frameworks.

twii 9y ago

Depends also on what job I guess?

x1798DE 9y ago

In general, I think there are a lot of things like whiteboard interviews and coding competitions where employers would prefer "good at X means good on the job", but are reasonably happy to settle for "bad at X means bad on the job."

qdev 9y ago

My viewpoint is taken from the context as someone who is a reasonably seasoned developer returning to the job market. I never did programming competitions in university, though I surely was someone who fit the profile (computer science guy with a discrete math bent). As part of my recent prep for a job search, I joined one of the programming competition websites and did a couple of contests. I posit that one of the dangers of spending a lot of time doing programming competitions and becoming very proficient at them is that, perhaps, you can come to believe that "true" programming, some sort of Platonic ideal of programming, is about coming up with the clever insight that solves an algorithmic puzzle. But, in fact, a fair bit of _commercial_ programming is down and dirty, with databases, and user interfaces, and a lot of the time is really just shuffling data from one place to another, maybe filtering it or combining it with another set of data. And that's just at the beginning of your career. Later on in your career, success means being able to work at larger scales in a team. That means organizing the code in a way that supports the efficient development of the codebase by individuals like yourself, by your team, by the development group as a whole... And at the architect level, you perhaps are looking at designing the system to support the efficient operation of the entire organization. So I can easily believe that success at a programming competition does not correlate with long-term success as a software engineer in commercial software development. The two are really very different. (Btw, I actually found the competitions that I did to be fun, but mentally exhausting. I'd say go ahead and do them, especially if you have an inclination for those types of problems. Just be prepared to use a different mindset for commercial software development.)

aqp 9y ago

http://ruberik.blogspot.com/2017/07/no-programming-competitions-dont.html http://ruberik.blogspot.com/2017/07/no-programming-competiti... It looks like it wasn't "being good at programming competitions" that was negatively correlated with job performance. It was "participated in programming competitions". And there are some more "how to interpret machine learning models" caveats in that blog post. It seems to me the biggest factor in explaining this is that the people who are just below the hiring line but participate in competitions get a bump over the line. Since there are more people just below the line than above it, the "participates" group is bottom-heavy, producing the correlation. I do a lot of interviews, and it seems to me that lots of people with experience perform below how they "should", because they're not practiced at solving problems from scratch, they work all day on modifying larger systems. Programming competitions would fix that for them, as would most open-source hobby projects.