3 ms·

John D Cook has a lot to say about this: [1] "One of the marks of a professional programmer is knowing how to organize software so that the complexity remains

John D Cook has a lot to say about this: [1] "One of the marks of a professional programmer is knowing how to organize software so that the complexity remains manageable as the size increases. Even among professionals there are large differences in ability. The programmers who can effectively manage 100,000-line projects are in a different league than those who can manage 10,000-line projects. ... Writing large buggy programs is hard. ... Writing large correct programs is much harder." Jeff Atwood's metrics will help you filter out engineers whose complexity ceiling is <1k lines -- StackOverflow answers, whoopee -- but that's not a terribly hard thing to interview for. Much harder to interview for the very best, the mythical 10x productivity programmers[2], those who can handle 100k LOC, 1M, or more. Perhaps this is the difference between an experienced non-expert and a real expert[3]. In my experience not a lot of employers care about this, perhaps because their challenges aren't those of complexity-in-scale, or perhaps because complexity hasn't bit them hard enough yet, or perhaps because they are "unconsciously incompetent"[4]. About the only hiring signal I've identified for this is interest in functional programming -- languages like Clojure and Scala exist precisely to raise the ceiling of complexity a human can handle[6] -- and as such I'm trying to learn this stuff and trying to find people via the community who care to hire engineers with these skills. Unfortunately my own bias may be blinding me, you never know which side of Dunning-Kruger[5] you're on until it's too late. If you care about these things: I'd love to know who you are and what you're working on, email me. [1] http://www.johndcook.com/blog/2008/09/19/writes-large-correct-programs/ http://www.johndcook.com/blog/2008/09/19/writes-large-correc... [2] I am not one of these, but I strive to be one someday. [3] http://www.dustingetz.com/how-to-become-an-expert-swegr http://www.dustingetz.com/how-to-become-an-expert-swegr [4] http://en.wikipedia.org/wiki/Four_stages_of_competence http://en.wikipedia.org/wiki/Four_stages_of_competence [5] http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect [6] Clojure creator Rich Hickey talking about complexity: http://www.infoq.com/presentations/Simple-Made-Easy http://www.infoq.com/presentations/Simple-Made-Easy edit: man, this got 2 downvotes in 2 minutes, cmon guys i put a lot of thought into this!

jacquesm 15y ago
> man, this got 2 downvotes in 2 minutes, cmon guys i put a lot of thought into this! That really sucks because there is some very good stuff in here. One thing that I find strange is to assume that those that are really good at this would have the time to maintain a stackoverflow profile. Likely they're too busy raking in the $. On the other hand start-ups likely can't afford those guys anyway. Until they drown in a 150K loc tangle that makes spaghetti look structured.
jamesli 15y ago
Have to show some support for this post. Engineering is a combination of technology and art. Therefore, it is impossible to have a simple process for interviewing engineers that magically works. Otherwise, such posts would not have been brought up so many times in HN. Also, some people are good at people evaluation, some are not. An excellent engineer could be a lousy interviewer. On the other hand, a mediocre engineer is lack of the ability to evaluate a top engineer within an hour. Therefore, the employer must first know who are his/her best interviewers. If s/he can't identify her/his own people, I highly doubt s/he is able to identify good candidates. The hiring signals I pay attention to are one's curiosity and one's depth in understanding in at least one programming language or one computer science topic, no matter if that topic or programming language has any relevance to the position to be filled. Here is the rationale. To be a very good engineer, one needs both internal motivation and intelligence. Both of the two signals speak for internal motivation. A deep understanding on any topic shows the candidate is sufficiently intelligent. Such a candidate, even if s/he knows little about the languages and/or the frameworks the position requires, s/he would learn it very fast and would be very good at it. To be realistic, it is really not hard to learn a programming language and a framework.
- kamaal 15y ago
  Good thoughts! I think software has a lot of reusable principles/concepts. And these days syntax is really skin. Programming languages get used a lot because their ecosystem. You are definitely very correct to demand perfection in at least one walk of our profession. Because those concepts get reused nearly everywhere. Another thing that I don't understand is rejecting people merely because they don't know answers to some questions from the algorithm and data structures text book. Software engineering today is so much about so many things. Above all I would say productivity and passion is the only factors I would use to judge people today. Because those factors decide nearly every other factor.
int3 15y ago
I'm very interested in this topic too. Aside from functional programming, I think examining how existing systems were designed is another good way to improve. I'm taking a class[1] where we read a bunch of classic papers on systems design, and I think it's been helpful. I just did a bit of googling, and it appears that MIT has a similar course[2] with a free online textbook -- I'm going to check it out later. (I haven't architected a really large codebase before, so take all this with a grain of salt...) [1]: http://plasma.cs.umass.edu/emery/grad-systems http://plasma.cs.umass.edu/emery/grad-systems [2]: http://ocw.mit.edu/resources/res-6-004-principles-of-computer-system-design-an-introduction-spring-2009/ http://ocw.mit.edu/resources/res-6-004-principles-of-compute...
- dustingetz 15y ago
  > it appears that MIT has a similar course[2] with a free online textbook lol. you're looking for SICP[1] and if you're interested in this stuff, and you haven't read it yet, you should a) buy it and burn all your other books so as not to distract you, b) google "sicp site:news.ycombinator.com" to convince yourself i'm not full of hot air, and c) email me so i can get you in on my online participate-as-you-have-time discussion group which is not yet organized but we will start in a week or two ;) [1] http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html#%_toc_start http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html#...
kamaal 15y ago
>>Much harder to interview for the very best, the mythical 10x productivity programmers[2], those who can handle 100k LOC, 1M, or more. Perhaps this is the difference between an experienced non-expert and a real expert[3]. Only that the 10x productive programmer deosn't quite really have time in his day job to memorize and master arcane facts and puzzles. And since it turns out the big Web brands are all about this facts and puzzles in their initial rounds. This means they are missing out on nearly all 10x productive programmers.
- dustingetz 15y ago
  well, you have to figure that the true 10x programmers are probably smart enough to figure out how to attract and close other 10x programmers. so either a) the puzzle-people aren't 10x programmers so who cares, or b) they are, and we don't understand their process because their process isn't designed to find us. i mean, i tend to go with (a), just sayin. Jane Street Capital (world-class FP shop) is famous for two things: a) having a ridiculously hard interview, and b) hiring the best of the best. their interviews are blogged all over google. they ask everything from "three bags - apples, oranges, mixed; all mislabeled; how many guesses do you need to label them properly" to "implement a perl-style regular expression parser. on the whiteboard. in 45 minutes."[1] i'm not sayin' i dig the puzzles, but I've read a few of their whitepapers, they are definitely better engineers than I am. and if i ever decide to interview again (once upon a time i insta flunked a phone interview), you can bet i will be a godly puzzle master. no matter the outcome, i bet i learn something about why they ask the questions they do. [1] https://www.quantnet.com/forum/threads/detailed-in-person-jane-street-interview-software-engineer.4842/ https://www.quantnet.com/forum/threads/detailed-in-person-ja...
  - kamaal 15y ago
    Well I am sure they are. But so far the best people in the industry I have seen so far have no time for puzzles and facts. They have better things to do and build. And they are busy enough and generally have enough of the actual work, to keep themselves busy enough to be adding meaningful value. I now follow the same principle. I've cut down on my social network interactions. Once a day email communication. No more fact hunting. No more evaluation of every single open source software that gets released. No more worrying about language wars. No more text book reading etc. I plan my actual work in a GTD model. I execute in pomodoro style. All aligned towards getting work done and keeping me busy and productive all day. I have received amazing results. I've developed new hobbies like music. I spend meaningful time on things like meditation and exercise in whatever spare time remains. I am for sure not going back to any puzzle and algorithm shopping any more. My experience reveals provided I can remain a order of magnitude more productive then others everything else will get automatically get taken care of.
ajuc 15y ago
I am interested in functional programming, because it's interesting. Nothing to do with managing complexity. And I have a few friends interested in exotic declarative languages becasue of "coolness factor", not because they need such languages in their job. I also have a few experiences with writing hobby projects in functional languages, which never went anywhere, and what I have finished, I did using the simplest tools I had on my hand (turbo pascal, python, java, javascript). So I think interest in functional languages isn't really good predictor of anything.
- Roboprog 15y ago
  Yeah, but give me a language (e.g. - Perl, etc) that lets me make closures, instead of anonymous inner classes that implement an interface with exactly one method, any day. Fascist OOP can lead to some real code bloat, and I suspect that is starting to dawn on the leadership of this industry. A bit tangential, but it's something I am reminded of day in and out trudging through Java code at work, and one of the "quick payback" reasons I'd like any new guy to understand FP. (oh, and Python and JavaScript are, at least in a "compatible style", functional programming languages)

jacquesm 15y ago

> man, this got 2 downvotes in 2 minutes, cmon guys i put a lot of thought into this! That really sucks because there is some very good stuff in here. One thing that I find strange is to assume that those that are really good at this would have the time to maintain a stackoverflow profile. Likely they're too busy raking in the $. On the other hand start-ups likely can't afford those guys anyway. Until they drown in a 150K loc tangle that makes spaghetti look structured.

jamesli 15y ago

Have to show some support for this post. Engineering is a combination of technology and art. Therefore, it is impossible to have a simple process for interviewing engineers that magically works. Otherwise, such posts would not have been brought up so many times in HN. Also, some people are good at people evaluation, some are not. An excellent engineer could be a lousy interviewer. On the other hand, a mediocre engineer is lack of the ability to evaluate a top engineer within an hour. Therefore, the employer must first know who are his/her best interviewers. If s/he can't identify her/his own people, I highly doubt s/he is able to identify good candidates. The hiring signals I pay attention to are one's curiosity and one's depth in understanding in at least one programming language or one computer science topic, no matter if that topic or programming language has any relevance to the position to be filled. Here is the rationale. To be a very good engineer, one needs both internal motivation and intelligence. Both of the two signals speak for internal motivation. A deep understanding on any topic shows the candidate is sufficiently intelligent. Such a candidate, even if s/he knows little about the languages and/or the frameworks the position requires, s/he would learn it very fast and would be very good at it. To be realistic, it is really not hard to learn a programming language and a framework.

kamaal 15y ago
Good thoughts! I think software has a lot of reusable principles/concepts. And these days syntax is really skin. Programming languages get used a lot because their ecosystem. You are definitely very correct to demand perfection in at least one walk of our profession. Because those concepts get reused nearly everywhere. Another thing that I don't understand is rejecting people merely because they don't know answers to some questions from the algorithm and data structures text book. Software engineering today is so much about so many things. Above all I would say productivity and passion is the only factors I would use to judge people today. Because those factors decide nearly every other factor.

int3 15y ago

I'm very interested in this topic too. Aside from functional programming, I think examining how existing systems were designed is another good way to improve. I'm taking a class[1] where we read a bunch of classic papers on systems design, and I think it's been helpful. I just did a bit of googling, and it appears that MIT has a similar course[2] with a free online textbook -- I'm going to check it out later. (I haven't architected a really large codebase before, so take all this with a grain of salt...) [1]: http://plasma.cs.umass.edu/emery/grad-systems http://plasma.cs.umass.edu/emery/grad-systems [2]: http://ocw.mit.edu/resources/res-6-004-principles-of-computer-system-design-an-introduction-spring-2009/ http://ocw.mit.edu/resources/res-6-004-principles-of-compute...

dustingetz 15y ago
> it appears that MIT has a similar course[2] with a free online textbook lol. you're looking for SICP[1] and if you're interested in this stuff, and you haven't read it yet, you should a) buy it and burn all your other books so as not to distract you, b) google "sicp site:news.ycombinator.com" to convince yourself i'm not full of hot air, and c) email me so i can get you in on my online participate-as-you-have-time discussion group which is not yet organized but we will start in a week or two ;) [1] http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html#%_toc_start http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-4.html#...

kamaal 15y ago

>>Much harder to interview for the very best, the mythical 10x productivity programmers[2], those who can handle 100k LOC, 1M, or more. Perhaps this is the difference between an experienced non-expert and a real expert[3]. Only that the 10x productive programmer deosn't quite really have time in his day job to memorize and master arcane facts and puzzles. And since it turns out the big Web brands are all about this facts and puzzles in their initial rounds. This means they are missing out on nearly all 10x productive programmers.

dustingetz 15y ago
well, you have to figure that the true 10x programmers are probably smart enough to figure out how to attract and close other 10x programmers. so either a) the puzzle-people aren't 10x programmers so who cares, or b) they are, and we don't understand their process because their process isn't designed to find us. i mean, i tend to go with (a), just sayin. Jane Street Capital (world-class FP shop) is famous for two things: a) having a ridiculously hard interview, and b) hiring the best of the best. their interviews are blogged all over google. they ask everything from "three bags - apples, oranges, mixed; all mislabeled; how many guesses do you need to label them properly" to "implement a perl-style regular expression parser. on the whiteboard. in 45 minutes."[1] i'm not sayin' i dig the puzzles, but I've read a few of their whitepapers, they are definitely better engineers than I am. and if i ever decide to interview again (once upon a time i insta flunked a phone interview), you can bet i will be a godly puzzle master. no matter the outcome, i bet i learn something about why they ask the questions they do. [1] https://www.quantnet.com/forum/threads/detailed-in-person-jane-street-interview-software-engineer.4842/ https://www.quantnet.com/forum/threads/detailed-in-person-ja...
- kamaal 15y ago
  Well I am sure they are. But so far the best people in the industry I have seen so far have no time for puzzles and facts. They have better things to do and build. And they are busy enough and generally have enough of the actual work, to keep themselves busy enough to be adding meaningful value. I now follow the same principle. I've cut down on my social network interactions. Once a day email communication. No more fact hunting. No more evaluation of every single open source software that gets released. No more worrying about language wars. No more text book reading etc. I plan my actual work in a GTD model. I execute in pomodoro style. All aligned towards getting work done and keeping me busy and productive all day. I have received amazing results. I've developed new hobbies like music. I spend meaningful time on things like meditation and exercise in whatever spare time remains. I am for sure not going back to any puzzle and algorithm shopping any more. My experience reveals provided I can remain a order of magnitude more productive then others everything else will get automatically get taken care of.

ajuc 15y ago

I am interested in functional programming, because it's interesting. Nothing to do with managing complexity. And I have a few friends interested in exotic declarative languages becasue of "coolness factor", not because they need such languages in their job. I also have a few experiences with writing hobby projects in functional languages, which never went anywhere, and what I have finished, I did using the simplest tools I had on my hand (turbo pascal, python, java, javascript). So I think interest in functional languages isn't really good predictor of anything.

Roboprog 15y ago
Yeah, but give me a language (e.g. - Perl, etc) that lets me make closures, instead of anonymous inner classes that implement an interface with exactly one method, any day. Fascist OOP can lead to some real code bloat, and I suspect that is starting to dawn on the leadership of this industry. A bit tangential, but it's something I am reminded of day in and out trudging through Java code at work, and one of the "quick payback" reasons I'd like any new guy to understand FP. (oh, and Python and JavaScript are, at least in a "compatible style", functional programming languages)