24 ms·

People bomb, including good people. That is sadly part of the system. Once I had this candidate. Damn, I know my questions are deceptively simple on purpose, b

by lrem 6y ago

People bomb, including good people. That is sadly part of the system. Once I had this candidate. Damn, I know my questions are deceptively simple on purpose, but he literally couldn't do anything. Not even a trivial brute force. Not even any related simple knowledge questions. Couldn't tell an average from a median. The only good thing I could write in the feedback was "seems to know some basic syntax". It was quite a learning moment to read that everyone else was praising how this was the most brilliant candidate they've seen in years. Well, mine was the interview just before lunch, in a schedule that was atypically later than usual. I guess starting an interview after most people start their lunch break was asking for it. He also got hired - apparently the committee agreed that "can't think when hungry" is not really that important a flaw.

bumby 6y ago
Interestingly, there's a famous study showing those doing the judging "can't think when hungry" either. (Or, at least can't think unbiasedly).[1] [1] https://www.pnas.org/content/108/17/6889 https://www.pnas.org/content/108/17/6889
- Tarq0n 6y ago
  http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html?m=1 http://daniellakens.blogspot.com/2017/07/impossibly-hungry-j...
- mattkrause 6y ago
  That study turned out to be quite flawed, because the cases were ordered [1]. In particular, cases were grouped by prison and within each prison, prisoners without an attorney went last. As the well-known saying goes, "the man who represents himself has a fool for a client": the pro se prisoners fare far worse than those with a lawyer. Judges tried to complete an entire prison before taking a meal break, and therefore ended one session with the statistically weakest cases and began the next with stronger set, thereby guaranteeing the result. If you look at the original data, some other oddities pop out. A physiological process, like becoming hungry, should depend more on the wallclock time than the number of cases heard. However "However, note that in an analysis that included both the cumulative minutes variable and the ordinal position counter, only the latter was significant." [1] https://www.pnas.org/content/108/42/E833?ijkey=21207497c684f21468872db76e164f93a585bc00&keytype2=tf_ipsecsha https://www.pnas.org/content/108/42/E833?ijkey=21207497c684f...
  - bumby 6y ago
    Ah, thank you! The more I look into a lot of these (in)famous behavioral psychology studies, many don’t hold up or aren’t replicable. As a layman who is simply interested in the topic, I appreciate the correction and insight!
  - pmiller2 6y ago
    Well, then. There's the solution. From now on, I'm bringing my attorney to all my on-site interviews. :P
opus132 6y ago
> Couldn't tell an average from a median Without meaning to attack you personally (especially in the context of the rest of your comment), a comment like this annoys me a bit. I presume by "average" you mean arithmetic mean, but the median is also an average, and depending on the context the median might be a far more useful statistic than the mean. Confusing the mean and the median is one thing, and perhaps you actually used this terminology in the interview; "confusing" "the average" and the median isn't really worthy of comment, and sounds more like a breakdown in communication between interviewer and candidate rather than a lack of technical knowledge. It just seems vaguely hypocritical to me to be expecting a certain level of ability from the candidate and then using imprecise/informal terminology.
- dimtion 6y ago
  I know nothing about GP, but note that it is possible that English is not his first language, and the subtleties of the story and vocabulary might be lost in translation. In my native language (French), as you describe it, there is no word for "average" (the technical and the common terms "moyenne" are exactly the same). Until your comment, in English, I assumed you could use "average" and "mean" interchangeably.
  - jlokier 6y ago
    My native language is English and I'm fluent and well educated. Although I do know that "average" covers mean, median and mode, when I was young I was not taught that way. First I was just taught "average" with a formula, and many years later "median" and "mode". I don't think I noticed the term "mean" until I learned about median and mode. You can see this reflected in scientific calculators used at school, where the button for calculating mean is labelled "Avg". And in Microsoft Excel or LibreOffice Calc, the function to calculate mean is called "AVERAGE". So nobody should be hard on themselves for not knowing the difference. You probably weren't taught the difference, and common tools work against it. It looks to me that the common word for statistical mean is "average" in English, but it's not the correct technical language.
    - 6y ago
thephyber 6y ago
> People bomb, including good people. That is sadly part of the system. It seems like there's probably some value in a "MoneyBall" company that identifies a way to interview without most of the anxiety. Perhaps hiring a few people with remarkably high EQ who customize the interview setting who review the inteviewees' work and watch them program over a few days in their own computer and IDE. I know I've interviewed people that have frozen up, but for whatever reason I didn't pause the interview and address the anxiety, but rather I tried to adjust the questions (which generally didn't work).
- groby_b 6y ago
  These people are usually called "references" :) As for interviewing for a few days... Yeah. No. Not in a country with an abysmally bad vacation policy like the US.
- andrewflnr 6y ago
  I do think that the people giving the questions and interacting with the applicant should be different from the ones judging their results. This also gives you the opportunity to blind the judging for race, gender, etc. The proctor should be explicitly on the applicant's "side" IMO.
  - hinkley 6y ago
    I used to 'sit in' on interviews, and basically say very little except to answer questions about working here or to serve as the 'linter' to get them unstuck. You hear something very different than the person who asked the question. And in one case I discovered that my coworker had been asking a question that he thought he knew the answer to and did not; a deep clone implementation (where he never asked about cyclic graphs).
    - andrewflnr 6y ago
      I assumed it rather than say it out loud, but my idea is to have standard, vetted questions, to reduce the risk of a question being mal-formed like that (unless you want to see how they make tradeoffs in response to an overambitious spec). Your situation sounds rather different. Do you think my scenario would work better?
koheripbal 6y ago
One trend I've noticed that is markedly different from when I started programming in 1998, is how dependent we've become to program via Google/StackExchange searches. I'm not sure I know how to write anything from scratch anymore, because I just search/read/alter/test. The breadth of what I work on is 100x wider than it used to be, and so I've become absolutely dependent on quickly reading docs, copying code found online, and then deep diving into testing and rolling it out the door. I deliver a TON more, but I'm convinced I'd bomb literally every interview I go to today. Today, for example, I updated our firewall rules, updated some Java business logic, tweaked some javascipt and PHP on the site, reversed an odd macro email-phishing virus we got, configured a new RDS server VM for clients, and updated some network GPOs. 20 years ago, I would have never worked on ALL of those things in the same week - let alone the same day. Maybe we should interview with small take-home tasks to be submitted with some write-ups to test how people research, reason, and write-up problems, rather than writing code on the spot on a whiteboard. ...but maybe that's just me.
- blisterpeanuts 6y ago
  It's not just you :) A much more realistic way to test people is to tell them to bring their laptop, point to a problem on the whiteboard and say "here, solve it, and use whatever resources you need." I'm somewhat like you; I do a wide variety of tasks every day and simply can't keep all the different rules and syntaxes in my head. I'm terrible at tests anyway and would probably bomb a FAANG interview badly because of my lousy memory. As an interviewer, I would rather someone be generally smart, have a good work ethic and lots of experience solving problems. In the long run that seems more useful than being able to write computer programs on a piece of paper in a conference room.
  - AnHonestComment 6y ago
    I see a lot of people impose constraints on themselves that aren’t there. I’ve said, at multiple interviews, “there’s a library function something like this — normally I’d google it” and made up some syntax. The point is to write something while being pleasant to work with — not memorize a bunch of syntax. My experience is that candidates would be better served by practicing being personable over coffee than leetcode — but that could be sampling bias or whatever.
foolfoolz 6y ago
i worked at a company where i had done over 150 interviews and the company did at least 10 interviews a day at the office i was at. we did metrics on what times of day or days of the week or rooms. before lunch or 4pm or later interviews we’re worse across all candidates certain rooms we knew were smelly, loud, or poorly converted from like a storage closet scored worse across all candidates we took some rooms out of the rotation and made sure everyone was well fed/had drinks. the effect is real
lfowles 6y ago
Oh no, confusing median vs mean is a very specific interview question I've bombed in the past so hard that the interview almost immediately stopped!
sjg007 6y ago
So is the moral of the story that this hire worked out or not? Was this a good coder?
- lrem 6y ago
  That's another downside of the system: learning that is too much effort, unless they land in your team. But knowing how we hire only obviously overqualified people, I trust he did well.
derefr 6y ago
> He also got hired - apparently the committee agreed that "can't think when hungry" is not really that important a flaw. I hope you suggested that this employee get tested for hypoglycemia. It'd probably greatly improve their quality-of-life to go from "my brain isn't working and I don't know why" to "my brain isn't working and I know exactly why; I need to drink some fruit juice."
- bananaface 6y ago
  Hah. "Did a tech interview, found out I had diabetes."
jussij 6y ago
> Couldn't tell an average from a median. Shouldn't that be tell a mean from a median?
- lrem 6y ago
  It should. There is a sibling thread about this ;)
dang 6y ago
(This subthread was originally a child of https://news.ycombinator.com/item?id=23848916 https://news.ycombinator.com/item?id=23848916)

by lrem 6y ago

bumby 6y ago

Interestingly, there's a famous study showing those doing the judging "can't think when hungry" either. (Or, at least can't think unbiasedly).[1] [1] https://www.pnas.org/content/108/17/6889 https://www.pnas.org/content/108/17/6889

Tarq0n 6y ago
http://daniellakens.blogspot.com/2017/07/impossibly-hungry-judges.html?m=1 http://daniellakens.blogspot.com/2017/07/impossibly-hungry-j...
mattkrause 6y ago
That study turned out to be quite flawed, because the cases were ordered [1]. In particular, cases were grouped by prison and within each prison, prisoners without an attorney went last. As the well-known saying goes, "the man who represents himself has a fool for a client": the pro se prisoners fare far worse than those with a lawyer. Judges tried to complete an entire prison before taking a meal break, and therefore ended one session with the statistically weakest cases and began the next with stronger set, thereby guaranteeing the result. If you look at the original data, some other oddities pop out. A physiological process, like becoming hungry, should depend more on the wallclock time than the number of cases heard. However "However, note that in an analysis that included both the cumulative minutes variable and the ordinal position counter, only the latter was significant." [1] https://www.pnas.org/content/108/42/E833?ijkey=21207497c684f21468872db76e164f93a585bc00&keytype2=tf_ipsecsha https://www.pnas.org/content/108/42/E833?ijkey=21207497c684f...
- bumby 6y ago
  Ah, thank you! The more I look into a lot of these (in)famous behavioral psychology studies, many don’t hold up or aren’t replicable. As a layman who is simply interested in the topic, I appreciate the correction and insight!
- pmiller2 6y ago
  Well, then. There's the solution. From now on, I'm bringing my attorney to all my on-site interviews. :P

opus132 6y ago

> Couldn't tell an average from a median Without meaning to attack you personally (especially in the context of the rest of your comment), a comment like this annoys me a bit. I presume by "average" you mean arithmetic mean, but the median is also an average, and depending on the context the median might be a far more useful statistic than the mean. Confusing the mean and the median is one thing, and perhaps you actually used this terminology in the interview; "confusing" "the average" and the median isn't really worthy of comment, and sounds more like a breakdown in communication between interviewer and candidate rather than a lack of technical knowledge. It just seems vaguely hypocritical to me to be expecting a certain level of ability from the candidate and then using imprecise/informal terminology.

dimtion 6y ago
I know nothing about GP, but note that it is possible that English is not his first language, and the subtleties of the story and vocabulary might be lost in translation. In my native language (French), as you describe it, there is no word for "average" (the technical and the common terms "moyenne" are exactly the same). Until your comment, in English, I assumed you could use "average" and "mean" interchangeably.
- jlokier 6y ago
  My native language is English and I'm fluent and well educated. Although I do know that "average" covers mean, median and mode, when I was young I was not taught that way. First I was just taught "average" with a formula, and many years later "median" and "mode". I don't think I noticed the term "mean" until I learned about median and mode. You can see this reflected in scientific calculators used at school, where the button for calculating mean is labelled "Avg". And in Microsoft Excel or LibreOffice Calc, the function to calculate mean is called "AVERAGE". So nobody should be hard on themselves for not knowing the difference. You probably weren't taught the difference, and common tools work against it. It looks to me that the common word for statistical mean is "average" in English, but it's not the correct technical language.
  - 6y ago

thephyber 6y ago

> People bomb, including good people. That is sadly part of the system. It seems like there's probably some value in a "MoneyBall" company that identifies a way to interview without most of the anxiety. Perhaps hiring a few people with remarkably high EQ who customize the interview setting who review the inteviewees' work and watch them program over a few days in their own computer and IDE. I know I've interviewed people that have frozen up, but for whatever reason I didn't pause the interview and address the anxiety, but rather I tried to adjust the questions (which generally didn't work).

groby_b 6y ago
These people are usually called "references" :) As for interviewing for a few days... Yeah. No. Not in a country with an abysmally bad vacation policy like the US.
andrewflnr 6y ago
I do think that the people giving the questions and interacting with the applicant should be different from the ones judging their results. This also gives you the opportunity to blind the judging for race, gender, etc. The proctor should be explicitly on the applicant's "side" IMO.
- hinkley 6y ago
  I used to 'sit in' on interviews, and basically say very little except to answer questions about working here or to serve as the 'linter' to get them unstuck. You hear something very different than the person who asked the question. And in one case I discovered that my coworker had been asking a question that he thought he knew the answer to and did not; a deep clone implementation (where he never asked about cyclic graphs).
  - andrewflnr 6y ago
    I assumed it rather than say it out loud, but my idea is to have standard, vetted questions, to reduce the risk of a question being mal-formed like that (unless you want to see how they make tradeoffs in response to an overambitious spec). Your situation sounds rather different. Do you think my scenario would work better?

koheripbal 6y ago

One trend I've noticed that is markedly different from when I started programming in 1998, is how dependent we've become to program via Google/StackExchange searches. I'm not sure I know how to write anything from scratch anymore, because I just search/read/alter/test. The breadth of what I work on is 100x wider than it used to be, and so I've become absolutely dependent on quickly reading docs, copying code found online, and then deep diving into testing and rolling it out the door. I deliver a TON more, but I'm convinced I'd bomb literally every interview I go to today. Today, for example, I updated our firewall rules, updated some Java business logic, tweaked some javascipt and PHP on the site, reversed an odd macro email-phishing virus we got, configured a new RDS server VM for clients, and updated some network GPOs. 20 years ago, I would have never worked on ALL of those things in the same week - let alone the same day. Maybe we should interview with small take-home tasks to be submitted with some write-ups to test how people research, reason, and write-up problems, rather than writing code on the spot on a whiteboard. ...but maybe that's just me.

blisterpeanuts 6y ago
It's not just you :) A much more realistic way to test people is to tell them to bring their laptop, point to a problem on the whiteboard and say "here, solve it, and use whatever resources you need." I'm somewhat like you; I do a wide variety of tasks every day and simply can't keep all the different rules and syntaxes in my head. I'm terrible at tests anyway and would probably bomb a FAANG interview badly because of my lousy memory. As an interviewer, I would rather someone be generally smart, have a good work ethic and lots of experience solving problems. In the long run that seems more useful than being able to write computer programs on a piece of paper in a conference room.
- AnHonestComment 6y ago
  I see a lot of people impose constraints on themselves that aren’t there. I’ve said, at multiple interviews, “there’s a library function something like this — normally I’d google it” and made up some syntax. The point is to write something while being pleasant to work with — not memorize a bunch of syntax. My experience is that candidates would be better served by practicing being personable over coffee than leetcode — but that could be sampling bias or whatever.

foolfoolz 6y ago

i worked at a company where i had done over 150 interviews and the company did at least 10 interviews a day at the office i was at. we did metrics on what times of day or days of the week or rooms. before lunch or 4pm or later interviews we’re worse across all candidates certain rooms we knew were smelly, loud, or poorly converted from like a storage closet scored worse across all candidates we took some rooms out of the rotation and made sure everyone was well fed/had drinks. the effect is real

lfowles 6y ago

Oh no, confusing median vs mean is a very specific interview question I've bombed in the past so hard that the interview almost immediately stopped!

sjg007 6y ago

So is the moral of the story that this hire worked out or not? Was this a good coder?

lrem 6y ago
That's another downside of the system: learning that is too much effort, unless they land in your team. But knowing how we hire only obviously overqualified people, I trust he did well.

derefr 6y ago

> He also got hired - apparently the committee agreed that "can't think when hungry" is not really that important a flaw. I hope you suggested that this employee get tested for hypoglycemia. It'd probably greatly improve their quality-of-life to go from "my brain isn't working and I don't know why" to "my brain isn't working and I know exactly why; I need to drink some fruit juice."

bananaface 6y ago
Hah. "Did a tech interview, found out I had diabetes."

jussij 6y ago

> Couldn't tell an average from a median. Shouldn't that be tell a mean from a median?

lrem 6y ago
It should. There is a sibling thread about this ;)

dang 6y ago

(This subthread was originally a child of https://news.ycombinator.com/item?id=23848916 https://news.ycombinator.com/item?id=23848916)