6 ms·

I can't tell you how many people I've interviewed that were "experienced" developers that were just terrible terrible engineers. My favorite interviews were wh

I can't tell you how many people I've interviewed that were "experienced" developers that were just terrible terrible engineers. My favorite interviews were when we were trying to hire a senior level engineer, and we were only interviewing people with 10+ years of development experience. We'd ask people to build a priority queue. Items with a higher priority need to be handled before those with a lower priority, but otherwise items should be handled in order. Can you guess what the most common methodology people chose was? Did you guess an XML document? These are people with degrees in comp sci, some claiming 15 years of experience as a developer, trying to get a role at a top 10 bank in New York City. Of course technical interviews shouldn't be the only test used for hiring, but they absolutely should be included. Maybe it's demeaning that you have to jump through these hoops. Or maybe you're bad at them. Or they're "beneath you" in some sort of way. But don't blame the interviewer, blame the hundreds of other people trying to get a job that they aren't qualified for.

learc83 6y ago
Did you ever actual hire any of those people? Did you ever verify that they actually had the experience they claimed? Did you follow up with their references? Were they able to walk you through projects they worked, or discuss solutions implemented? Every job gets applicants that aren't qualified. Only software engineering out of every single non-performance based (musicians, actors etc...) industry out there has white board interviews.
- shaftway 6y ago
  Actually, yeah, we did. The hiring manager got tired of waiting for us to find a good candidate, so he just picked one that he had screened and seemed best on paper. It was an unmitigated disaster. Constantly had to double check his work and redirect him from doing things bad ways (like using mutable objects with changing hash codes as dictionary keys kind of bad). Eventually I quit the job (for other reasons). On the last day he cornered me in the elevator and offered me $100/hr to do his work for him under the table so that he could claim it was his. > Did you ever verify that they actually had the experience they claimed? Employer would only verify employment dates and eligibility for re-hire. > Did you follow up with their references? Yeah, but when you ask someone for references they're not likely to give you people who wouldn't speak well of them. > Were they able to walk you through projects they worked, or discuss solutions implemented? Yeah. I can walk you through projects that I was involved in as a junior as if I had been the lead because I paid attention during meetings.
  - learc83 6y ago
    So you had one bad experience with one person who didn't do a whiteboard? How many bad experiences have you had with someone who did? I've had enough that I've lost count. >like using mutable objects with changing hash codes as dictionary keys kind of bad I don't think a leet code question would necessarily have screened stuff like that out at all. >Yeah, but when you ask someone for references they're not likely to give you people who wouldn't speak well of them. Did you actually talk with these references about what it was like to work with him? >On the last day he cornered me in the elevator and offered me $100/hr to do his work for him under the table so that he could claim it was his. That's $200k a year. I find it pretty hard to believe that he was serious.
    - shaftway 6y ago
      > So you had one bad experience with one person who didn't do a whiteboard? It was a bad experience with someone who utterly failed a whiteboard. > How many bad experiences have you had with someone who did? I've had enough that I've lost count. > I don't think a leet code question would necessarily have screened stuff like that out at all. I'm sure I have too, but I never advocated for only whiteboard interviewing. There should be a spread covering all sorts of aspects of their abilities. > Did you actually talk with these references about what it was like to work with him? Of course. I have a wide variety of coworkers with a wide variety of opinions of my skill-set. I'd select the ones that I think would resonate best with you. Barring that I have half a dozen friends that would tell you whatever I want them to. I fail to see how references are any sort of magic bullet. > That's $200k a year. I find it pretty hard to believe that he was serious. I was another senior engineer on the project, so I definitely could have covered the work. I had deeper experience, and given his quality I could have churned things out more quickly. I suspect he wasn't offering me $100 / hour at 40 hours per week. The job had a lot of paperwork and bureaucracy involved. I suspect if I was just doing the coding parts I could have covered it at about 10 hours per week, which is a lot more feasible. Obviously I turned this down for ethical reasons.
      - learc83 6y ago
        >Of course. I have a wide variety of coworkers with a wide variety of opinions of my skill-set. I'd select the ones that I think would resonate best with you. Could it be that you have people who will give you good references because you're a good developer? >Barring that I have half a dozen friends that would tell you whatever I want them to. So if I pull those friends up on linked in, they're developers too, and you'll have some plausible overlapping work history? > I fail to see how references are any sort of magic bullet. It's not a magic bullet. But faking 10 years of development work history and multiple decent quality references is a fairly high bar. It's not going to catch everyone but neither is whiteboarding. The goal shouldn't be to catch everyone. Sometimes you make a bad hire. Despite every test you put someone through, you'll get false positives. The problem is that if you ratchet your test up to far to avoid false positives, you end up turning off good engineers, or failing good engineers. I think the problem is that the company wasn't willing to just admit they made a mistake and fire the guy, not that he was hired in the first place. America (assuming you're in the US) has nearly unrestricted at will employment. Why are you so afraid of hiring one bad employee? If you can't identify bad employees after a few weeks of working with, you have zero chance identifying them before you hire them. Maybe instead of spending months trying to find the perfect candidate, just do what literally every other industry does and hire experienced candidates based on reputation, resume, talking to them about the job and past projects, and fire them if it turns out they were completely lying to you.
throwaway987978 6y ago
Is building a priority queue a common task that senior engineers would be required to do if they got the job?
- shaftway 6y ago
  Literally came out of our work queue. Couldn't find a satisfactory off-the-shelf component for it, needed one built. Afterwards we liked it as a technical interview question.
  - learc83 6y ago
    I can almost guarantee that the person who implemented it as part of their job brushed up on priority queues before they built it. I remember how to build one from a binary heap, but I wouldn't launch right into the code without reading a bit first.
    - shaftway 6y ago
      It was me. I can't remember how I built it. If I was doing it today I'd use some sort of balanced tree using the priority to navigate the nodes. Each node would have pointers to the head and tail of a singly linked list. Insert when the priority isn't in the tree is a simple tree insert with head and tail pointing to a new single-element linked list, insert when the priority is in there is adding it to the tail of the right node and moving the tail pointer, removing is finding the right-most node, taking the head and advancing the head pointer by one element. If the list is now empty then delete the node. But none of it matters. I don't care if someone would build it the way I would. I would never expect that. I don't even want an optimal or semi-optimal answer. I'd have accepted a singly linked list with O(n) insertion time and O(1) pop time. Or a plain list where new items are added to the end and a pop is an O(n) seek through the list. What I care about is whether the developer come up with any solution, describe the tradeoffs that they are making, why they are making them, and what research they would do if any to find a better solution. If i could have this conversation that we're having right here, right now with them then they'd probably pass my hiring bar.
triyambakam 6y ago
Conceptually I know what a priority queue is and how it works but I haven't built one, though I've used many. How satisfied would you be if an interviewee implementated it in pseudo code?
- shaftway 6y ago
  Thrilled. Probably a strong hire. I don't expect anyone to know what one is off the top of their head, but if I explain the goal to you I expect you to figure out roughly how to accomplish it without using XPath.
  - robocat 6y ago
    Now someone is going to use “Implement a priority queue using XSLT/XPATH given this XML data” as an interview question for a new hire... filtering for people willing to take the job even after being asked to do that!
_qulr 6y ago
Most arguments I've seen claiming the incompetence of senior engineers are circular. A: We need programming quizzes even for senior engineers, because many turn out of be incompetent. B: How do you know they're incompetent? A: They failed our programming quizzes! The reality is that you're testing for stage fright, not programming skills. An audition is not a good criterion for choosing people who aren't stage performers. https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/ https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
- robocat 6y ago
  Stage fright does not lead a senior level engineer to build a priority queue using XML. Assume good faith: let’s presume he knows some candidates get stage fright and has ways to help mitigate it. Or that despite stage fright, such tests are still a good signal (e.g. the signal is so strong the false negatives are a worthwhile cost, or stage fright might be highly correlated with poor performance, or stage performance could be highly correlated with talent!)
  - _qulr 6y ago
    > Assume good faith That's an interesting defense of an interview process that specifically does not assume good faith of the candidates. > has ways to help mitigate it I'm not sure this understands the nature of true stage fright. There aren't any good ways to mitigate it.
    - robocat 6y ago
      “Assume good faith” is an hn guideline for comments, not interviews. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Re mitigation: already answered with previous “or” clause.
      - _qulr 6y ago
        But "has ways to help mitigate" stage fright has nothing to do with assuming good faith. You'd have to assume magical powers. It's like if someone has a fear of spiders, and you bring out spiders during an interview but claim "we have ways of mitigating this fear."
shadowfox 6y ago
> Can you guess what the most common methodology people chose was? > Did you guess an XML document? If that is your most common experience with asking that question, maybe it is time you consider whether you are communicating what you want properly to your candidates.
- shaftway 6y ago
  The real WTF was probably the hiring agency that the hiring manager insisted we go through. I'm pretty sure they were coaching their candidates, but we couldn't break away from that agency for some reason (cough kickbacks cough). The answers were suspiciously similar. But I don't know how much better I could have described what I wanted: "I want a class for holding a queue of items in memory. The queue will only need to hold a few thousand items, and you don't need to worry about persisting it in case of a power failure or anything. The only wrinkle is that when an item is added to a queue it has a priority. Higher priority items need to come out of the queue before lower priority items, but items with the same priority should come out in the order they were inserted."
  - WWLink 6y ago
    I don't get it either. How did they suggest using an XML document to act as a priority queue? I actually would have found that highly amusing! My naive contribution: A linked list with pointers to the tails of various priorities. The grabber just points to the head and pops things off the top as it goes.
    - shaftway 6y ago
      > How did they suggest using an XML document to act as a priority queue? I actually would have found that highly amusing! I tried. It got tiring. Sometimes people would loop through all possible priorities plugging them into an XPath query and pop the first element of the highest priority that matched. Some would do two XPath queries, one across all nodes to get the max priority and a second to query the first element (ignoring the lack of a need to run the query a second time at all). Some would insist that XPath has some function that would let you select a node with an attribute that has the maximum numeric value. I'd have been fine with your solution.
decafninja 6y ago
As someone who has worked at a few of the "top 10 banks in NYC" for most of my career (and still am...albeit trying to desperately get out), I have to say that the quality of much of the engineering talent at these firms, and the typical candidates that would apply there, are pretty low. Yes, I realize that the above statement can be construed as spitting on my own face in a way. So I am not surprised. There may be a few small niches that attract good quality talent, but otherwise I don't know why anyone would opt to work as a technologist at a bank as opposed to say, a bonafide real tech company (all the banks are eager to say they are "tech companies" nowadays), or even a hedge fund.
walshemj 6y ago
Implementing a queue in XML why on earth would you do that

learc83 6y ago

Did you ever actual hire any of those people? Did you ever verify that they actually had the experience they claimed? Did you follow up with their references? Were they able to walk you through projects they worked, or discuss solutions implemented? Every job gets applicants that aren't qualified. Only software engineering out of every single non-performance based (musicians, actors etc...) industry out there has white board interviews.

shaftway 6y ago
Actually, yeah, we did. The hiring manager got tired of waiting for us to find a good candidate, so he just picked one that he had screened and seemed best on paper. It was an unmitigated disaster. Constantly had to double check his work and redirect him from doing things bad ways (like using mutable objects with changing hash codes as dictionary keys kind of bad). Eventually I quit the job (for other reasons). On the last day he cornered me in the elevator and offered me $100/hr to do his work for him under the table so that he could claim it was his. > Did you ever verify that they actually had the experience they claimed? Employer would only verify employment dates and eligibility for re-hire. > Did you follow up with their references? Yeah, but when you ask someone for references they're not likely to give you people who wouldn't speak well of them. > Were they able to walk you through projects they worked, or discuss solutions implemented? Yeah. I can walk you through projects that I was involved in as a junior as if I had been the lead because I paid attention during meetings.
- learc83 6y ago
  So you had one bad experience with one person who didn't do a whiteboard? How many bad experiences have you had with someone who did? I've had enough that I've lost count. >like using mutable objects with changing hash codes as dictionary keys kind of bad I don't think a leet code question would necessarily have screened stuff like that out at all. >Yeah, but when you ask someone for references they're not likely to give you people who wouldn't speak well of them. Did you actually talk with these references about what it was like to work with him? >On the last day he cornered me in the elevator and offered me $100/hr to do his work for him under the table so that he could claim it was his. That's $200k a year. I find it pretty hard to believe that he was serious.
  - shaftway 6y ago
    > So you had one bad experience with one person who didn't do a whiteboard? It was a bad experience with someone who utterly failed a whiteboard. > How many bad experiences have you had with someone who did? I've had enough that I've lost count. > I don't think a leet code question would necessarily have screened stuff like that out at all. I'm sure I have too, but I never advocated for only whiteboard interviewing. There should be a spread covering all sorts of aspects of their abilities. > Did you actually talk with these references about what it was like to work with him? Of course. I have a wide variety of coworkers with a wide variety of opinions of my skill-set. I'd select the ones that I think would resonate best with you. Barring that I have half a dozen friends that would tell you whatever I want them to. I fail to see how references are any sort of magic bullet. > That's $200k a year. I find it pretty hard to believe that he was serious. I was another senior engineer on the project, so I definitely could have covered the work. I had deeper experience, and given his quality I could have churned things out more quickly. I suspect he wasn't offering me $100 / hour at 40 hours per week. The job had a lot of paperwork and bureaucracy involved. I suspect if I was just doing the coding parts I could have covered it at about 10 hours per week, which is a lot more feasible. Obviously I turned this down for ethical reasons.
    - learc83 6y ago
      >Of course. I have a wide variety of coworkers with a wide variety of opinions of my skill-set. I'd select the ones that I think would resonate best with you. Could it be that you have people who will give you good references because you're a good developer? >Barring that I have half a dozen friends that would tell you whatever I want them to. So if I pull those friends up on linked in, they're developers too, and you'll have some plausible overlapping work history? > I fail to see how references are any sort of magic bullet. It's not a magic bullet. But faking 10 years of development work history and multiple decent quality references is a fairly high bar. It's not going to catch everyone but neither is whiteboarding. The goal shouldn't be to catch everyone. Sometimes you make a bad hire. Despite every test you put someone through, you'll get false positives. The problem is that if you ratchet your test up to far to avoid false positives, you end up turning off good engineers, or failing good engineers. I think the problem is that the company wasn't willing to just admit they made a mistake and fire the guy, not that he was hired in the first place. America (assuming you're in the US) has nearly unrestricted at will employment. Why are you so afraid of hiring one bad employee? If you can't identify bad employees after a few weeks of working with, you have zero chance identifying them before you hire them. Maybe instead of spending months trying to find the perfect candidate, just do what literally every other industry does and hire experienced candidates based on reputation, resume, talking to them about the job and past projects, and fire them if it turns out they were completely lying to you.

throwaway987978 6y ago

Is building a priority queue a common task that senior engineers would be required to do if they got the job?

shaftway 6y ago
Literally came out of our work queue. Couldn't find a satisfactory off-the-shelf component for it, needed one built. Afterwards we liked it as a technical interview question.
- learc83 6y ago
  I can almost guarantee that the person who implemented it as part of their job brushed up on priority queues before they built it. I remember how to build one from a binary heap, but I wouldn't launch right into the code without reading a bit first.
  - shaftway 6y ago
    It was me. I can't remember how I built it. If I was doing it today I'd use some sort of balanced tree using the priority to navigate the nodes. Each node would have pointers to the head and tail of a singly linked list. Insert when the priority isn't in the tree is a simple tree insert with head and tail pointing to a new single-element linked list, insert when the priority is in there is adding it to the tail of the right node and moving the tail pointer, removing is finding the right-most node, taking the head and advancing the head pointer by one element. If the list is now empty then delete the node. But none of it matters. I don't care if someone would build it the way I would. I would never expect that. I don't even want an optimal or semi-optimal answer. I'd have accepted a singly linked list with O(n) insertion time and O(1) pop time. Or a plain list where new items are added to the end and a pop is an O(n) seek through the list. What I care about is whether the developer come up with any solution, describe the tradeoffs that they are making, why they are making them, and what research they would do if any to find a better solution. If i could have this conversation that we're having right here, right now with them then they'd probably pass my hiring bar.

triyambakam 6y ago

Conceptually I know what a priority queue is and how it works but I haven't built one, though I've used many. How satisfied would you be if an interviewee implementated it in pseudo code?

shaftway 6y ago
Thrilled. Probably a strong hire. I don't expect anyone to know what one is off the top of their head, but if I explain the goal to you I expect you to figure out roughly how to accomplish it without using XPath.
- robocat 6y ago
  Now someone is going to use “Implement a priority queue using XSLT/XPATH given this XML data” as an interview question for a new hire... filtering for people willing to take the job even after being asked to do that!

_qulr 6y ago

Most arguments I've seen claiming the incompetence of senior engineers are circular. A: We need programming quizzes even for senior engineers, because many turn out of be incompetent. B: How do you know they're incompetent? A: They failed our programming quizzes! The reality is that you're testing for stage fright, not programming skills. An audition is not a good criterion for choosing people who aren't stage performers. https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/ https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/

robocat 6y ago
Stage fright does not lead a senior level engineer to build a priority queue using XML. Assume good faith: let’s presume he knows some candidates get stage fright and has ways to help mitigate it. Or that despite stage fright, such tests are still a good signal (e.g. the signal is so strong the false negatives are a worthwhile cost, or stage fright might be highly correlated with poor performance, or stage performance could be highly correlated with talent!)
- _qulr 6y ago
  > Assume good faith That's an interesting defense of an interview process that specifically does not assume good faith of the candidates. > has ways to help mitigate it I'm not sure this understands the nature of true stage fright. There aren't any good ways to mitigate it.
  - robocat 6y ago
    “Assume good faith” is an hn guideline for comments, not interviews. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Re mitigation: already answered with previous “or” clause.
    - _qulr 6y ago
      But "has ways to help mitigate" stage fright has nothing to do with assuming good faith. You'd have to assume magical powers. It's like if someone has a fear of spiders, and you bring out spiders during an interview but claim "we have ways of mitigating this fear."

shadowfox 6y ago

> Can you guess what the most common methodology people chose was? > Did you guess an XML document? If that is your most common experience with asking that question, maybe it is time you consider whether you are communicating what you want properly to your candidates.

shaftway 6y ago
The real WTF was probably the hiring agency that the hiring manager insisted we go through. I'm pretty sure they were coaching their candidates, but we couldn't break away from that agency for some reason (cough kickbacks cough). The answers were suspiciously similar. But I don't know how much better I could have described what I wanted: "I want a class for holding a queue of items in memory. The queue will only need to hold a few thousand items, and you don't need to worry about persisting it in case of a power failure or anything. The only wrinkle is that when an item is added to a queue it has a priority. Higher priority items need to come out of the queue before lower priority items, but items with the same priority should come out in the order they were inserted."
- WWLink 6y ago
  I don't get it either. How did they suggest using an XML document to act as a priority queue? I actually would have found that highly amusing! My naive contribution: A linked list with pointers to the tails of various priorities. The grabber just points to the head and pops things off the top as it goes.
  - shaftway 6y ago
    > How did they suggest using an XML document to act as a priority queue? I actually would have found that highly amusing! I tried. It got tiring. Sometimes people would loop through all possible priorities plugging them into an XPath query and pop the first element of the highest priority that matched. Some would do two XPath queries, one across all nodes to get the max priority and a second to query the first element (ignoring the lack of a need to run the query a second time at all). Some would insist that XPath has some function that would let you select a node with an attribute that has the maximum numeric value. I'd have been fine with your solution.

decafninja 6y ago

As someone who has worked at a few of the "top 10 banks in NYC" for most of my career (and still am...albeit trying to desperately get out), I have to say that the quality of much of the engineering talent at these firms, and the typical candidates that would apply there, are pretty low. Yes, I realize that the above statement can be construed as spitting on my own face in a way. So I am not surprised. There may be a few small niches that attract good quality talent, but otherwise I don't know why anyone would opt to work as a technologist at a bank as opposed to say, a bonafide real tech company (all the banks are eager to say they are "tech companies" nowadays), or even a hedge fund.

walshemj 6y ago

Implementing a queue in XML why on earth would you do that