4 ms·

Can you share what kind of things these are that give far more signal?

by cadr 4y ago

dekhn 4y ago
To be honest? every programmer needs to be able to pass FizzBuzz. Beyond that, I add some more basic work around byte representations of data (I have 4 symbols... how many bits do I need to encode a symbol?). I will typpically also include one major bug in a piece of code and ask the user to identify it (with hints). I once interviewed a guy- a CTO at a biotech- and he wouldn't answer the question "I have a million DNA sequences and want to count the number of occurrences of each sequence" (could be hash table, could be any number of other solutions; he just balked and exited the interview). Only if the person can get all that right would I even consider asking something more complicated.
- cadr 4y ago
  I agree liking to ask simpler stuff first. But what about this more complicated stuff? My problem is that when we talk about stuff having more "signal" - how are we determining if those questions are giving us more signal?
  - dekhn 4y ago
    My question is fairly open and allows me to continue asking significantly more complicated questions. So far, nobody has (for example) asked enough clarifying questions to determine that a bloom counting filter could potentially solve the problem- most people just store ASCII string keys in a hash table, which wastes tons of memory and requires a lookup for every string. So usually I end up after 45 minutes finding that the candidate has more or less tapped themselves out at "make a hash table of string keys, use it to store counts" (OK for a very junior programmer) or "make a perfect minimal hash" (I help them get there if they don't know what those are) or "use a probabilistic counting filter". This is about all I need to make a determination.
    - cadr 4y ago
      But have you had a chance to look back and see if those questions worked out? Like, are they a good predictor for how the person actually performed in the job?
      - dekhn 4y ago
        I've only been directly involved in hiring five people and it worked out well for all of those
        cadr 4y ago
        cool. Thank you!
    - visarga 4y ago
      Bloom filters have collisions. How would that work with counting sequences?
      - dekhn 4y ago
        Because (and this is absolutely a clarifying question I would love to get), approximate counts are acceptable. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-333 https://bmcbioinformatics.biomedcentral.com/articles/10.1186...
    - lazyant 4y ago
      The problem with this question is that outside the junior people who won't understand the hints, many good programmers would pass or fail just based on whether they have seen a similar problem or know what a Bloom filter is. CS is so broad that you may have been working for many years and never come across one topic that the interviewer deems standard.
      - dekhn 4y ago
        I don't fail people who don't suggest a bloom filter. What I meant was: knowledge of probabilistic data structures and knowing this is a situation where they could be used is a sign of a more experienced programmer (likely one who's worked in clickstream processing). If all you do in my interview is know that a hash table or other associative data structure is good for maintaining counts, that using a full ASCII byte to store symbols from { A G T C } is wasteful and compress the letters to 2 bits each, you can pack multiple 2 bits into bytes, and can write a function that codes and decodes such data, I'm happy and you get a pass. I'm even happy to sit there and help people through nearly all the steps of the codec. Not trying to trick anybody or select for obscure CS knowledxge. To me the real question is, how much hinting is reasonable for the coding and decoding function? Many programmers (including senior ones) struggle to implement: x <<= 2 # left shift the int to make room for the next item x |= pattern # or the new 2-bit item into the accumulator. Since I never use bit munging in my day job (it's 90% python data science) should I really ding somebody for not knowing about left shift or or-assignment? What I really don't get is why people immediately jump to trying to implement huffman coding, RLE, or lookbacks as a solution to reduce the size of the DNA string. I still wonder if how I present the question is giving everybody a fair chance to shine.
- akhmatova 4y ago
  I once interviewed a guy- a CTO at a biotech- and he wouldn't answer the question "I have a million DNA sequences and want to count the number of occurrences of each sequence" (could be hash table, could be any number of other solutions; he just balked and exited the interview). Maybe he's just been around the block a sufficient number of times to have gotten tired of the standard SV "here's a hoop, jump through it, now do it again, and again and again" ritual that it likes to call "interviewing" for some reason. Seriously - your questions are fine for junior or mid-level roles, but beyond that, you should have more important thinks to talk about. And both of you should be able to tell if the other is a bullshitter and/or a lightweight within about 30 minutes at most (and often far quicker than that) of a normal, focused technical conversation -- without having to specifically grill the other person, or otherwise put on the pretension that you're one whose bona fides are established beyond question, and they're the one who needs to jump through a sequence of hoops to prove that they are worthy of your time and attention. Just cut the crap, get down to brass tacks, and talk what needs to be done as if they're a peer. That's all you have to do.

Can you share what kind of things these are that give far more signal?

by cadr 4y ago

Can you share what kind of things these are that give far more signal?

To be honest? every programmer needs to be able to pass FizzBuzz. Beyond that, I add some more basic work around byte representations of data (I have 4 symbols... how many bits do I need to encode a symbol?). I will typpically also include one major bug in a piece of code and ask the user to identify it (with hints). I once interviewed a guy- a CTO at a biotech- and he wouldn't answer the question "I have a million DNA sequences and want to count the number of occurrences of each sequence" (could be hash table, could be any number of other solutions; he just balked and exited the interview). Only if the person can get all that right would I even consider asking something more complicated.

cadr 4y ago
I agree liking to ask simpler stuff first. But what about this more complicated stuff? My problem is that when we talk about stuff having more "signal" - how are we determining if those questions are giving us more signal?
- dekhn 4y ago
  My question is fairly open and allows me to continue asking significantly more complicated questions. So far, nobody has (for example) asked enough clarifying questions to determine that a bloom counting filter could potentially solve the problem- most people just store ASCII string keys in a hash table, which wastes tons of memory and requires a lookup for every string. So usually I end up after 45 minutes finding that the candidate has more or less tapped themselves out at "make a hash table of string keys, use it to store counts" (OK for a very junior programmer) or "make a perfect minimal hash" (I help them get there if they don't know what those are) or "use a probabilistic counting filter". This is about all I need to make a determination.
  - cadr 4y ago
    But have you had a chance to look back and see if those questions worked out? Like, are they a good predictor for how the person actually performed in the job?
    - dekhn 4y ago
      I've only been directly involved in hiring five people and it worked out well for all of those
      - cadr 4y ago
        cool. Thank you!
  - visarga 4y ago
    Bloom filters have collisions. How would that work with counting sequences?
    - dekhn 4y ago
      Because (and this is absolutely a clarifying question I would love to get), approximate counts are acceptable. https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-333 https://bmcbioinformatics.biomedcentral.com/articles/10.1186...
  - lazyant 4y ago
    The problem with this question is that outside the junior people who won't understand the hints, many good programmers would pass or fail just based on whether they have seen a similar problem or know what a Bloom filter is. CS is so broad that you may have been working for many years and never come across one topic that the interviewer deems standard.
    - dekhn 4y ago
      I don't fail people who don't suggest a bloom filter. What I meant was: knowledge of probabilistic data structures and knowing this is a situation where they could be used is a sign of a more experienced programmer (likely one who's worked in clickstream processing). If all you do in my interview is know that a hash table or other associative data structure is good for maintaining counts, that using a full ASCII byte to store symbols from { A G T C } is wasteful and compress the letters to 2 bits each, you can pack multiple 2 bits into bytes, and can write a function that codes and decodes such data, I'm happy and you get a pass. I'm even happy to sit there and help people through nearly all the steps of the codec. Not trying to trick anybody or select for obscure CS knowledxge. To me the real question is, how much hinting is reasonable for the coding and decoding function? Many programmers (including senior ones) struggle to implement: x <<= 2 # left shift the int to make room for the next item x |= pattern # or the new 2-bit item into the accumulator. Since I never use bit munging in my day job (it's 90% python data science) should I really ding somebody for not knowing about left shift or or-assignment? What I really don't get is why people immediately jump to trying to implement huffman coding, RLE, or lookbacks as a solution to reduce the size of the DNA string. I still wonder if how I present the question is giving everybody a fair chance to shine.
akhmatova 4y ago
I once interviewed a guy- a CTO at a biotech- and he wouldn't answer the question "I have a million DNA sequences and want to count the number of occurrences of each sequence" (could be hash table, could be any number of other solutions; he just balked and exited the interview). Maybe he's just been around the block a sufficient number of times to have gotten tired of the standard SV "here's a hoop, jump through it, now do it again, and again and again" ritual that it likes to call "interviewing" for some reason. Seriously - your questions are fine for junior or mid-level roles, but beyond that, you should have more important thinks to talk about. And both of you should be able to tell if the other is a bullshitter and/or a lightweight within about 30 minutes at most (and often far quicker than that) of a normal, focused technical conversation -- without having to specifically grill the other person, or otherwise put on the pretension that you're one whose bona fides are established beyond question, and they're the one who needs to jump through a sequence of hoops to prove that they are worthy of your time and attention. Just cut the crap, get down to brass tacks, and talk what needs to be done as if they're a peer. That's all you have to do.