5 ms·

In the course of interviewing a bunch of developers, and employing a few of them, I've concluded that this ability/inclination/something to do this deeper diggi

by niccl 2y ago

In the course of interviewing a bunch of developers, and employing a few of them, I've concluded that this ability/inclination/something to do this deeper digging is one of the things I prize most in a developer. They have to know when to go deep and when not to, though, and that's sometimes a hard balancing act. I've never found a good way of screening for the ability, and more, for when not to go deep, because everyone will come up with some example if you ask, and it's not the sort of thing that I can see highlighting in a coding test (and _certainly_ not in a leet-code test!). If anyone has any suggestions on how to uncover it during the hiring process I'd be ecstatic!

deleted 2y ago
[deleted]
giantg2 2y ago
"I've concluded that this ability/inclination/something to do this deeper digging is one of the things I prize most in a developer." Where have you been all my life? It seems most of teams I've been on value speed over future proofing bugs. The systems thinking approach is rare. If you want to test for this, you can create a PR for a fake project. Make sure the project runs but has error, code smells, etc. Have a few things like they talk about in the article, like a message of being out of disk space but missing critical message/logging infrastructure to cover other scenarios. The best part is, you can use the same PR for all levels that you're hiring for by expecting senior to get X% of the bugs, mids to get X/2% and noobs to get X/4%.
- niccl 2y ago
  That's a really good idea. Thanks
- jerf 2y ago
  "It seems most of teams I've been on value speed over future proofing bugs." So, obviously, if one team is future proofing bugs, and the other team just blasts out localized short-term fixes as quickly as possible, there will come a point where the first team will overtake the second, because the second team's velocity will by necessity has to slow down more than the first as the code base grows. If the crossover point is ten years hence, then it only makes sense to be the second team. However, what I find a bit horrifying as a developer is that my estimate of the crossover point keeps coming in. When I'm working by myself on greenfield code, I'd put it at about three weeks; yes, I'll go somewhat faster today if I just blast out code and skip the unit tests, but it's only weeks before I'm getting bitten by that. Bigger teams may have a somewhat farther cross over point, but it's still likely to be small single-digit months. There is of course overdoing it and being too perfectionist, and that does get some people, but the people, teams, managers, and companies who always vote for the short term code blasting simply have no idea how much performance they are leaving on the table almost immediately. Established code bases are slower to turn, naturally. But even so, I still think the constant short-term focus is vastly more expensive than those who choose it understand. And I don't even mean obvious stuff like "oh, you'll have more bugs" or "oh, it's so much harder to on board", even if that's true... no, I mean, even by the only metric you seem care about, the team that takes the time to fix fundamental issues and invests in better logging and metrics and all those things you think just slow you down can also smoke you on dev speed after a couple of months... and they'll have the solid code base, too! "Make sure the project runs but has error, code smells, etc." It is a hard problem to construct a test for this but it would be interesting to provide the candidate some code that compiles with warnings and just watch them react to the warnings. You may not learn everything you need but it'll certainly teach you something.
  - daelon 2y ago
    Slow is smooth, smooth is fast.
  - ozim 2y ago
    Unfortunately I believe there is no crossing point even in 10 years. If quick fix works it is most likely a proper fix, if it doesn’t work then you dig deeper. It is also case if feature to be fixed is even worth spending so much time.
    - rocqua 2y ago
      A quick fix works now. It makes the next fix or change much harder because it just added a special case, or ignored an edge case that wasn't possible in the configuration at that time.
      - ozim 2y ago
        My main point is That’s false dichotomy. There is bunch of stuff that could be “fixed better” or “properly” if someone took a better look but also a lot of times it is just good enough and is not somehow magically impeding proper fix.
        jerf 2y ago
        It is and it isn't a false dichotomy. It is a false dichotomy in that in the Aristotelian sense of "X -> Y" means that absolutely, positively every X must with 100% probability lead to Y, it is absolutely true that "This is a quick fix -> This not the best fix" is false. Sometimes the quick fix is correct. A quick example: I'm doing some math of some sort and literally typed minus instead of plus. The quick fix to change minus to plus is reasonable. (If you're wondering about testing, well, let's say I wrote unit tests to assert the wrong code. I've written plenty of unit tests that turn out to be asserting the wrong thing. So the quick fix may involve fixing those too.) It is true in the sense that if you plot the quickness of the fix versus the correctness of the fix, you're not going to get a perfectly uniformly random two dimensional graph that would indicate they are uncorrelated. You'll get some sort of Pareto-optimal[1] front that will develop, becoming more pronounced as the problem and minimum size fix become larger (and they can get pretty large in programming). It'll be a bit loose, you'll get occasional outliers where you have otherwise fantastic code that just happened to have this tiny screw loose that caused a lot of problems everywhere and one quick fix can fix a lot of issues at once; I think a lot of us will see those once or twice a decade or so, but for the most part, there will develop a definite trend that once you eliminate all the fixes that are neither terribly fast nor terribly good for the long term, there will develop a fairly normal "looks like 1/x" curve of tradeoffs between speed and long-term value. This is a very common pattern across many combinations of X and Y that don't literally, 100% oppose each other, but in the real world, with many complicated interrelated factors interacting with each other and many different distributions of effort and value interacting, do contradict each other... but only if you are actually on the Pareto frontier! For practical purposes in this case I think we usually are, at least relative to the local developers fixing the bug; nobody deliberately sets out to make a fix that is visibly obviously harder than it needs to be and less long-term valuable than it needs to be. My favorite "false dichotomy" that arises is the supposed contradiction between security and usability. It's true they oppose each other... but only if your program is already roughly optimally usable and secure on the Pareto frontier and now you really can't improve one without diminishing the other. Most programs aren't actually there, and thus there are both usability and security improvements that can be made without affecting the other. I'm posting this because this is one of those things that sounds really academic and abstruse and irrelevant, but if you learn to see it, becomes very practical and powerful for your own engineering. [1]: https://en.wikipedia.org/wiki/Pareto_front https://en.wikipedia.org/wiki/Pareto_front
- ozim 2y ago
  I have seen enough BSers who claimed that they need “do the proper fix” doing analysis and wasting everyone’s time. They would be vocal about it and then spend weeks delivering nothing “tweaking db indexes” while I immediately have seen code was crap and needed slight changes but I also don’t have time to fight all the fights in the company.
  - giantg2 2y ago
    That's the thing, my comment wasn't about that long analysis or doing the proper fix. It's all about asking if this is the root cause or not, or is there a similar related bug not yet identified. You could find a root cause and bring it back to the team if it's going to take weeks. At that point the team has the say on if that fix is necessary.
bongodongobob 2y ago
Knowing when to go down the rabbit hole is probably more about experience/age than anything. I work with a very intelligent junior that is constantly going down rabbit holes. His heart is in the right spot but sometimes you just need to make things work/get things done. I used to do it a lot too and I kind of had a "shit, I'm getting old" moment the other day when I was telling him something along the lines of "yeah, we could probably fix that deeper but it's going to take 6 weeks of meetings and 3 departments to approve this. Is that really what you want to spend your time on?" Like you said, it's definitely a balancing act and the older I get, the less I care about "doing things the right way" when no one actually cares or will know. I get paid to knock out tickets, so that's what I'm going to do. I'll let the juniors spin their wheels and burn mental CPU on the deep dives and I'm around to lend a hand when they need it.
- layer8 2y ago
  However, you have to overdo it a sufficient number of times when you’re still inexperienced, in order to gain the experience of when it’s worth it and when it’s not. You have to make mistakes in order to learn from them.
  - giantg2 2y ago
    When it's worth it and when it's not seems to be more of a business question for the product owner. It's all opinion. I've been on a where I had 2 weeks left and they didn't want me working on anything high priority during that time so it wouldn't be half finished when I left. I had a couple small stories I was assigned. Then I decide to cherrypick the backlog to see how much tech debt I could close for the team before I left. I cleared something like 11 stories out of 100. I was then chewed out by the product owner because she "would have assigned [me] other higher priority stories". But the whole point was that I wasn't suppose dto be on high priority tasks because I'm leaving...
    - seadan83 2y ago
      Why product owner? (Perhaps rather not say team lead?) Are these deeply technical product owners? Which ones would be best to make this decision and which less?
atoav 2y ago
Such qualities can sometimes be unearthed when you ask candidates to deal with a problem they can't know the answer to. In the end the ability to go deep has a lot to do with them being confident in their ability to be able to understand things that are new to them. Most people can go into a deep dive if you force them to do it, but how they conduct themselves while doing it can show you if this is a thing they would do on their own.
deleted 2y ago
[deleted]
iamcreasy 2y ago
Mike Acton said the same thing in an interview. He said curiosity is the best indicator if a candidate will be a good hire. Casey Muratori was interviewing him at HandmadeHero Con back in 2016. Here is the snippet: https://youtu.be/qWJpI2adCcs?si=ezSKud42PC3Ub-UO&t=3112 https://youtu.be/qWJpI2adCcs?si=ezSKud42PC3Ub-UO&t=3112
variadix 2y ago
Surely there is a way to present some piece of buggy code to a candidate and ask what’s wrong with it (letting them use whatever tools they want, doing this on a whiteboard is senseless), let them determine what the bug is and see how they fix it. Obviously there are constraints that make the code in question difficult to construct (needs to be simple and small enough to fit in an interview without the candidate having ever seen the code before, but not too simple to make it a non-differentiating question; the bug has to have multiple levels to it where there’s an obvious fix and possibly several better but less obvious ways to fix the issue, etc)

by niccl 2y ago

deleted 2y ago

[deleted]

giantg2 2y ago

"I've concluded that this ability/inclination/something to do this deeper digging is one of the things I prize most in a developer." Where have you been all my life? It seems most of teams I've been on value speed over future proofing bugs. The systems thinking approach is rare. If you want to test for this, you can create a PR for a fake project. Make sure the project runs but has error, code smells, etc. Have a few things like they talk about in the article, like a message of being out of disk space but missing critical message/logging infrastructure to cover other scenarios. The best part is, you can use the same PR for all levels that you're hiring for by expecting senior to get X% of the bugs, mids to get X/2% and noobs to get X/4%.

niccl 2y ago
That's a really good idea. Thanks
jerf 2y ago
"It seems most of teams I've been on value speed over future proofing bugs." So, obviously, if one team is future proofing bugs, and the other team just blasts out localized short-term fixes as quickly as possible, there will come a point where the first team will overtake the second, because the second team's velocity will by necessity has to slow down more than the first as the code base grows. If the crossover point is ten years hence, then it only makes sense to be the second team. However, what I find a bit horrifying as a developer is that my estimate of the crossover point keeps coming in. When I'm working by myself on greenfield code, I'd put it at about three weeks; yes, I'll go somewhat faster today if I just blast out code and skip the unit tests, but it's only weeks before I'm getting bitten by that. Bigger teams may have a somewhat farther cross over point, but it's still likely to be small single-digit months. There is of course overdoing it and being too perfectionist, and that does get some people, but the people, teams, managers, and companies who always vote for the short term code blasting simply have no idea how much performance they are leaving on the table almost immediately. Established code bases are slower to turn, naturally. But even so, I still think the constant short-term focus is vastly more expensive than those who choose it understand. And I don't even mean obvious stuff like "oh, you'll have more bugs" or "oh, it's so much harder to on board", even if that's true... no, I mean, even by the only metric you seem care about, the team that takes the time to fix fundamental issues and invests in better logging and metrics and all those things you think just slow you down can also smoke you on dev speed after a couple of months... and they'll have the solid code base, too! "Make sure the project runs but has error, code smells, etc." It is a hard problem to construct a test for this but it would be interesting to provide the candidate some code that compiles with warnings and just watch them react to the warnings. You may not learn everything you need but it'll certainly teach you something.
- daelon 2y ago
  Slow is smooth, smooth is fast.
- ozim 2y ago
  Unfortunately I believe there is no crossing point even in 10 years. If quick fix works it is most likely a proper fix, if it doesn’t work then you dig deeper. It is also case if feature to be fixed is even worth spending so much time.
  - rocqua 2y ago
    A quick fix works now. It makes the next fix or change much harder because it just added a special case, or ignored an edge case that wasn't possible in the configuration at that time.
    - ozim 2y ago
      My main point is That’s false dichotomy. There is bunch of stuff that could be “fixed better” or “properly” if someone took a better look but also a lot of times it is just good enough and is not somehow magically impeding proper fix.
      - jerf 2y ago
        It is and it isn't a false dichotomy. It is a false dichotomy in that in the Aristotelian sense of "X -> Y" means that absolutely, positively every X must with 100% probability lead to Y, it is absolutely true that "This is a quick fix -> This not the best fix" is false. Sometimes the quick fix is correct. A quick example: I'm doing some math of some sort and literally typed minus instead of plus. The quick fix to change minus to plus is reasonable. (If you're wondering about testing, well, let's say I wrote unit tests to assert the wrong code. I've written plenty of unit tests that turn out to be asserting the wrong thing. So the quick fix may involve fixing those too.) It is true in the sense that if you plot the quickness of the fix versus the correctness of the fix, you're not going to get a perfectly uniformly random two dimensional graph that would indicate they are uncorrelated. You'll get some sort of Pareto-optimal[1] front that will develop, becoming more pronounced as the problem and minimum size fix become larger (and they can get pretty large in programming). It'll be a bit loose, you'll get occasional outliers where you have otherwise fantastic code that just happened to have this tiny screw loose that caused a lot of problems everywhere and one quick fix can fix a lot of issues at once; I think a lot of us will see those once or twice a decade or so, but for the most part, there will develop a definite trend that once you eliminate all the fixes that are neither terribly fast nor terribly good for the long term, there will develop a fairly normal "looks like 1/x" curve of tradeoffs between speed and long-term value. This is a very common pattern across many combinations of X and Y that don't literally, 100% oppose each other, but in the real world, with many complicated interrelated factors interacting with each other and many different distributions of effort and value interacting, do contradict each other... but only if you are actually on the Pareto frontier! For practical purposes in this case I think we usually are, at least relative to the local developers fixing the bug; nobody deliberately sets out to make a fix that is visibly obviously harder than it needs to be and less long-term valuable than it needs to be. My favorite "false dichotomy" that arises is the supposed contradiction between security and usability. It's true they oppose each other... but only if your program is already roughly optimally usable and secure on the Pareto frontier and now you really can't improve one without diminishing the other. Most programs aren't actually there, and thus there are both usability and security improvements that can be made without affecting the other. I'm posting this because this is one of those things that sounds really academic and abstruse and irrelevant, but if you learn to see it, becomes very practical and powerful for your own engineering. [1]: https://en.wikipedia.org/wiki/Pareto_front https://en.wikipedia.org/wiki/Pareto_front
ozim 2y ago
I have seen enough BSers who claimed that they need “do the proper fix” doing analysis and wasting everyone’s time. They would be vocal about it and then spend weeks delivering nothing “tweaking db indexes” while I immediately have seen code was crap and needed slight changes but I also don’t have time to fight all the fights in the company.
- giantg2 2y ago
  That's the thing, my comment wasn't about that long analysis or doing the proper fix. It's all about asking if this is the root cause or not, or is there a similar related bug not yet identified. You could find a root cause and bring it back to the team if it's going to take weeks. At that point the team has the say on if that fix is necessary.

bongodongobob 2y ago

Knowing when to go down the rabbit hole is probably more about experience/age than anything. I work with a very intelligent junior that is constantly going down rabbit holes. His heart is in the right spot but sometimes you just need to make things work/get things done. I used to do it a lot too and I kind of had a "shit, I'm getting old" moment the other day when I was telling him something along the lines of "yeah, we could probably fix that deeper but it's going to take 6 weeks of meetings and 3 departments to approve this. Is that really what you want to spend your time on?" Like you said, it's definitely a balancing act and the older I get, the less I care about "doing things the right way" when no one actually cares or will know. I get paid to knock out tickets, so that's what I'm going to do. I'll let the juniors spin their wheels and burn mental CPU on the deep dives and I'm around to lend a hand when they need it.

layer8 2y ago
However, you have to overdo it a sufficient number of times when you’re still inexperienced, in order to gain the experience of when it’s worth it and when it’s not. You have to make mistakes in order to learn from them.
- giantg2 2y ago
  When it's worth it and when it's not seems to be more of a business question for the product owner. It's all opinion. I've been on a where I had 2 weeks left and they didn't want me working on anything high priority during that time so it wouldn't be half finished when I left. I had a couple small stories I was assigned. Then I decide to cherrypick the backlog to see how much tech debt I could close for the team before I left. I cleared something like 11 stories out of 100. I was then chewed out by the product owner because she "would have assigned [me] other higher priority stories". But the whole point was that I wasn't suppose dto be on high priority tasks because I'm leaving...
  - seadan83 2y ago
    Why product owner? (Perhaps rather not say team lead?) Are these deeply technical product owners? Which ones would be best to make this decision and which less?

atoav 2y ago

Such qualities can sometimes be unearthed when you ask candidates to deal with a problem they can't know the answer to. In the end the ability to go deep has a lot to do with them being confident in their ability to be able to understand things that are new to them. Most people can go into a deep dive if you force them to do it, but how they conduct themselves while doing it can show you if this is a thing they would do on their own.

deleted 2y ago

[deleted]

iamcreasy 2y ago

Mike Acton said the same thing in an interview. He said curiosity is the best indicator if a candidate will be a good hire. Casey Muratori was interviewing him at HandmadeHero Con back in 2016. Here is the snippet: https://youtu.be/qWJpI2adCcs?si=ezSKud42PC3Ub-UO&t=3112 https://youtu.be/qWJpI2adCcs?si=ezSKud42PC3Ub-UO&t=3112

variadix 2y ago

Surely there is a way to present some piece of buggy code to a candidate and ask what’s wrong with it (letting them use whatever tools they want, doing this on a whiteboard is senseless), let them determine what the bug is and see how they fix it. Obviously there are constraints that make the code in question difficult to construct (needs to be simple and small enough to fit in an interview without the candidate having ever seen the code before, but not too simple to make it a non-differentiating question; the bug has to have multiple levels to it where there’s an obvious fix and possibly several better but less obvious ways to fix the issue, etc)