8 ms·

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What ma

So most interview questions like "make a binary tree" are dead. The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" For posterity: https://www.gamedeveloper.com/programming/my-hardest-bug-ever https://www.gamedeveloper.com/programming/my-hardest-bug-eve...

bcantrill 4y ago
Yes, agreed! We ask a variant of this question (we call it "an analysis sample"); from the materials we ask candidates to submit[0]: "A significant challenge of engineering is dealing with a system when it doesn't, in fact, work correctly. When systems misbehave, engineers must flip their disposition: instead of a creator of their own heaven and earth, they must become a scientist, attempting to reason about a foreign world. Please provide an analysis sample: a written analysis of system misbehavior from some point in your career. If such an analysis is not readily available (as it might not be if one’s work has been strictly proprietary), please recount an incident in which you analyzed system misbehavior, including as much technical detail as you can recall." These samples are very revealing -- and it feels unlikely that generative AI is going to be of much help, even assuming a fabulist candidate. (And of very little assistance on our values-based questions like "when have you been happiest in your professional career and why?"). [0] https://docs.google.com/document/d/1Xtofg-fMQfZoq8Y3oSAKjEgDQCRHx-GMSmPcxdEea4A/edit?usp=sharing https://docs.google.com/document/d/1Xtofg-fMQfZoq8Y3oSAKjEgD...
- epicureanideal 4y ago
  To be honest, this sounds extremely difficult and not in a good way. That sounds like many many hours of writing work, to describe a problem that might be many years in the past, that might have been solved by extremely intricate methods that are easy to forget, using technologies that are now not commonly in use, etc. A good question to ask about each interview question might be: would a good liar have an easier time answering this than a person trying to answer honestly? And if so, retire the question.
  - bcantrill 4y ago
    Having read many, many, many answers to this question, I don't think that a good liar has a particularly easy time answering this question -- or certainly not in a way that gets them further consideration! And yes, it's many hours of work -- but the work itself that we are doing is quite hard, and if someone washes out in the application process because it feels unduly arduous, we are likely not a fit for one another.
    - l33t233372 4y ago
      > I don't think that a good liar has a particularly easy time answering this question -- or certainly not in a way that gets them further consideration How would you know? > And yes, it's many hours of work -- but the work itself that we are doing is quite hard, and if someone washes out in the application process because it feels unduly arduous, we are likely not a fit for one another. I sincerely hope that I never accidentally apply for a company that thinks an unpaid, long form writing prompt is an appropriate interview question because the work happens to be hard.
    - xvector 4y ago
      Eh, it also excludes people that don't have spectacular long-term memory, or people that don't keep a diary about bugs that they've chased down at work. Personally, I think you're overfitting to fight cheating, but maybe you work at a desirable enough place that you can afford to exclude so many people but still get enough good candidates. IMO a good question provides the necessary context itself, and the candidate's thinking and reasoning skills are what's tested. With your question, it's basically turned into a competition of which candidate has tackled the most ridiculous/obscure/complex bug, so candidates aren't being judged on even footing.
      - mypalmike 4y ago
        Agreed completely. This problem borders on that common category of questions which test whether the applicant shares a specific fine-grained flavor of nerdiness as the interviewer, rather than whether the candidate is a good fit for the job.
razor_router 4y ago
That's interesting! What did you learn from debugging that weird bug?
- aeyes 4y ago
  I learned that off-by one errors, mixing up arguments and caches are hard to debug. I have easily spent days debugging many such problems which were almost always solved by a one line change. And rarely did I find ways to prevent similar bugs in the future bugs by improving testing or code factoring.
BoorishBears 4y ago
While I'm not complaining that people are realizing they're dead... why is ChatGPT the final straw for those ridiculous "make a binary tree" questions? Why wasn't it the fact that these questions became such a gameable system, that we started referring to them by the copyrighted name of a site where you can access nearly every permutation that will ever be asked of you, along with extremely detailed solutions with rationale: https://leetcode.com/ https://leetcode.com/ It's crazy to me that of everything that ChatGPT can do, regurgitating well known answers to well known interview questions is what kills anything off...
- dinkumthinkum 4y ago
  I mean if you can’t make a binary tree, what do you really know about programming? If ChatGPT is so great, why would even need a job interview, why not just eliminate the jobs altogether?
  - BoorishBears 4y ago
    You know, the only thing I get more sure of over the years is that defining a "good programmer" is hard Now, I can't tell if you're being facetious or not, but if one seriously conflates being able to write a binary tree with knowing how to program... they're at least making defining the antonym a bit easier. - Also no one is saying ChatGPT is "so great" in this specific case, if anything the point is that ChatGPT can do impressive things, but again, regurgitating leetcode answers isn't one of them.
    - dinkumthinkum 4y ago
      I’m not trying to be mean but “conflating” is not the same as “necessary.” There is a very common notion of “necessary but not sufficient”. Yes, I think if you don’t know how to make a binary tree, you don’t know much about programming. The converse is not true.
  - psyklic 4y ago
    I view general algorithm problems as a "programming subfield", just like graphics/frontend/AI are. Certain coding techniques are overrepresented in each subfield, and each subfield has fundamental "base code", design paradigms, methods of testing, need-to-know algorithms, and canonical examples.
meindnoch 4y ago
>So most interview questions like "make a binary tree" are dead. "Make a binary tree and explain it on this whiteboard"
- enraged_camel 4y ago
  ChatGPT also has decent explanations tbh.
  - saghm 4y ago
    How is it at writing physically on a whiteboard?
    - CharlesW 4y ago
      It currently requires a human for that.
      - gptgpp 4y ago
        I hear that it'll be an app you can run on your neuralink. I'm very excited to beta test it. Should be any day now just like full self driving.
    - shthed 4y ago
      If you ask it to write it on an ascii art whiteboard it makes a pretty good attempt
      - saghm 4y ago
        Higher up in the thread, the question was about how to ask an interview question that could both test someone's skill and allow you to detect whether something is a bot or not. If I ask someone to code and explain something on a whiteboard and they give me ascii art, I think it did the job of identifying whether they were a bot.
  - MarcoZavala 4y ago
epicureanideal 4y ago
> "What's the weirdest bug you debugged? What made it weird?" If you can remember, 5-10 years after you solved it.
- hiq 4y ago
  It doesn't have to be true though, it just has to be a nice story. And this is the kind of interview questions you can prepare.
  - epicureanideal 4y ago
    > It doesn't have to be true though ... This is exactly why it's a bad question.
- Swizec 4y ago
  Can’t have been that good a bug, if you can’t. Been close to 15 years since some of the horror stories I can tell. Hell, over 20 years for some of my favorite lessons. edit: Point is the good shit stays. You’ll remember the best stories, fear not.
  - hgsgm 4y ago
    When you get older, other memories crowd out specific bugs. Memory is associative. The father you are from the bug, the harder it is to remember. More importantly, you don't need to remember your own bugs, you can get a story from the Internet or ChatGPT.
- BoorishBears 4y ago
  I would 100% accept that as an answer since it tells me the candidate is not good with subtext, and that's really needed to solve a lot of interesting problems. After all, if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating. I am not in fact asking you to recall every single food you've ever eaten and choose one... - From my comment below since every reply seems to be bent on ignoring the subtext even in a theoretical discussion about picking up subtext...: Again, the subtext is "interesting example we're going to discuss". If there's one you can't discuss for any reason (doesn't even have to be you forgot: could be an NDA) then it's already excused from the discussion. An even half-decent interview is not adversarial: just like day to day work, it requires interpreting some level of useful subtext and some level of open communication - I mean, you forgot the details so it's not like it's not like you're going to start monologuing if you just touch on it: "You know, there's a real doozy from X years ago where Y but the details escape me, more recently Z happened" If there are none that you remember that are interesting: "There aren't many interesting bugs, but there was this really interesting product requirement, could we go over that?"
canadianfella 4y ago
[dead]
excerionsforte 4y ago
I'd be interested in seeing how we can incorporate AI into interviews. Example being entry level software engineers using AI to jam through a small project with tests in a limited time span. Lazy engineers won't check the work while others will use whatever AI generates as their draft and correct whatever bugs that are in it. I believe that we should be taking advantage of this productivity boost across the board.
netfortius 4y ago
> The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" @ChatGPT: Give me three sample answers to the following questions related to <insert your interview language> programming: "What`s the weirdest bug you debugged? What made it weird?"
- CamperBob2 4y ago
  "One time, I was working on a project where the UI kept freezing when a certain button was pressed. I spent hours debugging the code, but couldn't figure out what was causing the problem. Eventually, I discovered that the button's event handler was accidentally triggering an infinite loop that was consuming all the CPU resources, causing the UI to freeze. It was a weird bug because the symptoms were not immediately obvious and it took a lot of digging to uncover the root cause. I once spent an entire day trying to figure out why my code was behaving erratically when processing a certain data file. It turns out that the file had some hidden control characters in it that were causing my program to misinterpret the data. The bug was weird because I had never encountered a situation where hidden characters were causing issues before. One time, I was working on a codebase that had been written by multiple developers over a long period of time. I was debugging a particularly tricky issue where certain data was being lost between different parts of the system. After a lot of investigation, I discovered that one of the earlier developers had implemented a workaround for a different issue that involved storing data in a global variable that was being accidentally cleared by another part of the code. It was a weird bug because it was caused by a seemingly innocuous change that had been made months earlier and was difficult to trace back to its origin." Welp, we're boned
  - alex_sf 4y ago
    I mean the first and the third aren't great. So, 33% chance of being boned.
    - deleted 4y ago
      [deleted]
    - 4y ago
lumost 4y ago
is a hard bug the best interview question? When I do my job right 98% of the time is not debugging. This ratio changed dramatically over the course of my career.
- squeaky-clean 4y ago
  I agree it's the best question because it's a good topic to begin a bigger conversation about your programming experience. It's hard to fake and you can take the discussion in many directions. Simply asking for their weirdest bug, getting an answer, and going "ok good, let's move on" is a bad interview question. For example, you say the ratio of debugging:development changed dramatically over the course of your career. I would follow up and ask what key things you would attribute that to? Testing? Changing languages? Changing programming paradigms? Maybe it's simply that you have a wider knowledge of CS concepts and a stronger intuition for the correct way to model your logic. There's no right or wrong answers, just trying to see that you actually do have some opinions of your own.
- TheCleric 4y ago
  I find the ratio doesn't change, but the trickiness of the bugs gets more complicated as you solve harder problems and avoid previously unknown problems.
dinkumthinkum 4y ago
Why is itdead? An interview is live, apart from sone kind of sci-fi implant thing, ChatGPT won’t help you cheat your way through a live interview; in fact using ChatGPT on a take home interview test and failing to answer basic questions in an interview may reveal aspects of your character and competence.
Angostura 4y ago
"it involved a program that ran perfectly fine during testing but crashed whenever it was deployed to the production environment. After much investigation, it was found that the bug was caused by a combination of the production environment's security settings and the program's use of random number generation. The program used a random number generator to generate a unique ID for each user that logged in. However, the production environment had security settings that blocked certain types of random number generation for security reasons. This caused the program to crash whenever it tried to generate a unique ID for a user. The solution to this weird bug was to modify the program to use a different random number generator that was allowed by the production environment's security settings. The lesson learned from this bug is to always be aware of the production environment's settings and limitations and to thoroughly test software in the target environment before deploying it."
- taneq 4y ago
  Back in my first job we had a database which worked fine in testing, worked fine on the customer’s server… then started crashing a week later. Turns out their MS SQL Server install was configured for MM-DD-YYYY output which only crashed when we reached the 13th of the month. Important lessons were learned that day!
permo-w 4y ago
typically how much time might you get to answer a question like that?
techie128 4y ago
no, in-person whiteboard / laptop based interviews can still ask these questions when the interviewer is shadowing you IDE.
grogenaut 4y ago
One of the smartest I ever felt was hearing some teammates a cube wall over saying "this one customer from poland says he's crashing a lot" (in our ui). I was like "is he in the polish locale/language? They asked.... yes... "I bet polish has different seperators and our split("/")[1] is crashing"... they checked, had him change to english, crash went away. I felt real yoda like. We had customer on chat so much easier to debug.
- hgsgm 4y ago
  Debugging is easy when you know you are coding incorrectly. But why would Polish have a different path separator? That's a filesystem choice not a language choice.
  - lmm 4y ago
    Windows uses 0x5C as the path separator, and the character rendering of that codepoint is locale-dependent (most famously in Japanese encodings it's ¥). Apparently in most Polish encodings it's still \, although there's one obscure one where it becomes Ś.
  - grogenaut 4y ago
    This was actually on date/time I believe... we had the date in the path
thefourthchime 4y ago
God I hope so, the last time I went job hunting I spend a couple of months doing the leet coding stuff, and I got "pretty good", but I'm certain it would have taken me another couple of months to pass the gauntlet somewhere.
sanderjd 4y ago
In my last round of interviews, by far my favorite session was "here is a real project with a real bug, debug it, then if you have time, describe how to fix it, then if you have time, write the fix". (At Stripe FWIW.) It was super representative of what my work is actually like, and what I'm good at, and ChatGPT would not have figured it out. yet
- grugagag 4y ago
  That sounds very fair. I’d prefer that over leetcode type of questions anytime
  - sanderjd 4y ago
    Yeah! I didn't end up taking a job at Stripe, but I came away very impressed with their process. Only one interview was leetcode-ish and it was a phone screen and purposefully not a high bar. Then the real interviews were: that debugging one which I loved, an "integration" one which involved live coding while plugging a few different unfamiliar libraries and APIs together (also something I really do in my job), a standard system design interview (still don't really love these...), and an in depth conversation with a hiring manager. Still a tough day, but pretty decent process I think!
- tacocataco 4y ago
  Did they pay you for doing the interview?
  - sanderjd 4y ago
    No of course not. Do you mean this debugging interview in particular, because it could potentially be useful work fixing a real bug for them? To be clear, it isn't an extant bug. I think the way they do it is they take an interesting (but fairly straightforward) real bug that they fixed at some point, and they back out the fix.
matwood 4y ago
> So most interview questions like "make a binary tree" are dead. By this logic, those questions were already killed by Google Search.
angarg12 4y ago
I've been quite outspoken about the impact that ChatGPT will have in the tech interview. My solution is not subtle: I believe we should burn the leetcode-style coding interview to the ground and replace it with something else entirely, possibly embracing LLMs. I'm in the vast minority though. My interviewer colleagues things this doesn't matter and we should keep doing business as usual. I guess we'll find out how it works out in the next couple of years.
darzu 4y ago
Personally I hate this question. And I get it in every interview. Somehow my brain just doesn't care to remember the gritty details for tough bugs. I'll spend days on a bug but soon after I solve it I'll only remember the actionable takeaways like "next time I should try using X tool sooner" or "check your assumptions on Y part of the stack". I think it's because I never spend any time revisiting that bug in my mind. I've got new problems to solve. You need to revisit something to remember it.

bcantrill 4y ago

Yes, agreed! We ask a variant of this question (we call it "an analysis sample"); from the materials we ask candidates to submit[0]: "A significant challenge of engineering is dealing with a system when it doesn't, in fact, work correctly. When systems misbehave, engineers must flip their disposition: instead of a creator of their own heaven and earth, they must become a scientist, attempting to reason about a foreign world. Please provide an analysis sample: a written analysis of system misbehavior from some point in your career. If such an analysis is not readily available (as it might not be if one’s work has been strictly proprietary), please recount an incident in which you analyzed system misbehavior, including as much technical detail as you can recall." These samples are very revealing -- and it feels unlikely that generative AI is going to be of much help, even assuming a fabulist candidate. (And of very little assistance on our values-based questions like "when have you been happiest in your professional career and why?"). [0] https://docs.google.com/document/d/1Xtofg-fMQfZoq8Y3oSAKjEgDQCRHx-GMSmPcxdEea4A/edit?usp=sharing https://docs.google.com/document/d/1Xtofg-fMQfZoq8Y3oSAKjEgD...

epicureanideal 4y ago
To be honest, this sounds extremely difficult and not in a good way. That sounds like many many hours of writing work, to describe a problem that might be many years in the past, that might have been solved by extremely intricate methods that are easy to forget, using technologies that are now not commonly in use, etc. A good question to ask about each interview question might be: would a good liar have an easier time answering this than a person trying to answer honestly? And if so, retire the question.
- bcantrill 4y ago
  Having read many, many, many answers to this question, I don't think that a good liar has a particularly easy time answering this question -- or certainly not in a way that gets them further consideration! And yes, it's many hours of work -- but the work itself that we are doing is quite hard, and if someone washes out in the application process because it feels unduly arduous, we are likely not a fit for one another.
  - l33t233372 4y ago
    > I don't think that a good liar has a particularly easy time answering this question -- or certainly not in a way that gets them further consideration How would you know? > And yes, it's many hours of work -- but the work itself that we are doing is quite hard, and if someone washes out in the application process because it feels unduly arduous, we are likely not a fit for one another. I sincerely hope that I never accidentally apply for a company that thinks an unpaid, long form writing prompt is an appropriate interview question because the work happens to be hard.
  - xvector 4y ago
    Eh, it also excludes people that don't have spectacular long-term memory, or people that don't keep a diary about bugs that they've chased down at work. Personally, I think you're overfitting to fight cheating, but maybe you work at a desirable enough place that you can afford to exclude so many people but still get enough good candidates. IMO a good question provides the necessary context itself, and the candidate's thinking and reasoning skills are what's tested. With your question, it's basically turned into a competition of which candidate has tackled the most ridiculous/obscure/complex bug, so candidates aren't being judged on even footing.
    - mypalmike 4y ago
      Agreed completely. This problem borders on that common category of questions which test whether the applicant shares a specific fine-grained flavor of nerdiness as the interviewer, rather than whether the candidate is a good fit for the job.

razor_router 4y ago

That's interesting! What did you learn from debugging that weird bug?

aeyes 4y ago
I learned that off-by one errors, mixing up arguments and caches are hard to debug. I have easily spent days debugging many such problems which were almost always solved by a one line change. And rarely did I find ways to prevent similar bugs in the future bugs by improving testing or code factoring.

BoorishBears 4y ago

While I'm not complaining that people are realizing they're dead... why is ChatGPT the final straw for those ridiculous "make a binary tree" questions? Why wasn't it the fact that these questions became such a gameable system, that we started referring to them by the copyrighted name of a site where you can access nearly every permutation that will ever be asked of you, along with extremely detailed solutions with rationale: https://leetcode.com/ https://leetcode.com/ It's crazy to me that of everything that ChatGPT can do, regurgitating well known answers to well known interview questions is what kills anything off...

dinkumthinkum 4y ago
I mean if you can’t make a binary tree, what do you really know about programming? If ChatGPT is so great, why would even need a job interview, why not just eliminate the jobs altogether?
- BoorishBears 4y ago
  You know, the only thing I get more sure of over the years is that defining a "good programmer" is hard Now, I can't tell if you're being facetious or not, but if one seriously conflates being able to write a binary tree with knowing how to program... they're at least making defining the antonym a bit easier. - Also no one is saying ChatGPT is "so great" in this specific case, if anything the point is that ChatGPT can do impressive things, but again, regurgitating leetcode answers isn't one of them.
  - dinkumthinkum 4y ago
    I’m not trying to be mean but “conflating” is not the same as “necessary.” There is a very common notion of “necessary but not sufficient”. Yes, I think if you don’t know how to make a binary tree, you don’t know much about programming. The converse is not true.
- psyklic 4y ago
  I view general algorithm problems as a "programming subfield", just like graphics/frontend/AI are. Certain coding techniques are overrepresented in each subfield, and each subfield has fundamental "base code", design paradigms, methods of testing, need-to-know algorithms, and canonical examples.

meindnoch 4y ago

>So most interview questions like "make a binary tree" are dead. "Make a binary tree and explain it on this whiteboard"

enraged_camel 4y ago
ChatGPT also has decent explanations tbh.
- saghm 4y ago
  How is it at writing physically on a whiteboard?
  - CharlesW 4y ago
    It currently requires a human for that.
    - gptgpp 4y ago
      I hear that it'll be an app you can run on your neuralink. I'm very excited to beta test it. Should be any day now just like full self driving.
  - shthed 4y ago
    If you ask it to write it on an ascii art whiteboard it makes a pretty good attempt
    - saghm 4y ago
      Higher up in the thread, the question was about how to ask an interview question that could both test someone's skill and allow you to detect whether something is a bot or not. If I ask someone to code and explain something on a whiteboard and they give me ascii art, I think it did the job of identifying whether they were a bot.
- MarcoZavala 4y ago

epicureanideal 4y ago

> "What's the weirdest bug you debugged? What made it weird?" If you can remember, 5-10 years after you solved it.

hiq 4y ago
It doesn't have to be true though, it just has to be a nice story. And this is the kind of interview questions you can prepare.
- epicureanideal 4y ago
  > It doesn't have to be true though ... This is exactly why it's a bad question.
Swizec 4y ago
Can’t have been that good a bug, if you can’t. Been close to 15 years since some of the horror stories I can tell. Hell, over 20 years for some of my favorite lessons. edit: Point is the good shit stays. You’ll remember the best stories, fear not.
- hgsgm 4y ago
  When you get older, other memories crowd out specific bugs. Memory is associative. The father you are from the bug, the harder it is to remember. More importantly, you don't need to remember your own bugs, you can get a story from the Internet or ChatGPT.
BoorishBears 4y ago
I would 100% accept that as an answer since it tells me the candidate is not good with subtext, and that's really needed to solve a lot of interesting problems. After all, if I ask "what's your favorite food you've ever eaten", there's an unspoken implication that it's a food you remember eating. I am not in fact asking you to recall every single food you've ever eaten and choose one... - From my comment below since every reply seems to be bent on ignoring the subtext even in a theoretical discussion about picking up subtext...: Again, the subtext is "interesting example we're going to discuss". If there's one you can't discuss for any reason (doesn't even have to be you forgot: could be an NDA) then it's already excused from the discussion. An even half-decent interview is not adversarial: just like day to day work, it requires interpreting some level of useful subtext and some level of open communication - I mean, you forgot the details so it's not like it's not like you're going to start monologuing if you just touch on it: "You know, there's a real doozy from X years ago where Y but the details escape me, more recently Z happened" If there are none that you remember that are interesting: "There aren't many interesting bugs, but there was this really interesting product requirement, could we go over that?"

canadianfella 4y ago

[dead]

excerionsforte 4y ago

I'd be interested in seeing how we can incorporate AI into interviews. Example being entry level software engineers using AI to jam through a small project with tests in a limited time span. Lazy engineers won't check the work while others will use whatever AI generates as their draft and correct whatever bugs that are in it. I believe that we should be taking advantage of this productivity boost across the board.

netfortius 4y ago

> The best interview question that will never die: "What's the weirdest bug you debugged? What made it weird?" @ChatGPT: Give me three sample answers to the following questions related to <insert your interview language> programming: "What`s the weirdest bug you debugged? What made it weird?"

CamperBob2 4y ago
"One time, I was working on a project where the UI kept freezing when a certain button was pressed. I spent hours debugging the code, but couldn't figure out what was causing the problem. Eventually, I discovered that the button's event handler was accidentally triggering an infinite loop that was consuming all the CPU resources, causing the UI to freeze. It was a weird bug because the symptoms were not immediately obvious and it took a lot of digging to uncover the root cause. I once spent an entire day trying to figure out why my code was behaving erratically when processing a certain data file. It turns out that the file had some hidden control characters in it that were causing my program to misinterpret the data. The bug was weird because I had never encountered a situation where hidden characters were causing issues before. One time, I was working on a codebase that had been written by multiple developers over a long period of time. I was debugging a particularly tricky issue where certain data was being lost between different parts of the system. After a lot of investigation, I discovered that one of the earlier developers had implemented a workaround for a different issue that involved storing data in a global variable that was being accidentally cleared by another part of the code. It was a weird bug because it was caused by a seemingly innocuous change that had been made months earlier and was difficult to trace back to its origin." Welp, we're boned
- alex_sf 4y ago
  I mean the first and the third aren't great. So, 33% chance of being boned.
  - deleted 4y ago
    [deleted]
  - 4y ago

lumost 4y ago

is a hard bug the best interview question? When I do my job right 98% of the time is not debugging. This ratio changed dramatically over the course of my career.

squeaky-clean 4y ago
I agree it's the best question because it's a good topic to begin a bigger conversation about your programming experience. It's hard to fake and you can take the discussion in many directions. Simply asking for their weirdest bug, getting an answer, and going "ok good, let's move on" is a bad interview question. For example, you say the ratio of debugging:development changed dramatically over the course of your career. I would follow up and ask what key things you would attribute that to? Testing? Changing languages? Changing programming paradigms? Maybe it's simply that you have a wider knowledge of CS concepts and a stronger intuition for the correct way to model your logic. There's no right or wrong answers, just trying to see that you actually do have some opinions of your own.
TheCleric 4y ago
I find the ratio doesn't change, but the trickiness of the bugs gets more complicated as you solve harder problems and avoid previously unknown problems.

dinkumthinkum 4y ago

Why is itdead? An interview is live, apart from sone kind of sci-fi implant thing, ChatGPT won’t help you cheat your way through a live interview; in fact using ChatGPT on a take home interview test and failing to answer basic questions in an interview may reveal aspects of your character and competence.

Angostura 4y ago

"it involved a program that ran perfectly fine during testing but crashed whenever it was deployed to the production environment. After much investigation, it was found that the bug was caused by a combination of the production environment's security settings and the program's use of random number generation. The program used a random number generator to generate a unique ID for each user that logged in. However, the production environment had security settings that blocked certain types of random number generation for security reasons. This caused the program to crash whenever it tried to generate a unique ID for a user. The solution to this weird bug was to modify the program to use a different random number generator that was allowed by the production environment's security settings. The lesson learned from this bug is to always be aware of the production environment's settings and limitations and to thoroughly test software in the target environment before deploying it."

taneq 4y ago
Back in my first job we had a database which worked fine in testing, worked fine on the customer’s server… then started crashing a week later. Turns out their MS SQL Server install was configured for MM-DD-YYYY output which only crashed when we reached the 13th of the month. Important lessons were learned that day!

permo-w 4y ago

typically how much time might you get to answer a question like that?

techie128 4y ago

no, in-person whiteboard / laptop based interviews can still ask these questions when the interviewer is shadowing you IDE.

grogenaut 4y ago

One of the smartest I ever felt was hearing some teammates a cube wall over saying "this one customer from poland says he's crashing a lot" (in our ui). I was like "is he in the polish locale/language? They asked.... yes... "I bet polish has different seperators and our split("/")[1] is crashing"... they checked, had him change to english, crash went away. I felt real yoda like. We had customer on chat so much easier to debug.

hgsgm 4y ago
Debugging is easy when you know you are coding incorrectly. But why would Polish have a different path separator? That's a filesystem choice not a language choice.
- lmm 4y ago
  Windows uses 0x5C as the path separator, and the character rendering of that codepoint is locale-dependent (most famously in Japanese encodings it's ¥). Apparently in most Polish encodings it's still \, although there's one obscure one where it becomes Ś.
- grogenaut 4y ago
  This was actually on date/time I believe... we had the date in the path

thefourthchime 4y ago

God I hope so, the last time I went job hunting I spend a couple of months doing the leet coding stuff, and I got "pretty good", but I'm certain it would have taken me another couple of months to pass the gauntlet somewhere.

sanderjd 4y ago

In my last round of interviews, by far my favorite session was "here is a real project with a real bug, debug it, then if you have time, describe how to fix it, then if you have time, write the fix". (At Stripe FWIW.) It was super representative of what my work is actually like, and what I'm good at, and ChatGPT would not have figured it out. yet

grugagag 4y ago
That sounds very fair. I’d prefer that over leetcode type of questions anytime
- sanderjd 4y ago
  Yeah! I didn't end up taking a job at Stripe, but I came away very impressed with their process. Only one interview was leetcode-ish and it was a phone screen and purposefully not a high bar. Then the real interviews were: that debugging one which I loved, an "integration" one which involved live coding while plugging a few different unfamiliar libraries and APIs together (also something I really do in my job), a standard system design interview (still don't really love these...), and an in depth conversation with a hiring manager. Still a tough day, but pretty decent process I think!
tacocataco 4y ago
Did they pay you for doing the interview?
- sanderjd 4y ago
  No of course not. Do you mean this debugging interview in particular, because it could potentially be useful work fixing a real bug for them? To be clear, it isn't an extant bug. I think the way they do it is they take an interesting (but fairly straightforward) real bug that they fixed at some point, and they back out the fix.

matwood 4y ago

> So most interview questions like "make a binary tree" are dead. By this logic, those questions were already killed by Google Search.

angarg12 4y ago

I've been quite outspoken about the impact that ChatGPT will have in the tech interview. My solution is not subtle: I believe we should burn the leetcode-style coding interview to the ground and replace it with something else entirely, possibly embracing LLMs. I'm in the vast minority though. My interviewer colleagues things this doesn't matter and we should keep doing business as usual. I guess we'll find out how it works out in the next couple of years.

darzu 4y ago

Personally I hate this question. And I get it in every interview. Somehow my brain just doesn't care to remember the gritty details for tough bugs. I'll spend days on a bug but soon after I solve it I'll only remember the actionable takeaways like "next time I should try using X tool sooner" or "check your assumptions on Y part of the stack". I think it's because I never spend any time revisiting that bug in my mind. I've got new problems to solve. You need to revisit something to remember it.