5 ms·

Because juniors don't know when they're being taken down a rabbit hole. So they'll let the LLM go too deep in its hallucinations. I have a Jr that was supposed

by kaydub 1y ago

Because juniors don't know when they're being taken down a rabbit hole. So they'll let the LLM go too deep in its hallucinations. I have a Jr that was supposed to deploy a terraform module I built. This task has been hanging out for a while so I went to check in on them. They told me the problem they're having and asked me to take a look. Their repo is a disaster, it's very obvious claude took them down a rabbit hole just from looking. When I asked, "Hey, why is all this python in here? The module has it self contained" and they respond with "I don't know, claude did that" affirming my assumptions. They lack the experience and they're overly reliant on the LLM tools. Not just in the design and implementation phases but also for troubleshooting. And if you're troubleshooting with something that's hallucinating and you don't know enough to know it's hallucinating you're in for a long ride. Meanwhile the LLM tools have taken away a lot of the type of work I hated doing. I can quickly tell when the LLM is going down a rabbit hole (in most cases at least) and prevent it from continuing. It's kinda re-lit my passion for coding and building software. So that's ended up in me producing more and giving better results.

shaky-carrousel 1y ago
Unfortunately, the type of work you hate doing is perfect for a junior. Easy tasks to get a hold on the system.
- reactordev 1y ago
  Yup, sounds like a great opportunity to show you’re senior by mentoring.
  - bayarearefugee 1y ago
    Which will (sadly) offer you zero extrinsic benefit at almost every job, and will often actually count against you as a waste of time relative to the vast majority of productivity metrics that companies actually use.
    - mystifyingpoi 1y ago
      Which is unfortunately very true. I think, that in a healthy organization such kind of mentoring requires extremely well defined boundaries and rules. Can I spend 1h of my time explaining basic stuff to a junior, who will then be able to finish his task in 2h instead of 8h? Mathematically this is a win for the company. But economically, maybe that junior dev should be fired.
      - xp84 1y ago
        That's the problem -- except in the rare type of org that thinks past quarterly or annual results and thus values training up a pipeline to be the future seniors -- economically speaking, all the junior devs should be fired, and the simple tasks they can do are the same set of tasks that can be accomplished by a senior at the helm of AI at 10-50x the speed.
    - godelski 1y ago
      There's a lot of benefits you get by mentoring. When you have to teach you're forced to think more deeply about problems, see it from different angles, and revisit questions you had when learning but tabled at that time. That last one is pretty common. There's always questions you have when learning that you have to avoid because you don't have the knowledge to answer yet, because they would take you down too deep a rabbit hole. But later you have the experience to navigate those problems. Personally, I've learned a ton while teaching. It's given me many new ideas and insights. But I'm obviously not alone in this. Even Feynman talks about how teaching results in better learning
      - thirdsun 1y ago
        I don't think you have to convince the parent poster or seniors developers in general. You have to convince their employers that this is a valuable use of time.
        godelski 1y ago
        I interpreted the extrinsic benefits differently, but fair. Still, I think being vocal about this helps. We can put pressure on our managers
  - aeternum 1y ago
    Mentorship is unfortunately broken because gone are the days of apprenticeships. If you find a great mentor, do everything you can to learn quickly then jump ship to big tech and cash in.
    - qnleigh 1y ago
      This is kind of sad. I'm probably missing context here, but surely it's not necessary to rush out of a good working relationship with a good mentor. The most important things you'll learn from a good mentor can't be learned in a rush (like prioritization, project management, how to be a good mentor...). In any case, I would replace 'jump ship' with 'pay it forward.'
    - ljm 1y ago
      A great mentor won’t waste their time on someone cynically using their time to cash out at Meta. Such a person can just get a CS degree and launch into Amazon for that. Big Tech are just the IT enterprises of the modern day. It’s not where the action is and that experience is not so hot on the CV when it comes to early stage development.
    - pjmlp 1y ago
      Depends on the country, they are pretty much alive across many European countries. Here are two examples, A German site to search for technical schools with apprenticeship, https://www.ausbildung.de/ https://www.ausbildung.de/ The same for Portugal https://www.escolasprofissionais.com/ https://www.escolasprofissionais.com/
    - pydry 1y ago
      There's also not much in it for the mentor other than the warm glow in the heart.
    - alexashka 1y ago
      There's unfortunately a crack epidemic. Try to quickly become a drug dealer and cash in. --- Chef's kiss. I'd like to thank people like you for where we are.
    - reactordev 1y ago
  - TeMPOraL 1y ago
    Mentoring is a different kind of work than coding. Like senior+ level in some companies, it's just faux management - people work, with all the expectations and responsibilities of the non-technical track, but none of the status and privileges.
- morkalork 1y ago
  >How'd you get so good at debugging and navigating code you've never seen before? >Because I spent a couple internships and a whole year as a junior debugging, triaging and patching every single issue reported by other developers and the QA team Was I jealous that the full time and senior devs got to do all the feature work and architecture design? Yes. Am I a better developer having done that grind? Also yes.
  - _heimdall 1y ago
    100%. I shouldn't have done the architecture work on any meaningful scale early on, but I was always interested. My early roles were largely me being able to architect smaller pieces in a larger plan, building some architecture skills while learning larger patterns that worked and thought processes along the way. I'd recommend that approach to anyone working with more junior devs.
deleted 1y ago
[deleted]
dotnet00 1y ago
The deciding factor for being able to effectively utilize LLMs and dodge hallucinations is ability to read code and intuition for how a solution should look. I think juniors are especially hesitant to just dig into understanding some source code unless they have no other choice, e.g. preferring to wait on email response from the author of the code over piecing things together. This makes LLM tools so tempting, you don't even have to wait on the email response anymore! But of course, this is basically going in blind, and it's no wonder that they end up in hallucination mazes.
kromokromo 1y ago
I think a lot of the problems lies in their prompting. AI is usually at its worst when just saying «deploy terraform module». And off it goes spitting out code. What they should have done as juniors was to have a conversation about the topic and task first. «Help me understand …» learning and planning is especially important with LLM coding.
- SubiculumCode 1y ago
  I don't write terribly complex things: Just data processing pipelines for neuroimaging, but I know I get good results because I take time being specific in my prompts, saying what I want, but also what I don't want, suggesting certain tools, what options I want available, how I want logs, etc. I really does help it to know what you want and relaying that with an effortful prompt.
victor9000 1y ago
> I don't know, claude did that I'm the type of reviewer that actually reads code and asks probing questions, and I've heard this from junior and senior devs alike. It's maddening how people say this with a straight face and expect to keep their jobs. If people are pushing code they don't understand, they're liability to their team, product, and employer.
- boringg 1y ago
  It is going to happen with higher frequency - buckle up!
- deleted 1y ago
  [deleted]
- qazxcvbnmlp 1y ago
  "I don't know Claude did that" isn't a bad thing in and of itself... If someone is spending a bunch of time on code that Claude could have done and easily verified it was correct, they are going to move slower and produce less useful things of value than someone who cares about reading every line of code.
  - torben-friis 1y ago
    Any situation where you’re submitting under your signature code to production without knowing what it does should be at the very least cause for a talk. I’m kinda shocked that this even has to be said.
    - deleted 1y ago
      [deleted]
  - ponector 1y ago
    If you don't understand you code how you can be sure it's correct? You actually are pushing it into your colleagues who will verify and fix the code later.
catlifeonmars 1y ago
> I don't know, claude did that also likes to blame shop accidents on the table saw
aunty_helen 1y ago
It’s like having a malicious mentor. But the frequency of which I’m bailing on reviews on the first line due to stupid stuff that has made it to a commit is quite stunning. “Oh I thought that would be useful at some point so I just committed it.” Beating it into developers that they need to review their own work they submit before asking someone else to waste time is the best way I’ve found so far.
boredatoms 1y ago
If they dont know what every line does, it shouldn’t be ready for review - regardless of if they or AI wrote it initially
userbinator 1y ago
The problem with beginners going down the wrong path has always been there, but AI lets them go much further than before.
jongjong 1y ago
That makes sense. When Claude Code suggests a bad approach, I kind of shrug it off and suggest a different approach. I don't think of it like a negative because I basically go through a similar process of ideation before I implement a feature and I typically discard ideas before arriving at the right one so Claude Code is part of the ideation process for me. The code it produces gives me a clear idea about how complex/ugly the solution will be. I know what 'ugly' looks like. I imagine a junior might actually jump at the first solution because they don't have any other arrows in their quiver so-to-speak. The LLM is acting like the engineering manager but it's actually really bad at that. The LLM is like a stereotypical programmer with 0 common sense. The kind who can produce good code but you have to micromanage every single decision. It's terrible if you put it in charge. It doesn't have any opinions about architecture or code quality so it just follows the structure of the existing code base... There's a tendency towards increasing complexity, more hacks more chaos. It often tries to hack/cheat its way to a solution. It requires a constant effort to maintain order.
smsm42 1y ago
> they respond with "I don't know, claude did that" A huge red flag right here. Its ok to not know things, its ok to use LLM to fill the gaps. Its ok to fail. It is in no way ok to not care and only fess up to having a mass of code you have no idea about when your senior reviewer asks you about it. At the very minimum the ask should go the other direction - "I got this generated code and I am not sure I understand what's going on, could you help me to see if that's the right direction" would be acceptable. Just not caring is not.
sharperguy 1y ago
Honestly I made similar mistakes back before AI as a junior developer, by just copy/pasting code or confusing two things.
protocolture 1y ago
>I don't know, claude did that Instant dismissal.
dingi 1y ago
No amount of “own the code you generate” policies will prevent the rise of “I don’t know, the AI wrote it” excuses in the LLM era. What’s more likely is that reviewers will be flooded with unvetted, generated code. Over time, this can lead to reviewer fatigue and a gradual decline in rigor. If the trend continues, the impact could be significant once the most experienced engineers begin to retire.

by kaydub 1y ago

shaky-carrousel 1y ago

Unfortunately, the type of work you hate doing is perfect for a junior. Easy tasks to get a hold on the system.

reactordev 1y ago
Yup, sounds like a great opportunity to show you’re senior by mentoring.
- bayarearefugee 1y ago
  Which will (sadly) offer you zero extrinsic benefit at almost every job, and will often actually count against you as a waste of time relative to the vast majority of productivity metrics that companies actually use.
  - mystifyingpoi 1y ago
    Which is unfortunately very true. I think, that in a healthy organization such kind of mentoring requires extremely well defined boundaries and rules. Can I spend 1h of my time explaining basic stuff to a junior, who will then be able to finish his task in 2h instead of 8h? Mathematically this is a win for the company. But economically, maybe that junior dev should be fired.
    - xp84 1y ago
      That's the problem -- except in the rare type of org that thinks past quarterly or annual results and thus values training up a pipeline to be the future seniors -- economically speaking, all the junior devs should be fired, and the simple tasks they can do are the same set of tasks that can be accomplished by a senior at the helm of AI at 10-50x the speed.
  - godelski 1y ago
    There's a lot of benefits you get by mentoring. When you have to teach you're forced to think more deeply about problems, see it from different angles, and revisit questions you had when learning but tabled at that time. That last one is pretty common. There's always questions you have when learning that you have to avoid because you don't have the knowledge to answer yet, because they would take you down too deep a rabbit hole. But later you have the experience to navigate those problems. Personally, I've learned a ton while teaching. It's given me many new ideas and insights. But I'm obviously not alone in this. Even Feynman talks about how teaching results in better learning
    - thirdsun 1y ago
      I don't think you have to convince the parent poster or seniors developers in general. You have to convince their employers that this is a valuable use of time.
      - godelski 1y ago
        I interpreted the extrinsic benefits differently, but fair. Still, I think being vocal about this helps. We can put pressure on our managers
- aeternum 1y ago
  Mentorship is unfortunately broken because gone are the days of apprenticeships. If you find a great mentor, do everything you can to learn quickly then jump ship to big tech and cash in.
  - qnleigh 1y ago
    This is kind of sad. I'm probably missing context here, but surely it's not necessary to rush out of a good working relationship with a good mentor. The most important things you'll learn from a good mentor can't be learned in a rush (like prioritization, project management, how to be a good mentor...). In any case, I would replace 'jump ship' with 'pay it forward.'
  - ljm 1y ago
    A great mentor won’t waste their time on someone cynically using their time to cash out at Meta. Such a person can just get a CS degree and launch into Amazon for that. Big Tech are just the IT enterprises of the modern day. It’s not where the action is and that experience is not so hot on the CV when it comes to early stage development.
  - pjmlp 1y ago
    Depends on the country, they are pretty much alive across many European countries. Here are two examples, A German site to search for technical schools with apprenticeship, https://www.ausbildung.de/ https://www.ausbildung.de/ The same for Portugal https://www.escolasprofissionais.com/ https://www.escolasprofissionais.com/
  - pydry 1y ago
    There's also not much in it for the mentor other than the warm glow in the heart.
  - alexashka 1y ago
    There's unfortunately a crack epidemic. Try to quickly become a drug dealer and cash in. --- Chef's kiss. I'd like to thank people like you for where we are.
  - reactordev 1y ago
- TeMPOraL 1y ago
  Mentoring is a different kind of work than coding. Like senior+ level in some companies, it's just faux management - people work, with all the expectations and responsibilities of the non-technical track, but none of the status and privileges.
morkalork 1y ago
>How'd you get so good at debugging and navigating code you've never seen before? >Because I spent a couple internships and a whole year as a junior debugging, triaging and patching every single issue reported by other developers and the QA team Was I jealous that the full time and senior devs got to do all the feature work and architecture design? Yes. Am I a better developer having done that grind? Also yes.
- _heimdall 1y ago
  100%. I shouldn't have done the architecture work on any meaningful scale early on, but I was always interested. My early roles were largely me being able to architect smaller pieces in a larger plan, building some architecture skills while learning larger patterns that worked and thought processes along the way. I'd recommend that approach to anyone working with more junior devs.

deleted 1y ago

[deleted]

dotnet00 1y ago

The deciding factor for being able to effectively utilize LLMs and dodge hallucinations is ability to read code and intuition for how a solution should look. I think juniors are especially hesitant to just dig into understanding some source code unless they have no other choice, e.g. preferring to wait on email response from the author of the code over piecing things together. This makes LLM tools so tempting, you don't even have to wait on the email response anymore! But of course, this is basically going in blind, and it's no wonder that they end up in hallucination mazes.

kromokromo 1y ago

I think a lot of the problems lies in their prompting. AI is usually at its worst when just saying «deploy terraform module». And off it goes spitting out code. What they should have done as juniors was to have a conversation about the topic and task first. «Help me understand …» learning and planning is especially important with LLM coding.

SubiculumCode 1y ago
I don't write terribly complex things: Just data processing pipelines for neuroimaging, but I know I get good results because I take time being specific in my prompts, saying what I want, but also what I don't want, suggesting certain tools, what options I want available, how I want logs, etc. I really does help it to know what you want and relaying that with an effortful prompt.

victor9000 1y ago

> I don't know, claude did that I'm the type of reviewer that actually reads code and asks probing questions, and I've heard this from junior and senior devs alike. It's maddening how people say this with a straight face and expect to keep their jobs. If people are pushing code they don't understand, they're liability to their team, product, and employer.

boringg 1y ago
It is going to happen with higher frequency - buckle up!
deleted 1y ago
[deleted]
qazxcvbnmlp 1y ago
"I don't know Claude did that" isn't a bad thing in and of itself... If someone is spending a bunch of time on code that Claude could have done and easily verified it was correct, they are going to move slower and produce less useful things of value than someone who cares about reading every line of code.
- torben-friis 1y ago
  Any situation where you’re submitting under your signature code to production without knowing what it does should be at the very least cause for a talk. I’m kinda shocked that this even has to be said.
  - deleted 1y ago
    [deleted]
- ponector 1y ago
  If you don't understand you code how you can be sure it's correct? You actually are pushing it into your colleagues who will verify and fix the code later.

catlifeonmars 1y ago

> I don't know, claude did that also likes to blame shop accidents on the table saw

aunty_helen 1y ago

It’s like having a malicious mentor. But the frequency of which I’m bailing on reviews on the first line due to stupid stuff that has made it to a commit is quite stunning. “Oh I thought that would be useful at some point so I just committed it.” Beating it into developers that they need to review their own work they submit before asking someone else to waste time is the best way I’ve found so far.

boredatoms 1y ago

If they dont know what every line does, it shouldn’t be ready for review - regardless of if they or AI wrote it initially

userbinator 1y ago

The problem with beginners going down the wrong path has always been there, but AI lets them go much further than before.

jongjong 1y ago

That makes sense. When Claude Code suggests a bad approach, I kind of shrug it off and suggest a different approach. I don't think of it like a negative because I basically go through a similar process of ideation before I implement a feature and I typically discard ideas before arriving at the right one so Claude Code is part of the ideation process for me. The code it produces gives me a clear idea about how complex/ugly the solution will be. I know what 'ugly' looks like. I imagine a junior might actually jump at the first solution because they don't have any other arrows in their quiver so-to-speak. The LLM is acting like the engineering manager but it's actually really bad at that. The LLM is like a stereotypical programmer with 0 common sense. The kind who can produce good code but you have to micromanage every single decision. It's terrible if you put it in charge. It doesn't have any opinions about architecture or code quality so it just follows the structure of the existing code base... There's a tendency towards increasing complexity, more hacks more chaos. It often tries to hack/cheat its way to a solution. It requires a constant effort to maintain order.

smsm42 1y ago

> they respond with "I don't know, claude did that" A huge red flag right here. Its ok to not know things, its ok to use LLM to fill the gaps. Its ok to fail. It is in no way ok to not care and only fess up to having a mass of code you have no idea about when your senior reviewer asks you about it. At the very minimum the ask should go the other direction - "I got this generated code and I am not sure I understand what's going on, could you help me to see if that's the right direction" would be acceptable. Just not caring is not.

sharperguy 1y ago

Honestly I made similar mistakes back before AI as a junior developer, by just copy/pasting code or confusing two things.

protocolture 1y ago

>I don't know, claude did that Instant dismissal.

dingi 1y ago

No amount of “own the code you generate” policies will prevent the rise of “I don’t know, the AI wrote it” excuses in the LLM era. What’s more likely is that reviewers will be flooded with unvetted, generated code. Over time, this can lead to reviewer fatigue and a gradual decline in rigor. If the trend continues, the impact could be significant once the most experienced engineers begin to retire.