5 ms·

> there are some juniors who use AI to assist... and some who use it to delegate all of their work to. Hmmm. Is there any way to distinguish between these two

> there are some juniors who use AI to assist... and some who use it to delegate all of their work to. Hmmm. Is there any way to distinguish between these two categories? Because I agree, if someone is delegating all their work to an LLM or similar tool, cut out the middleman. Same as if someone just copy/pasted from Stackoverflow 5 years ago. I think it is also important to think about incentives. What incentive does the newer developer have to understand the LLM output? There's the long term incentive, but is there a short term one?

supriyo-biswas 1y ago
Dealing with an intern at work who I suspect is doing exactly this, I discussed this with a colleague. One way seems to be to organize a face to face meeting where you test their problem solving skills without AI use, the other may be to question them about their thought process as you review a PR. Unfortunately, the use of LLMs has brought about a lot of mistrust in the workplace. Earlier you’d simply assume that a junior making mistakes is simply part of being a junior and can be coached; whereas nowadays said junior may not be willing to take your advice as they see it as sermonizing when an “easy” process to get “acceptable” results exists.
- throwaway2037 1y ago
  > Earlier you’d simply assume that a junior making mistakes is simply part of being a junior and can be coached; whereas nowadays said junior may not be willing to take your advice Hot take: This reads like an old person looking down upon young people. Can you explain why it isn't? Else, this reads like: "When I was young, we worked hard and listened to our elders. These days, young people ignore our advice." Every time I see inter-generational commentary like this (which is inevitably from personal experience), I am immediately suspicious. I can assure you that when I was young, I did not listen to older people's advice and I tried to do everything my own way. Why would this be any different in the current generation? In my experience, it isn't. On a positive note: I can remember mentoring some young people and watching them comb through blogs to learn about programming. I am so old that my shelf is/was full of O'Reilly books. By the time I was mentoring them, few people under 25 were reading O'Reilly books. It opened my eyes that how people changes more than what people learn. Example: Someone is trying to learning about access control modifiers for classes/methods in a programming language. Old days: Get the O'Reilly book for that programming language. Lookup access modifiers in the index. 10 year ago: Google for a blog with an intro to the programming language. There will be a tip about what access modifiers can do. Today: Ask ChatGPT. In my (somewhat contrived) example, the how is changing, but not the what.
  - transfer92 1y ago
    > I can assure you that when I was young, I did not listen to older people's advice and I tried to do everything my own way. Hot take: This reads like a person who was difficult to work with. Senior people have responsibility, therefore in a business situation they have authority. Junior people who think they know it all don't like this. If there's a disagreement between a senior person and a junior person about something, they should, of course, listen to each other respectfully. If that's not happening, then one of them is not being a good employee. But if they are, then the supervisor makes the final call.
  - shagie 1y ago
    > Old days: Get the O'Reilly book for that programming language. Lookup access modifiers in the index. 10 year ago: Google for a blog with an intro to the programming language. There will be a tip about what access modifiers can do. Today: Ask ChatGPT. In my (somewhat contrived) example, the how is changing, but not the what. The tangent to that is it is also changing with the how much one internalizes about the problem domain and is able to apply that knowledge later. Hard fought knowledge from the old days is something that shapes how I design systems today. However, the tendency of people who reach for ChatGPT today to solve a problem results in them making the same mistakes again the next time since the information is so easy to access. It also results in things that are larger are more difficult... the "how do you architect this larger system" is something you learn by building the smaller systems and learning about them so that their advantages and disadvantages and how and such becomes an inherent part of how you conceive of the system as a whole. ... Being able to have ChatGPT do it means people often don't think about the larger problem or how it fits together. I believe that is harder for a junior who is using ChatGPT to advance to being a mid level or senior developer than it is for a junior from the old days because of the lack of retention of the knowledge of the problems and solutions.
    - moosedev 1y ago
      They’re going to get promoted anyway. The “senior” title will simply (continue to) lose meaning to inflation.
  - 1y ago
- chairmansteve 1y ago
  The intern is not producing code that is up to the standard you expect, and will not change it? I saw a situation like this many years ago. The newly hired midlevel engineer thought he was smarter than the supervisor. Kept on arguing about code style, system design etc. He was fired after 6 months. But I was friendly with him, so we kept in touch. He ended up working at MSFT for 3 times the salary.
lll-o-lll 1y ago
> Is there any way to distinguish between these two categories? Yes, it should be obvious. At least at the current state of LLMs. > There's the long term incentive, but is there a short term one? The short term incentive is keeping their job.
hombre_fatal 1y ago
Just like anything, anyone who did the work themself should be able to speak intelligently about the work and the decisions behind its idiosyncrasies. For software, I can imagine a process where junior developers create a PR and then run through it with another engineer side by side. The short-term incentive would be that they can do it, else they'd get exposed.
icedchai 1y ago
There are some definite signs of over reliance on AI. From emojis in comments, to updates completely unrelated to the task at hand, if you ask "why did you make this change?", you'll typically get no answer. I don't mind if AI is used as a tool, but the output needs to be vetted.
- wwweston 1y ago
  Exactly. Use LLMs as a tutor, a tool, and make sure you understand the output.
  - agumonkey 1y ago
    My favorite prompt is "your goal is to retire yourself"
- throwaway2037 1y ago
  What is wrong with emojis in comments? I see no issue with it. Do I do it myself? No. Would I pushback if a young person added emojis to comments? No. I am looking at "the content, not the colour".
  - chihuahua 1y ago
    I think GP may be thinking that emojis in PR comments (plus the other red flags they mentioned) are the result of copy/paste from LLM output, which might imply that the person who does mindless copy/pasting is not adding anything and could be replaced by LLM automation.
  - venturecruelty 1y ago
    The point is that heavy emoji use means AI was likely used to produce a changeset, not that emojis are inherently bad.
  - icedchai 1y ago
    The emojis are not a problem themselves. They're a warning sign: slop is (probably) present, look deeper.
bryanrasmussen 1y ago
having dealt with a few people who just copy/pasted Stackoverflow I really feel that using an LLM is an improvement. That is at least for the people who don't understand what they're doing, the LLM tends to come out with something I can at least turn into something useful. It might be reversed though for people who know what they're doing. IF they know what they're doing they might theoretically be able to put together some stackoverflow results that make sense, and build something up from that better than what gets generated from LLM (I am not asserting this would happen, and thinking it might be the case) However I don't know as I've never known anyone who knew what they were doing who also just copy/pasted some stackoverflow or delegated to LLM significantly.
water-data-dude 1y ago
Is/was copy/pasting from Stackoverflow considered harmful? You have a problem, you do a web search and you find someone who asked the same question on SO, and there's often a solution. You might be specifically talking about people who copy/paste without understanding, but I think it's still OK-ish to do that, since you can't make an entire [whatever you're coding up] by copy/pasting snippets from SO like you're cutting words out of a magazine for a ransom note. There's still thought involved, so it's more like training wheels that you eventually outgrow as you get more understanding.
- vkou 1y ago
  > Is/was copy/pasting from Stackoverflow considered harmful? It at least forces you to tinker with whatever you copied over.
gunsch 1y ago
Pair programming! Get hands-on with your junior engineers and their development process. Push them to think through things and not just ask the LLM everything.
- johnnyanmac 1y ago
  I've seen some overly excessive pair programming initiatives out there, but it does baffle me why less people who struggle with this do it. Take even just 30 minutes to pair program on a problem and see their process and you can reveal so much. But I suppose my question is rhetorical. We're laying off hundreds of thousands of engineers and maming existing ones do the work of 3-4 engineers. Not much time to help the juniors.

supriyo-biswas 1y ago

Dealing with an intern at work who I suspect is doing exactly this, I discussed this with a colleague. One way seems to be to organize a face to face meeting where you test their problem solving skills without AI use, the other may be to question them about their thought process as you review a PR. Unfortunately, the use of LLMs has brought about a lot of mistrust in the workplace. Earlier you’d simply assume that a junior making mistakes is simply part of being a junior and can be coached; whereas nowadays said junior may not be willing to take your advice as they see it as sermonizing when an “easy” process to get “acceptable” results exists.

throwaway2037 1y ago
> Earlier you’d simply assume that a junior making mistakes is simply part of being a junior and can be coached; whereas nowadays said junior may not be willing to take your advice Hot take: This reads like an old person looking down upon young people. Can you explain why it isn't? Else, this reads like: "When I was young, we worked hard and listened to our elders. These days, young people ignore our advice." Every time I see inter-generational commentary like this (which is inevitably from personal experience), I am immediately suspicious. I can assure you that when I was young, I did not listen to older people's advice and I tried to do everything my own way. Why would this be any different in the current generation? In my experience, it isn't. On a positive note: I can remember mentoring some young people and watching them comb through blogs to learn about programming. I am so old that my shelf is/was full of O'Reilly books. By the time I was mentoring them, few people under 25 were reading O'Reilly books. It opened my eyes that how people changes more than what people learn. Example: Someone is trying to learning about access control modifiers for classes/methods in a programming language. Old days: Get the O'Reilly book for that programming language. Lookup access modifiers in the index. 10 year ago: Google for a blog with an intro to the programming language. There will be a tip about what access modifiers can do. Today: Ask ChatGPT. In my (somewhat contrived) example, the how is changing, but not the what.
- transfer92 1y ago
  > I can assure you that when I was young, I did not listen to older people's advice and I tried to do everything my own way. Hot take: This reads like a person who was difficult to work with. Senior people have responsibility, therefore in a business situation they have authority. Junior people who think they know it all don't like this. If there's a disagreement between a senior person and a junior person about something, they should, of course, listen to each other respectfully. If that's not happening, then one of them is not being a good employee. But if they are, then the supervisor makes the final call.
- shagie 1y ago
  > Old days: Get the O'Reilly book for that programming language. Lookup access modifiers in the index. 10 year ago: Google for a blog with an intro to the programming language. There will be a tip about what access modifiers can do. Today: Ask ChatGPT. In my (somewhat contrived) example, the how is changing, but not the what. The tangent to that is it is also changing with the how much one internalizes about the problem domain and is able to apply that knowledge later. Hard fought knowledge from the old days is something that shapes how I design systems today. However, the tendency of people who reach for ChatGPT today to solve a problem results in them making the same mistakes again the next time since the information is so easy to access. It also results in things that are larger are more difficult... the "how do you architect this larger system" is something you learn by building the smaller systems and learning about them so that their advantages and disadvantages and how and such becomes an inherent part of how you conceive of the system as a whole. ... Being able to have ChatGPT do it means people often don't think about the larger problem or how it fits together. I believe that is harder for a junior who is using ChatGPT to advance to being a mid level or senior developer than it is for a junior from the old days because of the lack of retention of the knowledge of the problems and solutions.
  - moosedev 1y ago
    They’re going to get promoted anyway. The “senior” title will simply (continue to) lose meaning to inflation.
- 1y ago
chairmansteve 1y ago
The intern is not producing code that is up to the standard you expect, and will not change it? I saw a situation like this many years ago. The newly hired midlevel engineer thought he was smarter than the supervisor. Kept on arguing about code style, system design etc. He was fired after 6 months. But I was friendly with him, so we kept in touch. He ended up working at MSFT for 3 times the salary.

lll-o-lll 1y ago

> Is there any way to distinguish between these two categories? Yes, it should be obvious. At least at the current state of LLMs. > There's the long term incentive, but is there a short term one? The short term incentive is keeping their job.

hombre_fatal 1y ago

Just like anything, anyone who did the work themself should be able to speak intelligently about the work and the decisions behind its idiosyncrasies. For software, I can imagine a process where junior developers create a PR and then run through it with another engineer side by side. The short-term incentive would be that they can do it, else they'd get exposed.

icedchai 1y ago

There are some definite signs of over reliance on AI. From emojis in comments, to updates completely unrelated to the task at hand, if you ask "why did you make this change?", you'll typically get no answer. I don't mind if AI is used as a tool, but the output needs to be vetted.

wwweston 1y ago
Exactly. Use LLMs as a tutor, a tool, and make sure you understand the output.
- agumonkey 1y ago
  My favorite prompt is "your goal is to retire yourself"
throwaway2037 1y ago
What is wrong with emojis in comments? I see no issue with it. Do I do it myself? No. Would I pushback if a young person added emojis to comments? No. I am looking at "the content, not the colour".
- chihuahua 1y ago
  I think GP may be thinking that emojis in PR comments (plus the other red flags they mentioned) are the result of copy/paste from LLM output, which might imply that the person who does mindless copy/pasting is not adding anything and could be replaced by LLM automation.
- venturecruelty 1y ago
  The point is that heavy emoji use means AI was likely used to produce a changeset, not that emojis are inherently bad.
- icedchai 1y ago
  The emojis are not a problem themselves. They're a warning sign: slop is (probably) present, look deeper.

bryanrasmussen 1y ago

having dealt with a few people who just copy/pasted Stackoverflow I really feel that using an LLM is an improvement. That is at least for the people who don't understand what they're doing, the LLM tends to come out with something I can at least turn into something useful. It might be reversed though for people who know what they're doing. IF they know what they're doing they might theoretically be able to put together some stackoverflow results that make sense, and build something up from that better than what gets generated from LLM (I am not asserting this would happen, and thinking it might be the case) However I don't know as I've never known anyone who knew what they were doing who also just copy/pasted some stackoverflow or delegated to LLM significantly.

water-data-dude 1y ago

Is/was copy/pasting from Stackoverflow considered harmful? You have a problem, you do a web search and you find someone who asked the same question on SO, and there's often a solution. You might be specifically talking about people who copy/paste without understanding, but I think it's still OK-ish to do that, since you can't make an entire [whatever you're coding up] by copy/pasting snippets from SO like you're cutting words out of a magazine for a ransom note. There's still thought involved, so it's more like training wheels that you eventually outgrow as you get more understanding.

vkou 1y ago
> Is/was copy/pasting from Stackoverflow considered harmful? It at least forces you to tinker with whatever you copied over.

gunsch 1y ago

Pair programming! Get hands-on with your junior engineers and their development process. Push them to think through things and not just ask the LLM everything.

johnnyanmac 1y ago
I've seen some overly excessive pair programming initiatives out there, but it does baffle me why less people who struggle with this do it. Take even just 30 minutes to pair program on a problem and see their process and you can reveal so much. But I suppose my question is rhetorical. We're laying off hundreds of thousands of engineers and maming existing ones do the work of 3-4 engineers. Not much time to help the juniors.