7 ms·

> A major project will discover that it has merged a lot of AI-generated code My friend works at a well-known tech company in San Francisco. He was reviewing h

> A major project will discover that it has merged a lot of AI-generated code My friend works at a well-known tech company in San Francisco. He was reviewing his junior team member's pull request. When asked what a chunk of code did, the team member matter-of-factly replied "I don't know, chatgpt wrote that"

hardbants 2y ago
[dead]
deadbabe 2y ago
I hope that junior engineer was reprimanded or even put on a PIP instead of just having the reviewer say lgtm and approve the request.
- WaxProlix 2y ago
  Probably depends a lot on the team culture. Depending on what part of the product lifecycle you're on (proving a concept, rushing to market, scaling for the next million TPS, moving into new verticals,...) and where the team currently is, it makes a lot of sense to generate more of the codebase by AI. Write some decent tests, commit, move on. I wish my reports would use more AI tools for parts of our codebase that don't need a high bar of scrutiny, boilerplate at enterprise scale is a major source of friction and - tbh - burnout.
  - bradly 2y ago
    Yes and the team could be missing structures to support junior engineers. What made them not ask for help or pairing is really important to dig in to and I would expect a senior manager to understand this and be introspective on what environment they have created where this human made this choice.
    - deleted 2y ago
      [deleted]
  - not2b 2y ago
    Unless the plan is to quickly produce a prototype that will be mostly thrown away, any code that gets into the product is going to generate far more work maintaining it over the lifetime of a product than the cost to code it in the first place. As a reviewer I'd push back, and say that I'll only be able to approve the review when the junior programmer can explain what it does and why it's correct. I wouldn't reject it solely because chatgpt made it, but if the checkin causes breakage it normally gets assigned back to the person who checked it in, and if that person has no clue we have a problem.
    - KronisLV 2y ago
      > I wouldn't reject it solely because chatgpt made it, but if the checkin causes breakage it normally gets assigned back to the person who checked it in, and if that person has no clue we have a problem. That's a fair point, but regardless of who wrote the code (or what tools were used) it should also probably be as clear as possible to everyone who reads it, because chances are that at some point that person will be elsewhere and some other person will have to take over.
      - not2b 2y ago
        True, but you're talking about the difference between "only one person understands this, that's a risk!" and "zero people understand this".
    - solatic 2y ago
      Not being willing to throw out bad/unused features is a different trap that organizations fall into. The amount of work that goes into, shall we say fortifying the foundations of a particular feature, ideally should be proportional to how much revenue that feature is responsible for. Test code also has to be maintained, and increasing the maintenance burden on something that has its own maintenance burden when customers don't even like it is shortsighted at the very least.
  - GeoAtreides 2y ago
    > Write some decent tests, commit, move on. Move on to what?! Where does a junior programmer who doesn't understand what the code does moves on to?
- XorNot 2y ago
  I mean if that was an answer I got given by a junior during a code review the next email I'd be sending would be to my team lead about it.
AlwaysRock 2y ago
This is more of an early career engineer thing than a ChatGPT thing. 'I don't know, I found it on stackoverflow' could have easily been the answer for the last ten years.
- devsda 2y ago
  The main problem is not the source of solution but not making an effort to understand the code they have put in. The "I don't know" might as well be "I don't care".
  - arkh 2y ago
    That's where you'd like your solution engine to be able to tell you how to get the solution it is giving you. Something good answers on Stack Overflow will do: links to the relevant documentation, steps you can go through to get a better diagnostic of your problem etc. Get the fire lit with the explanation of where to get wood and how to light it in your condition so next time you don't need to consult you solution engine.
- mrweasel 2y ago
  There's nothing inherently wrong with getting help from either and LLM, or StackOverflow, it's the "I don't know' part that bothers me. One the funnier reactions to "I got it from StackOverflow" is the followup question "From the question or the answers?" If you just adds code, without understanding how it works, regardless of where it came from and potential licensing issues, then I question your view on programming. If I have a paint come in and paint my house and get paint all over the place, floors, windows, electrical socket but still get the walls the color I want, then I wouldn't consider that person a professional painter.
- Vampiero 2y ago
  No, a real engineer goes on SO to understand. A junior goes on SO to copy and paste. If your answer is "I don't know I just copied" you're not doing any engineering and it's awful to pretend you are. Our job is literally about asking "why" and "how" until we don't need to anymore because our pattern matching skills allow us to generalize. At this point in my career I rarely ever go to SO, and when I do it's because of some obscure thing that 7 other people came across and decided to post a question about. Or to look up "how to do the most basic shit in language I am not familiar with", but that role was taken over by LLMs.
userbinator 2y ago
At least he's honest.
DowsingSpoon 2y ago
I am fairly certain that if someone did that where I work then security would be escorting them off the property within the hour. This is NOT Okay.
- phinnaeus 2y ago
  Are you hiring?
- bitmasher9 2y ago
  Where I work we are actively encouraged to use more AI tools while coding, to the point where my direct supervisor asked why my team’s usage statistics were lower than company average.
  - dehrmann 2y ago
    It's not necessarily the use of AI tools (though the license parts are an issue), is that someone submitted code for review without knowing how it works.
    - masteruvpuppetz 2y ago
      I think we should / have already reached to a place where AI written code is acceptable.
      - bigstrat2003 2y ago
        Whether it's acceptable or not to submit AI code, it is clearly unacceptable to submit code that you don't even understand. If that's all an employee is capable of, why on earth would the employer pay them software engineer salary versus hiring someone to do the exact same for minimum wage?
        userbinator 2y ago
        Or even replace them with the AI directly.
gunian 2y ago
the saddest part is if i wrote the code myself it would be worse lol GPT is coding at a intern level and as a dumb human being I feel sad I have been replaced but not as catastrophic as they made it seem it's interesting to see the underlying anxiety among devs though I think there is a place in the back of their minds that knows the models will get better and better and someday could get to staff engineer level
- nozzlegear 2y ago
  I don't think that's the concern at all. The concern (imo) is that you should at least understand what the code is doing before you accept it verbatim and add it to your company's codebase. The potential it has to introduce bugs or security flaws is too great to just accept it without understanding it.
  - gunian 2y ago
    Exactly why devs are getting the bug bucks that is right now at some point what if someone figures out a way to make it deterministic and able to write code without bugs?
    - eggnet 2y ago
      Then the programming language becomes natural language and you’ll have to be very good at describing what you want. Unless you are talking about AGI, aka, the singularity. Which is a whole other topic.
      - gunian 2y ago
        not AGI at that point all human jobs can be replaced that's my personal bar at least i'm thinking like models get small enough, you fine tune them on your code, you add fuzzing, rewriting it may not be bug free but could it become self healing with minimal / known natural language locations? or instead of x engineers one feeds the skeleton to chatgpt 20 or something and instead of giving you the result immediately it does it iteratively would still be cheaper than x devs
alisonatwork 2y ago
I have heard the same response from junior devs and external contractors for years, either because they copied something from StackOverflow, or because they copied something from a former client/employer (popular one in China), or even because they just uncritically copied something from another piece of code in the same project. From the point of view of these sorts of developers they are being paid to make the tests go green or to make some button appear on a page that kindasorta does something in the vague direction of what was in the spec, and that's the end of their responsibility. Unused variables? Doesn't matter. Unreachable code blocks? Doesn't matter. Comments and naming that have nothing to do with the actual business case the code is supposed to be addressing? Doesn't matter. I have spent a lot of time trying to mentor these sorts of devs and help them to understand why just doing the bare minimum isn't really a good investment in their own career not to mention it's disrespectful of their colleagues who now need to waste time puzzling through their nonsense and eventually (inevitably) fixing their bugs... Seems to get through about 20% of the time. Most of the rest of the time these folks just smile and nod and continue not caring, and companies can't afford the hassle of firing them, then you open LinkedIn years later and turns out somehow they've failed up to manager, architect or executive while you're still struggling along as a code peasant who happens to take pride in their work. Sorry, got a little carried away. Anywho, the point is LLMs are just another tool for these folks. It's not new, it's just worse now because of the mixed messaging where executives are hyping the tech as a magical solution that will allow them to ship more features for less cost.
- bryanrasmussen 2y ago
  >Unused variables? Doesn't matter. Unreachable code blocks? Doesn't matter. Comments and naming that have nothing to do with the actual business case the code is supposed to be addressing? Doesn't matter. maybe I am just supremely lucky but while I have encountered people like (in the coding part) it is somewhat rare from my experience. These comments on HN always makes it seem like it's at least 30% of the people out there.
  - alisonatwork 2y ago
    I think even though these types of developers are fairly rare, they have a disproportionate negative impact on the quality of the code and the morale of their colleagues, which is perhaps why people remember them and talk about it more often. The p95 developers who are more-or-less okay aren't really notable enough to be worth complaining about on HN, since they are us.
ghxst 2y ago
Was this a case of something along the lines of an isolated function that had a bunch of bit shifting magic for some hyper optimization that was required, or was it just regular code? Not saying it's acceptable, but the first example is maybe worth a thoughtful discussion while the latter would make me lose hope.
ErrantX 2y ago
Feels like a controls failure as much as anything else. Any decently sized company that allows unrestricted access to llms, well that's going to be the tip of the iceberg. Also, the culture of don't care comes from somewhere, not ChatGPT
johnisgood 2y ago
There is no shame, damn.
- deleted 2y ago
  [deleted]
rixrax 2y ago
Essentially you're paying human to be a proxy between the requirements, LLM and codebase. Some people I'm talking to lament having to pay top dollar to their junior (and other kinds I'm sure) devs for this, but I think this is and will be the new reality and new normal. And instead we should start thinking how to make best of it, and how to help maximize success for these devs. Few decades down the road though we are likely to be viewing this current situation similar to how we're looking at 'human computers'[0] of yesteryear. [0] https://en.wikipedia.org/wiki/Women_in_computing https://en.wikipedia.org/wiki/Women_in_computing
sofixa 2y ago
I have a better one, a senior architect who wrote a proposal for a new piece of documentation, and when asked about his 3 main topics in the doc and why them, said "LLM said those are the main ones". The rest of the doc was obviously incoherent LLM soup as well.
BiteCode_dev 2y ago
I'm a strong proponent of using LLM and use them extensively. But this is a fireable offense in my book.
ginko 2y ago
>When asked what a chunk of code did, the team member matter-of-factly replied "I don't know, chatgpt wrote that" That'd be an immediate -2 from me.
ben_w 2y ago
> He was reviewing his junior team member's pull request. When asked what a chunk of code did, the team member matter-of-factly replied "I don't know, chatgpt wrote that" I remember being a junior nearly 20 years back, a co-worker someone asked me how I'd implemented an invulnerability status, and I said something equally stupid despite knowing perfectly well how I'd implemented it and there not being any consumer grade AI more impressive than spam filters and Office's spelling and grammar checking. Which may or may not be relevant to the example of your friend's coworker, but I do still wonder how much of my answers as a human are on auto-complete. It's certainly more than none, and not just from that anecdote… https://duckduckgo.com/?t=h_&q=enjoy+your+meal+thanks+you+too&iax=images&ia=images https://duckduckgo.com/?t=h_&q=enjoy+your+meal+thanks+you+to...
stcroixx 2y ago
This is the norm on my majority H1B team. Nobody sees anything wrong with it but me so I stopped caring too.
- sumedh 2y ago
  If the management does not care why should the employees?

hardbants 2y ago

[dead]

deadbabe 2y ago

I hope that junior engineer was reprimanded or even put on a PIP instead of just having the reviewer say lgtm and approve the request.

WaxProlix 2y ago
Probably depends a lot on the team culture. Depending on what part of the product lifecycle you're on (proving a concept, rushing to market, scaling for the next million TPS, moving into new verticals,...) and where the team currently is, it makes a lot of sense to generate more of the codebase by AI. Write some decent tests, commit, move on. I wish my reports would use more AI tools for parts of our codebase that don't need a high bar of scrutiny, boilerplate at enterprise scale is a major source of friction and - tbh - burnout.
- bradly 2y ago
  Yes and the team could be missing structures to support junior engineers. What made them not ask for help or pairing is really important to dig in to and I would expect a senior manager to understand this and be introspective on what environment they have created where this human made this choice.
  - deleted 2y ago
    [deleted]
- not2b 2y ago
  Unless the plan is to quickly produce a prototype that will be mostly thrown away, any code that gets into the product is going to generate far more work maintaining it over the lifetime of a product than the cost to code it in the first place. As a reviewer I'd push back, and say that I'll only be able to approve the review when the junior programmer can explain what it does and why it's correct. I wouldn't reject it solely because chatgpt made it, but if the checkin causes breakage it normally gets assigned back to the person who checked it in, and if that person has no clue we have a problem.
  - KronisLV 2y ago
    > I wouldn't reject it solely because chatgpt made it, but if the checkin causes breakage it normally gets assigned back to the person who checked it in, and if that person has no clue we have a problem. That's a fair point, but regardless of who wrote the code (or what tools were used) it should also probably be as clear as possible to everyone who reads it, because chances are that at some point that person will be elsewhere and some other person will have to take over.
    - not2b 2y ago
      True, but you're talking about the difference between "only one person understands this, that's a risk!" and "zero people understand this".
  - solatic 2y ago
    Not being willing to throw out bad/unused features is a different trap that organizations fall into. The amount of work that goes into, shall we say fortifying the foundations of a particular feature, ideally should be proportional to how much revenue that feature is responsible for. Test code also has to be maintained, and increasing the maintenance burden on something that has its own maintenance burden when customers don't even like it is shortsighted at the very least.
- GeoAtreides 2y ago
  > Write some decent tests, commit, move on. Move on to what?! Where does a junior programmer who doesn't understand what the code does moves on to?
XorNot 2y ago
I mean if that was an answer I got given by a junior during a code review the next email I'd be sending would be to my team lead about it.

AlwaysRock 2y ago

This is more of an early career engineer thing than a ChatGPT thing. 'I don't know, I found it on stackoverflow' could have easily been the answer for the last ten years.

devsda 2y ago
The main problem is not the source of solution but not making an effort to understand the code they have put in. The "I don't know" might as well be "I don't care".
- arkh 2y ago
  That's where you'd like your solution engine to be able to tell you how to get the solution it is giving you. Something good answers on Stack Overflow will do: links to the relevant documentation, steps you can go through to get a better diagnostic of your problem etc. Get the fire lit with the explanation of where to get wood and how to light it in your condition so next time you don't need to consult you solution engine.
mrweasel 2y ago
There's nothing inherently wrong with getting help from either and LLM, or StackOverflow, it's the "I don't know' part that bothers me. One the funnier reactions to "I got it from StackOverflow" is the followup question "From the question or the answers?" If you just adds code, without understanding how it works, regardless of where it came from and potential licensing issues, then I question your view on programming. If I have a paint come in and paint my house and get paint all over the place, floors, windows, electrical socket but still get the walls the color I want, then I wouldn't consider that person a professional painter.
Vampiero 2y ago
No, a real engineer goes on SO to understand. A junior goes on SO to copy and paste. If your answer is "I don't know I just copied" you're not doing any engineering and it's awful to pretend you are. Our job is literally about asking "why" and "how" until we don't need to anymore because our pattern matching skills allow us to generalize. At this point in my career I rarely ever go to SO, and when I do it's because of some obscure thing that 7 other people came across and decided to post a question about. Or to look up "how to do the most basic shit in language I am not familiar with", but that role was taken over by LLMs.

userbinator 2y ago

At least he's honest.

DowsingSpoon 2y ago

I am fairly certain that if someone did that where I work then security would be escorting them off the property within the hour. This is NOT Okay.

phinnaeus 2y ago
Are you hiring?
bitmasher9 2y ago
Where I work we are actively encouraged to use more AI tools while coding, to the point where my direct supervisor asked why my team’s usage statistics were lower than company average.
- dehrmann 2y ago
  It's not necessarily the use of AI tools (though the license parts are an issue), is that someone submitted code for review without knowing how it works.
  - masteruvpuppetz 2y ago
    I think we should / have already reached to a place where AI written code is acceptable.
    - bigstrat2003 2y ago
      Whether it's acceptable or not to submit AI code, it is clearly unacceptable to submit code that you don't even understand. If that's all an employee is capable of, why on earth would the employer pay them software engineer salary versus hiring someone to do the exact same for minimum wage?
      - userbinator 2y ago
        Or even replace them with the AI directly.

gunian 2y ago

the saddest part is if i wrote the code myself it would be worse lol GPT is coding at a intern level and as a dumb human being I feel sad I have been replaced but not as catastrophic as they made it seem it's interesting to see the underlying anxiety among devs though I think there is a place in the back of their minds that knows the models will get better and better and someday could get to staff engineer level

nozzlegear 2y ago
I don't think that's the concern at all. The concern (imo) is that you should at least understand what the code is doing before you accept it verbatim and add it to your company's codebase. The potential it has to introduce bugs or security flaws is too great to just accept it without understanding it.
- gunian 2y ago
  Exactly why devs are getting the bug bucks that is right now at some point what if someone figures out a way to make it deterministic and able to write code without bugs?
  - eggnet 2y ago
    Then the programming language becomes natural language and you’ll have to be very good at describing what you want. Unless you are talking about AGI, aka, the singularity. Which is a whole other topic.
    - gunian 2y ago
      not AGI at that point all human jobs can be replaced that's my personal bar at least i'm thinking like models get small enough, you fine tune them on your code, you add fuzzing, rewriting it may not be bug free but could it become self healing with minimal / known natural language locations? or instead of x engineers one feeds the skeleton to chatgpt 20 or something and instead of giving you the result immediately it does it iteratively would still be cheaper than x devs

alisonatwork 2y ago

I have heard the same response from junior devs and external contractors for years, either because they copied something from StackOverflow, or because they copied something from a former client/employer (popular one in China), or even because they just uncritically copied something from another piece of code in the same project. From the point of view of these sorts of developers they are being paid to make the tests go green or to make some button appear on a page that kindasorta does something in the vague direction of what was in the spec, and that's the end of their responsibility. Unused variables? Doesn't matter. Unreachable code blocks? Doesn't matter. Comments and naming that have nothing to do with the actual business case the code is supposed to be addressing? Doesn't matter. I have spent a lot of time trying to mentor these sorts of devs and help them to understand why just doing the bare minimum isn't really a good investment in their own career not to mention it's disrespectful of their colleagues who now need to waste time puzzling through their nonsense and eventually (inevitably) fixing their bugs... Seems to get through about 20% of the time. Most of the rest of the time these folks just smile and nod and continue not caring, and companies can't afford the hassle of firing them, then you open LinkedIn years later and turns out somehow they've failed up to manager, architect or executive while you're still struggling along as a code peasant who happens to take pride in their work. Sorry, got a little carried away. Anywho, the point is LLMs are just another tool for these folks. It's not new, it's just worse now because of the mixed messaging where executives are hyping the tech as a magical solution that will allow them to ship more features for less cost.

bryanrasmussen 2y ago
>Unused variables? Doesn't matter. Unreachable code blocks? Doesn't matter. Comments and naming that have nothing to do with the actual business case the code is supposed to be addressing? Doesn't matter. maybe I am just supremely lucky but while I have encountered people like (in the coding part) it is somewhat rare from my experience. These comments on HN always makes it seem like it's at least 30% of the people out there.
- alisonatwork 2y ago
  I think even though these types of developers are fairly rare, they have a disproportionate negative impact on the quality of the code and the morale of their colleagues, which is perhaps why people remember them and talk about it more often. The p95 developers who are more-or-less okay aren't really notable enough to be worth complaining about on HN, since they are us.

ghxst 2y ago

Was this a case of something along the lines of an isolated function that had a bunch of bit shifting magic for some hyper optimization that was required, or was it just regular code? Not saying it's acceptable, but the first example is maybe worth a thoughtful discussion while the latter would make me lose hope.

ErrantX 2y ago

Feels like a controls failure as much as anything else. Any decently sized company that allows unrestricted access to llms, well that's going to be the tip of the iceberg. Also, the culture of don't care comes from somewhere, not ChatGPT

johnisgood 2y ago

There is no shame, damn.

deleted 2y ago
[deleted]

rixrax 2y ago

Essentially you're paying human to be a proxy between the requirements, LLM and codebase. Some people I'm talking to lament having to pay top dollar to their junior (and other kinds I'm sure) devs for this, but I think this is and will be the new reality and new normal. And instead we should start thinking how to make best of it, and how to help maximize success for these devs. Few decades down the road though we are likely to be viewing this current situation similar to how we're looking at 'human computers'[0] of yesteryear. [0] https://en.wikipedia.org/wiki/Women_in_computing https://en.wikipedia.org/wiki/Women_in_computing

sofixa 2y ago

I have a better one, a senior architect who wrote a proposal for a new piece of documentation, and when asked about his 3 main topics in the doc and why them, said "LLM said those are the main ones". The rest of the doc was obviously incoherent LLM soup as well.

BiteCode_dev 2y ago

I'm a strong proponent of using LLM and use them extensively. But this is a fireable offense in my book.

ginko 2y ago

>When asked what a chunk of code did, the team member matter-of-factly replied "I don't know, chatgpt wrote that" That'd be an immediate -2 from me.

ben_w 2y ago

> He was reviewing his junior team member's pull request. When asked what a chunk of code did, the team member matter-of-factly replied "I don't know, chatgpt wrote that" I remember being a junior nearly 20 years back, a co-worker someone asked me how I'd implemented an invulnerability status, and I said something equally stupid despite knowing perfectly well how I'd implemented it and there not being any consumer grade AI more impressive than spam filters and Office's spelling and grammar checking. Which may or may not be relevant to the example of your friend's coworker, but I do still wonder how much of my answers as a human are on auto-complete. It's certainly more than none, and not just from that anecdote… https://duckduckgo.com/?t=h_&q=enjoy+your+meal+thanks+you+too&iax=images&ia=images https://duckduckgo.com/?t=h_&q=enjoy+your+meal+thanks+you+to...

stcroixx 2y ago

This is the norm on my majority H1B team. Nobody sees anything wrong with it but me so I stopped caring too.

sumedh 2y ago
If the management does not care why should the employees?