5 ms·

In my opinion this is another case where people look at it as a technical problem when it's actually a people problem. If someone does it once, they get a stern

In my opinion this is another case where people look at it as a technical problem when it's actually a people problem. If someone does it once, they get a stern message about it. If it happens twice, it gets rejected and sent to their manager. Regardless of how you authored a pull request, you are signing off on it with your name. If it's garbage, then you're responsible.

Ekaros 1y ago
Maybe the process should have actual two stage pull requests. First stage is you have to comment the request and show some test cases against it. And only then next person has to take a look. Not sure if such flow is even possible with current tools.
- oblio 1y ago
  Build the PR and run tests against it. Supported by all major CI/CD tools.
tyleo 1y ago
I agree and I’m surprised more people don’t get this. Bad behaviors aren’t suddenly okay because AI makes them easy. If you are wasting time you may be value negative to a business. If you are value negative over the long run you should be let go. We’re ultimately here to make money, not just pump out characters into text files.
- jackblemming 1y ago
  How do you know the net value add isn’t greater with the AI, even if it requires more code review comments (and angrier coworkers)?
  - y0eswddl 1y ago
    all the recent studies (that are constantly posted here) that say so.
    - CuriouslyC 1y ago
      The Stanford study showed mixed results, and you can stratify the data to show that AI failures are driven by process differences as much as circumstantial differences. The MIT study just has a whole host of problems, but ultimately it boils down to: giving your engineers cursor and telling them to be 10x doesn't work. Beyond each individual engineer being skilled at using AI, you have to adjust your process for it. Code review is a perfect example; until you optimize the review process to reduce human friction, AI tools are going to be massively bottlenecked.
  - tyleo 1y ago
    Because we know what the value is without AI. I’ve been in the industry for about ten years and others have been in it longer than I have. Folks have enough experience to know what good looks like and to know what bad looks like.
  - dmurvihill 1y ago
    You have it exactly backwards. If you are consuming my time with slop, it’s on you to prove there’s still a net benefit.
  - nucleardog 1y ago
    In a scenario where what we're doing is describing and assigning work to someone, having them paste that into an LLM, sending the LLM changes to me to review, me reviewing the LLM output, them pasting that back into the LLM and sending the results for me to review... What value is that person adding? I can fire up claude code/cursor/whatever myself and get the same result with less overhead. It's not a matter of "is AI valuable", it's a matter of "is this person adding value to the process". In the above case... no, none at all.
- pklausler 1y ago
  > We’re ultimately here to make money, not just pump out characters into text files. Different projects have other incentives. Dealing with AI slop from internet randos is a very real problem in open-source codebases. I've pretty much just stopped reviewing code from people that I don't know on one project that I work on when it's obviously going to take way more time than it would have done to do the patch myself. There used to be an incentive to help educate new contributors, of course, but now I can't know whether that is even happening.
  - tyleo 1y ago
    Yeah, fair enough. This applies to both business cases and OSS and the OSS incentive often isn’t money.
lezojeda 1y ago
What do you do if the manager enables it?
Macha 1y ago
The problem is leadership buy in. The person throwing the LLM slop at github has great metrics when the leadership are looking at cursor usage, lines of code, PR numbers, while the person slowing down to actually read wtf the other people are submitting is now so drowning in slop that they have less time to produce on their own. So the execs look at it as the person complaining "not keeping up with the times".
- bloppe 1y ago
  If leadership is that inept, then this is likely only 1 of many problems they are creating for the organization. I would be looking for alternative employment ASAP.
  - GuinansEyebrows 1y ago
    the issue isn't recognizing malign influence within your current organization... it's an issue throughout the entire industry, and I think what we're all afraid of is that it's becoming more inevitable every day, because we're not the ones who have the final say. the luddites essentially failed, after all, because the wider world was not and is not ready for a discussion about quality versus profit.
    - bloppe 1y ago
      A poor quality product can only be profitable if no high quality alternative exists (at a similar price point). Every time that's the case, it's an epic opportunity for anybody with the wherewithal to raise some funding and build that high quality alternative themselves. A dysfunctional industry running on AI slop will not be able to keep you from eating their lunch unless they can achieve some sort of regulatory capture, which would be a separate (political) issue. Regarding your Luddite reference, I think the cost-vs-quality debate was actually the centerpiece of that incident. Would you rather pay $100 for a T-shirt that's only marginally better than one that costs $10? I certainly would not. People are constantly evaluating cost-quality tradeoffs when making purchasing decisions. The exact ratio of the tradeoff matters. There's always a price point at which something starts (or stops) making sense.
lubujackson 1y ago
The solve is just rejecting the commit with a "clean this up" message as soon as you spot some BS. Trust is earned!
crazygringo 1y ago
This a million times. If you do this three times, that's grounds for firing. You're literally not doing your job and lying that you are. It's bizarre to me that people want to blame LLMs instead of the employees themselves. (With open source projects and slop pull requests, it's another story of course.)
- ryandrake 1y ago
  Seriously. What companies are these where “Oops, Claude wrote that and I have no idea what it does” is even remotely acceptable? No company I have ever worked at. This kind of behavior would be corrected immediately, at all levels from the eng TL up to director if it had to go that far.
- dmurvihill 1y ago
  Maybe that’s true of each individual case, but we still have the systemic problem of the way these AI tools interact with human psychology. It tends to turn off the user’s brain, even people who know better.
- int_19h 1y ago
  What happens in practice is teams are made "leaner" by laying off people (and, as seen in Microsoft layoffs, the people targeted are often veterans, due to their large salaries - which means that the team loses a lot of deep knowledge of the code in the process). And then the remaining ones are told to deal with the same amount of work as before, since "AI makes you more productive". I can't blame the remaining developers for saying, "fuck you then, I'm going to do the bare minimum". The blame is entirely on the management, and every single big tech company is complicit.
travisgriggs 1y ago
I largely agree with sibling responses. BUT... How do have code review be an educational experience for onboarding/teaching if any bad submission is cut down with due prejudice? I am happy to work with a junior engineer and is trying, and we have to loop on some silly mistakes, and pick and choose which battles to balance building confidence with developing good skills. But I am not happy to have a junior engineer throw LLM stuff at me, inspired the confidence that the psycophantic AI engendered in it, and then have to churn on that. And if you're not in the same office, how do you even hope to sift through which bad parts are which kind?
- skydhash 1y ago
  To mentor requires a mentee. If a junior is not willing to learn (reasoning, coming up, with an hypothesis, implementing the concept, and verifying it), then why should a senior bother to teach. As a philosopher has once said, a teacher is not meant to give you the solution, but to help you come up with your own.
- CuriouslyC 1y ago
  Code review as an educational device is done. We're going to stop caring about the code before people who are bad programmers right now have time to get good. We need to focus on architectural/system patterns and let go of code ownership in the traditional sense.
  - deltaburnt 1y ago
    Aren't you effectively saying that no one will understand the code they're actually deploying? That's always true to an extent, but at least today you mostly understand the code in your sub area. If we're saying the future is AI + careful review, how am I going to have enough context to even do that review?
    - CuriouslyC 1y ago
      I expect that in most cases you'll review "hot spots" that AI itself identifies while trusting AI review for the majority of code. When you need to go deeper, I expect you'll have to essentially learn the code to fix it, in roughly the same way people will occasionally need to look at the compiler output to hunt down bugs.
pfannkuchen 1y ago
Yeah it doesn’t really seem different from people copy/pasting from stack overflow without reading through it. This isn’t really a new thing, though I guess nobody was really acting like SO was the second coming so it’s probably happening more now.
- jjav 1y ago
  > Yeah it doesn’t really seem different from people copy/pasting from stack overflow without reading through it. It is vastly different because there are no (as far as I've ever seen) multi-thousand line blocks of code to cut & paste as-is from stack overflow. If you're pasting a couple dozen lines of code from a third party without understanding it, that's bad, but not unbearable to discover in a code review. But if you're posting a 5000 line pull request that you've never read and expect me to do all your work validating it, we have a problem.
  - pfannkuchen 1y ago
    What I mean is that, at least where I’ve worked, copy/pasting from SO without checking it and taking responsibility for it was already not acceptable. Making a machine generate it instead of a human is… still not acceptable.

Ekaros 1y ago

Maybe the process should have actual two stage pull requests. First stage is you have to comment the request and show some test cases against it. And only then next person has to take a look. Not sure if such flow is even possible with current tools.

oblio 1y ago
Build the PR and run tests against it. Supported by all major CI/CD tools.

tyleo 1y ago

I agree and I’m surprised more people don’t get this. Bad behaviors aren’t suddenly okay because AI makes them easy. If you are wasting time you may be value negative to a business. If you are value negative over the long run you should be let go. We’re ultimately here to make money, not just pump out characters into text files.

jackblemming 1y ago
How do you know the net value add isn’t greater with the AI, even if it requires more code review comments (and angrier coworkers)?
- y0eswddl 1y ago
  all the recent studies (that are constantly posted here) that say so.
  - CuriouslyC 1y ago
    The Stanford study showed mixed results, and you can stratify the data to show that AI failures are driven by process differences as much as circumstantial differences. The MIT study just has a whole host of problems, but ultimately it boils down to: giving your engineers cursor and telling them to be 10x doesn't work. Beyond each individual engineer being skilled at using AI, you have to adjust your process for it. Code review is a perfect example; until you optimize the review process to reduce human friction, AI tools are going to be massively bottlenecked.
- tyleo 1y ago
  Because we know what the value is without AI. I’ve been in the industry for about ten years and others have been in it longer than I have. Folks have enough experience to know what good looks like and to know what bad looks like.
- dmurvihill 1y ago
  You have it exactly backwards. If you are consuming my time with slop, it’s on you to prove there’s still a net benefit.
- nucleardog 1y ago
  In a scenario where what we're doing is describing and assigning work to someone, having them paste that into an LLM, sending the LLM changes to me to review, me reviewing the LLM output, them pasting that back into the LLM and sending the results for me to review... What value is that person adding? I can fire up claude code/cursor/whatever myself and get the same result with less overhead. It's not a matter of "is AI valuable", it's a matter of "is this person adding value to the process". In the above case... no, none at all.
pklausler 1y ago
> We’re ultimately here to make money, not just pump out characters into text files. Different projects have other incentives. Dealing with AI slop from internet randos is a very real problem in open-source codebases. I've pretty much just stopped reviewing code from people that I don't know on one project that I work on when it's obviously going to take way more time than it would have done to do the patch myself. There used to be an incentive to help educate new contributors, of course, but now I can't know whether that is even happening.
- tyleo 1y ago
  Yeah, fair enough. This applies to both business cases and OSS and the OSS incentive often isn’t money.

lezojeda 1y ago

What do you do if the manager enables it?

Macha 1y ago

The problem is leadership buy in. The person throwing the LLM slop at github has great metrics when the leadership are looking at cursor usage, lines of code, PR numbers, while the person slowing down to actually read wtf the other people are submitting is now so drowning in slop that they have less time to produce on their own. So the execs look at it as the person complaining "not keeping up with the times".

bloppe 1y ago
If leadership is that inept, then this is likely only 1 of many problems they are creating for the organization. I would be looking for alternative employment ASAP.
- GuinansEyebrows 1y ago
  the issue isn't recognizing malign influence within your current organization... it's an issue throughout the entire industry, and I think what we're all afraid of is that it's becoming more inevitable every day, because we're not the ones who have the final say. the luddites essentially failed, after all, because the wider world was not and is not ready for a discussion about quality versus profit.
  - bloppe 1y ago
    A poor quality product can only be profitable if no high quality alternative exists (at a similar price point). Every time that's the case, it's an epic opportunity for anybody with the wherewithal to raise some funding and build that high quality alternative themselves. A dysfunctional industry running on AI slop will not be able to keep you from eating their lunch unless they can achieve some sort of regulatory capture, which would be a separate (political) issue. Regarding your Luddite reference, I think the cost-vs-quality debate was actually the centerpiece of that incident. Would you rather pay $100 for a T-shirt that's only marginally better than one that costs $10? I certainly would not. People are constantly evaluating cost-quality tradeoffs when making purchasing decisions. The exact ratio of the tradeoff matters. There's always a price point at which something starts (or stops) making sense.

lubujackson 1y ago

The solve is just rejecting the commit with a "clean this up" message as soon as you spot some BS. Trust is earned!

crazygringo 1y ago

This a million times. If you do this three times, that's grounds for firing. You're literally not doing your job and lying that you are. It's bizarre to me that people want to blame LLMs instead of the employees themselves. (With open source projects and slop pull requests, it's another story of course.)

ryandrake 1y ago
Seriously. What companies are these where “Oops, Claude wrote that and I have no idea what it does” is even remotely acceptable? No company I have ever worked at. This kind of behavior would be corrected immediately, at all levels from the eng TL up to director if it had to go that far.
dmurvihill 1y ago
Maybe that’s true of each individual case, but we still have the systemic problem of the way these AI tools interact with human psychology. It tends to turn off the user’s brain, even people who know better.
int_19h 1y ago
What happens in practice is teams are made "leaner" by laying off people (and, as seen in Microsoft layoffs, the people targeted are often veterans, due to their large salaries - which means that the team loses a lot of deep knowledge of the code in the process). And then the remaining ones are told to deal with the same amount of work as before, since "AI makes you more productive". I can't blame the remaining developers for saying, "fuck you then, I'm going to do the bare minimum". The blame is entirely on the management, and every single big tech company is complicit.

travisgriggs 1y ago

I largely agree with sibling responses. BUT... How do have code review be an educational experience for onboarding/teaching if any bad submission is cut down with due prejudice? I am happy to work with a junior engineer and is trying, and we have to loop on some silly mistakes, and pick and choose which battles to balance building confidence with developing good skills. But I am not happy to have a junior engineer throw LLM stuff at me, inspired the confidence that the psycophantic AI engendered in it, and then have to churn on that. And if you're not in the same office, how do you even hope to sift through which bad parts are which kind?

skydhash 1y ago
To mentor requires a mentee. If a junior is not willing to learn (reasoning, coming up, with an hypothesis, implementing the concept, and verifying it), then why should a senior bother to teach. As a philosopher has once said, a teacher is not meant to give you the solution, but to help you come up with your own.
CuriouslyC 1y ago
Code review as an educational device is done. We're going to stop caring about the code before people who are bad programmers right now have time to get good. We need to focus on architectural/system patterns and let go of code ownership in the traditional sense.
- deltaburnt 1y ago
  Aren't you effectively saying that no one will understand the code they're actually deploying? That's always true to an extent, but at least today you mostly understand the code in your sub area. If we're saying the future is AI + careful review, how am I going to have enough context to even do that review?
  - CuriouslyC 1y ago
    I expect that in most cases you'll review "hot spots" that AI itself identifies while trusting AI review for the majority of code. When you need to go deeper, I expect you'll have to essentially learn the code to fix it, in roughly the same way people will occasionally need to look at the compiler output to hunt down bugs.

pfannkuchen 1y ago

Yeah it doesn’t really seem different from people copy/pasting from stack overflow without reading through it. This isn’t really a new thing, though I guess nobody was really acting like SO was the second coming so it’s probably happening more now.

jjav 1y ago
> Yeah it doesn’t really seem different from people copy/pasting from stack overflow without reading through it. It is vastly different because there are no (as far as I've ever seen) multi-thousand line blocks of code to cut & paste as-is from stack overflow. If you're pasting a couple dozen lines of code from a third party without understanding it, that's bad, but not unbearable to discover in a code review. But if you're posting a 5000 line pull request that you've never read and expect me to do all your work validating it, we have a problem.
- pfannkuchen 1y ago
  What I mean is that, at least where I’ve worked, copy/pasting from SO without checking it and taking responsibility for it was already not acceptable. Making a machine generate it instead of a human is… still not acceptable.