5 ms·

I make a lot of drive-by contributions, and I use AI coding tools. I submitted my first PR that is a cross between those two recently. It's somewhere between "v

I make a lot of drive-by contributions, and I use AI coding tools. I submitted my first PR that is a cross between those two recently. It's somewhere between "vibe-coded" and "vibe-engineered", where I definitely read the resulting code, had the agent make multiple revisions, and deployed the result on my own infrastructure before submitting a PR. In the PR I clearly stated that it was done by a coding agent. I can't imagine that any policy against LLM code would allow this sort of thing, but I also imagine that if I don't say "this was made by a coding agent", that no one would ever know. So, should I just stop contributing, or start lying? [append] Getting a lot of hate for this, which I guess is a pretty clear answer. I guess the reason I'm not receiving the "fuck off" clearly is because when I see these threads of people complaining about AI content, it's really clearly low-quality crap that (for example) doesn't even compile, and wastes everyone's time. I feel different from those cases because I did spend my time to resolve the issue for myself, did review the code, did test it, and do stand by what I'm putting under my name. Hmm.

TingPing 11mo ago
I think it’s disrespectful of others to throw generated code their way. They become responsible for it and often donate their time.
- CGamesPlay 11mo ago
  I think it's disrespectful to throw bad code on somebody else, regardless of its provenance. And (as a professional who uses AI coding agents) I understand that LLM code is often bad code. But the OP isn't complaining about the deluge of bad code being submitted as PRs, they're complaining about LLM code being submitted as PRs. I've also been on the other side of this, receiving some spammy LLM-generated irrelevant "security vulnerabilities", so I also get the desire for some filtering. I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen.
  - cyphar 11mo ago
    (OP here.) Well, we don't receive that many low-quality PRs in general (I opened this issue to discuss solutions before it becomes a real problem). Speaking personally, when it does happen I try to help mentor the person to improve their code or (in the case where the person isn't responsive) I sit down and make the improvements I would've made and explain why they were made as a comment in the PR. When it comes to LLM-generated code, I am now going to be going back-and-forth with someone who is probably just going to copy-paste my comments into an LLM (probably not even bothering to read them). It just feels disrespectful. > I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen. Well, this is a two-way street -- all of the LLM-generated PRs and issues I've seen so far do not say that they are LLM-generated, in a way that I am tempted to describe as "dishonest". If every LLM-generated PR was tagged as such, I might have a different outlook on the situation (and might instead be willing to reviewing these issues but with lower priority).
    - CGamesPlay 11mo ago
      Thanks for the response here. I also read your follow-up on the Github issue. I definitely agree with you on the mentorship point. The "CPU quota allocation mismatch" issue felt particularly agitating to read from the point of view as a maintainer. Human being acting as a proxy for ChatGPT. Have you considered putting an "LLM usage disclosure" question on the issue/PR templates? Something like "To what extent were AI tools used to create this issue/PR? What validation steps did you perform to ensure accuracy?" The "hard-line policy" would then shift from being "used LLM tools" to "lied on the LLM usage disclosure", and it feels a lot less like selective enforcement (from my perspective). Obviously it won't stop these spammy issues/PRs, but neither will a hard-line policy against all AI.
      - cyphar 11mo ago
        I've been thinking about this a bit over the past few days, and I think this is a fairly reasonable middle ground (and also allows maintainers to decide how they wish to engage with LLM-generated work).
Blackthorn 11mo ago
They don't want your contribution, so don't disrespect them by trying to make it.
dingnuts 11mo ago
[dead]
b00ty4breakfast 11mo ago
"should I stop doing this thing that people are explicitly saying they don't want me to do or should I keep doing this thing that people are explicitly saying they don't want me to do???"
- alex77456 11mo ago
  As with evertything, there's always nuance. If everyone followed similar midset to the comment you were replying to, likely llm generated pr issues wouldn't be as much of a problem and we wouldn't even be here discussing it
zzo38computer 11mo ago
I would potentially accept it (although you should not lie about it, and if someone says in their policy that they specifically do not want this then you should not send it), although for my own projects, there are reasons (having to do with the way the version control is set up) that I cannot accept direct contributions anyways, and I usually make my own changes to any contributions anyways (but anyone else can make their own copy with their own changes however they want to do, whether or not I accept it). Since it is honest, and that you had reviewed it by yourself before submitting it, and also tested it, and that I would review it again anyways as well, it would be acceptable, although I would still much more prefer to receive contributions that do not use LLM, still the way you do it is much better than the other stuff some by LLM which does not meet the threshold of being acceptable.
SleekoNiko 11mo ago
If I were the maintainer, and you specified up front that your PR was largely written by an LLM, I would appreciate it. I may prioritize it lower than other PRs, perhaps, but that's about it. I think it's also important to disclose how rigorously you tested your changes, too. I would hate to spend my time looking at a change that was never even tested by a human. It sounds like you do both of these. Judging by the other replies, it seems that other reviewers may take a harsher stance, given the heavily polarized nature of LLMs. Still, if you made the changes and you're up front about your methodology, why not? In the worst case, your PR gets closed and everybody moves on.
ack_complete 11mo ago
> I can't imagine that any policy against LLM code would allow this sort of thing, but I also imagine that if I don't say "this was made by a coding agent", that no one would ever know. So, should I just stop contributing, or start lying? If a project has a stated policy that code written with an LLM-based aid is not accepted, then it shouldn't be submitted, same as with anything else that might be prohibited. If you attempt to circumvent this by hiding it and it is revealed that you knowingly did so in violation of the policy, then it would be unsurprising for you to receive a harsh reply and/or ban, as well as a revert if the PR was committed. This would be the same as any other prohibition, such as submitting code copied from another project with an incompatible license. You could argue that such a blanket ban is unwarranted, and you might be right. But the project maintainers have a right to set the submission rules for their project, even if it rules out high-quality LLM assisted submissions. The right way to deal with this is to ask the project maintainers if they would be willing the adjust the policy, not to try to slip such code into the project anyway.
darkwater 11mo ago
> [append] Getting a lot of hate for this, which I guess is a pretty clear answer. I guess the reason I'm not receiving the "fuck off" clearly is because when I see these threads of people complaining about AI content, it's really clearly low-quality crap that (for example) doesn't even compile, and wastes everyone's time. I think you don't deserve the downvotes and, if you really do what you say you do, that's the ONLY way to use LLMs for coding and contributing to opensource software, or to a company's software. Sadly, the vast majority of LLM users don't and will never use it like that. And while they can get fired for being useless monkeys in a real company, they will keep sending PRs to opensource software, so that's clearly a different scenario that needs a different solution.

TingPing 11mo ago

I think it’s disrespectful of others to throw generated code their way. They become responsible for it and often donate their time.

CGamesPlay 11mo ago
I think it's disrespectful to throw bad code on somebody else, regardless of its provenance. And (as a professional who uses AI coding agents) I understand that LLM code is often bad code. But the OP isn't complaining about the deluge of bad code being submitted as PRs, they're complaining about LLM code being submitted as PRs. I've also been on the other side of this, receiving some spammy LLM-generated irrelevant "security vulnerabilities", so I also get the desire for some filtering. I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen.
- cyphar 11mo ago
  (OP here.) Well, we don't receive that many low-quality PRs in general (I opened this issue to discuss solutions before it becomes a real problem). Speaking personally, when it does happen I try to help mentor the person to improve their code or (in the case where the person isn't responsive) I sit down and make the improvements I would've made and explain why they were made as a comment in the PR. When it comes to LLM-generated code, I am now going to be going back-and-forth with someone who is probably just going to copy-paste my comments into an LLM (probably not even bothering to read them). It just feels disrespectful. > I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen. Well, this is a two-way street -- all of the LLM-generated PRs and issues I've seen so far do not say that they are LLM-generated, in a way that I am tempted to describe as "dishonest". If every LLM-generated PR was tagged as such, I might have a different outlook on the situation (and might instead be willing to reviewing these issues but with lower priority).
  - CGamesPlay 11mo ago
    Thanks for the response here. I also read your follow-up on the Github issue. I definitely agree with you on the mentorship point. The "CPU quota allocation mismatch" issue felt particularly agitating to read from the point of view as a maintainer. Human being acting as a proxy for ChatGPT. Have you considered putting an "LLM usage disclosure" question on the issue/PR templates? Something like "To what extent were AI tools used to create this issue/PR? What validation steps did you perform to ensure accuracy?" The "hard-line policy" would then shift from being "used LLM tools" to "lied on the LLM usage disclosure", and it feels a lot less like selective enforcement (from my perspective). Obviously it won't stop these spammy issues/PRs, but neither will a hard-line policy against all AI.
    - cyphar 11mo ago
      I've been thinking about this a bit over the past few days, and I think this is a fairly reasonable middle ground (and also allows maintainers to decide how they wish to engage with LLM-generated work).

Blackthorn 11mo ago

They don't want your contribution, so don't disrespect them by trying to make it.

dingnuts 11mo ago

[dead]

b00ty4breakfast 11mo ago

"should I stop doing this thing that people are explicitly saying they don't want me to do or should I keep doing this thing that people are explicitly saying they don't want me to do???"

alex77456 11mo ago
As with evertything, there's always nuance. If everyone followed similar midset to the comment you were replying to, likely llm generated pr issues wouldn't be as much of a problem and we wouldn't even be here discussing it

zzo38computer 11mo ago

I would potentially accept it (although you should not lie about it, and if someone says in their policy that they specifically do not want this then you should not send it), although for my own projects, there are reasons (having to do with the way the version control is set up) that I cannot accept direct contributions anyways, and I usually make my own changes to any contributions anyways (but anyone else can make their own copy with their own changes however they want to do, whether or not I accept it). Since it is honest, and that you had reviewed it by yourself before submitting it, and also tested it, and that I would review it again anyways as well, it would be acceptable, although I would still much more prefer to receive contributions that do not use LLM, still the way you do it is much better than the other stuff some by LLM which does not meet the threshold of being acceptable.

SleekoNiko 11mo ago

If I were the maintainer, and you specified up front that your PR was largely written by an LLM, I would appreciate it. I may prioritize it lower than other PRs, perhaps, but that's about it. I think it's also important to disclose how rigorously you tested your changes, too. I would hate to spend my time looking at a change that was never even tested by a human. It sounds like you do both of these. Judging by the other replies, it seems that other reviewers may take a harsher stance, given the heavily polarized nature of LLMs. Still, if you made the changes and you're up front about your methodology, why not? In the worst case, your PR gets closed and everybody moves on.

ack_complete 11mo ago

> I can't imagine that any policy against LLM code would allow this sort of thing, but I also imagine that if I don't say "this was made by a coding agent", that no one would ever know. So, should I just stop contributing, or start lying? If a project has a stated policy that code written with an LLM-based aid is not accepted, then it shouldn't be submitted, same as with anything else that might be prohibited. If you attempt to circumvent this by hiding it and it is revealed that you knowingly did so in violation of the policy, then it would be unsurprising for you to receive a harsh reply and/or ban, as well as a revert if the PR was committed. This would be the same as any other prohibition, such as submitting code copied from another project with an incompatible license. You could argue that such a blanket ban is unwarranted, and you might be right. But the project maintainers have a right to set the submission rules for their project, even if it rules out high-quality LLM assisted submissions. The right way to deal with this is to ask the project maintainers if they would be willing the adjust the policy, not to try to slip such code into the project anyway.

darkwater 11mo ago

> [append] Getting a lot of hate for this, which I guess is a pretty clear answer. I guess the reason I'm not receiving the "fuck off" clearly is because when I see these threads of people complaining about AI content, it's really clearly low-quality crap that (for example) doesn't even compile, and wastes everyone's time. I think you don't deserve the downvotes and, if you really do what you say you do, that's the ONLY way to use LLMs for coding and contributing to opensource software, or to a company's software. Sadly, the vast majority of LLM users don't and will never use it like that. And while they can get fired for being useless monkeys in a real company, they will keep sending PRs to opensource software, so that's clearly a different scenario that needs a different solution.