6 ms·

I think it’s disrespectful of others to throw generated code their way. They become responsible for it and often donate their time.

by TingPing 11mo ago

CGamesPlay 11mo ago
I think it's disrespectful to throw bad code on somebody else, regardless of its provenance. And (as a professional who uses AI coding agents) I understand that LLM code is often bad code. But the OP isn't complaining about the deluge of bad code being submitted as PRs, they're complaining about LLM code being submitted as PRs. I've also been on the other side of this, receiving some spammy LLM-generated irrelevant "security vulnerabilities", so I also get the desire for some filtering. I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen.
- cyphar 11mo ago
  (OP here.) Well, we don't receive that many low-quality PRs in general (I opened this issue to discuss solutions before it becomes a real problem). Speaking personally, when it does happen I try to help mentor the person to improve their code or (in the case where the person isn't responsive) I sit down and make the improvements I would've made and explain why they were made as a comment in the PR. When it comes to LLM-generated code, I am now going to be going back-and-forth with someone who is probably just going to copy-paste my comments into an LLM (probably not even bothering to read them). It just feels disrespectful. > I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen. Well, this is a two-way street -- all of the LLM-generated PRs and issues I've seen so far do not say that they are LLM-generated, in a way that I am tempted to describe as "dishonest". If every LLM-generated PR was tagged as such, I might have a different outlook on the situation (and might instead be willing to reviewing these issues but with lower priority).
  - CGamesPlay 11mo ago
    Thanks for the response here. I also read your follow-up on the Github issue. I definitely agree with you on the mentorship point. The "CPU quota allocation mismatch" issue felt particularly agitating to read from the point of view as a maintainer. Human being acting as a proxy for ChatGPT. Have you considered putting an "LLM usage disclosure" question on the issue/PR templates? Something like "To what extent were AI tools used to create this issue/PR? What validation steps did you perform to ensure accuracy?" The "hard-line policy" would then shift from being "used LLM tools" to "lied on the LLM usage disclosure", and it feels a lot less like selective enforcement (from my perspective). Obviously it won't stop these spammy issues/PRs, but neither will a hard-line policy against all AI.
    - cyphar 11mo ago
      I've been thinking about this a bit over the past few days, and I think this is a fairly reasonable middle ground (and also allows maintainers to decide how they wish to engage with LLM-generated work).
  - grayhatter 11mo ago
    Hope you can forgive this tangent, I considered posting this in the GH thread, but you asked nicely not to... So hopefully this is a middle ground, you can excuse or ignore First, my original comment was going to ask if you're looked at what any other reputable repos are doing. Specifically popular FOSS projects that are not backed by a company looking to sell AI. Do any of them have a positive Policy, or positions that you want to include? Second, if I was forced to take a stand on AI, I would duplicate the policy from Zig. I feel their policy hits the exact ethos FOSS should strive for. They even ban AI for translations, because the reader is just as capable a participant. And importantly, asking the author to do their best (without AI), and trust the reader to also try their best encourages human communication. It also gives the reader control and knowledge over the exact amount of uncertainty introduced by the LLM, which is critically important to understanding a poor quality bug report from a helpful users who is honestly trying to help. Lobste.rs github disallows AI contribution for an entirely different reason I haven't seen covered in your GH thread yet Finally, you posted the Issue as an RFC. but then explicitly excluded, HN from commenting on the issue. I think that was a fantastic decision, and expertly written. (I also appreciate that lesson in tactfulness :) ) That said, if you're actually interested in requesting comments or thoughts you wouldn't have considered, I would encourage you to make a top level RFC comment this thread. There will likely be a lot of human slop to wade through, but occasionally I'll uncover a genuinely great comment on HN that improves my understanding. Here I think the smart pro-AI crowd that might have an argument I want to consider, but would be unlikely to on my own because of my bias on the quality of AI. Such a comment might would be likely to appear on HN, but the smart people who I'd want to learn from, would never comment on the GH thread now, and I appreciate it when smart people I disagree with, contribute to my understanding. PS Thanks for working on opencontainers, and caring enough to keep trying to make it better, and healthier! I like having good quality software to work with :)

I think it’s disrespectful of others to throw generated code their way. They become responsible for it and often donate their time.

CGamesPlay 11mo ago

I think it's disrespectful to throw bad code on somebody else, regardless of its provenance. And (as a professional who uses AI coding agents) I understand that LLM code is often bad code. But the OP isn't complaining about the deluge of bad code being submitted as PRs, they're complaining about LLM code being submitted as PRs. I've also been on the other side of this, receiving some spammy LLM-generated irrelevant "security vulnerabilities", so I also get the desire for some filtering. I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen.

cyphar 11mo ago
(OP here.) Well, we don't receive that many low-quality PRs in general (I opened this issue to discuss solutions before it becomes a real problem). Speaking personally, when it does happen I try to help mentor the person to improve their code or (in the case where the person isn't responsive) I sit down and make the improvements I would've made and explain why they were made as a comment in the PR. When it comes to LLM-generated code, I am now going to be going back-and-forth with someone who is probably just going to copy-paste my comments into an LLM (probably not even bothering to read them). It just feels disrespectful. > I hope projects don't adopt blanket hard-line "no AI" policies, which will only be selectively enforced against new contributors where the code "smells like" LLM code, but that's what I'm afraid will happen. Well, this is a two-way street -- all of the LLM-generated PRs and issues I've seen so far do not say that they are LLM-generated, in a way that I am tempted to describe as "dishonest". If every LLM-generated PR was tagged as such, I might have a different outlook on the situation (and might instead be willing to reviewing these issues but with lower priority).
- CGamesPlay 11mo ago
  Thanks for the response here. I also read your follow-up on the Github issue. I definitely agree with you on the mentorship point. The "CPU quota allocation mismatch" issue felt particularly agitating to read from the point of view as a maintainer. Human being acting as a proxy for ChatGPT. Have you considered putting an "LLM usage disclosure" question on the issue/PR templates? Something like "To what extent were AI tools used to create this issue/PR? What validation steps did you perform to ensure accuracy?" The "hard-line policy" would then shift from being "used LLM tools" to "lied on the LLM usage disclosure", and it feels a lot less like selective enforcement (from my perspective). Obviously it won't stop these spammy issues/PRs, but neither will a hard-line policy against all AI.
  - cyphar 11mo ago
    I've been thinking about this a bit over the past few days, and I think this is a fairly reasonable middle ground (and also allows maintainers to decide how they wish to engage with LLM-generated work).
- grayhatter 11mo ago
  Hope you can forgive this tangent, I considered posting this in the GH thread, but you asked nicely not to... So hopefully this is a middle ground, you can excuse or ignore First, my original comment was going to ask if you're looked at what any other reputable repos are doing. Specifically popular FOSS projects that are not backed by a company looking to sell AI. Do any of them have a positive Policy, or positions that you want to include? Second, if I was forced to take a stand on AI, I would duplicate the policy from Zig. I feel their policy hits the exact ethos FOSS should strive for. They even ban AI for translations, because the reader is just as capable a participant. And importantly, asking the author to do their best (without AI), and trust the reader to also try their best encourages human communication. It also gives the reader control and knowledge over the exact amount of uncertainty introduced by the LLM, which is critically important to understanding a poor quality bug report from a helpful users who is honestly trying to help. Lobste.rs github disallows AI contribution for an entirely different reason I haven't seen covered in your GH thread yet Finally, you posted the Issue as an RFC. but then explicitly excluded, HN from commenting on the issue. I think that was a fantastic decision, and expertly written. (I also appreciate that lesson in tactfulness :) ) That said, if you're actually interested in requesting comments or thoughts you wouldn't have considered, I would encourage you to make a top level RFC comment this thread. There will likely be a lot of human slop to wade through, but occasionally I'll uncover a genuinely great comment on HN that improves my understanding. Here I think the smart pro-AI crowd that might have an argument I want to consider, but would be unlikely to on my own because of my bias on the quality of AI. Such a comment might would be likely to appear on HN, but the smart people who I'd want to learn from, would never comment on the GH thread now, and I appreciate it when smart people I disagree with, contribute to my understanding. PS Thanks for working on opencontainers, and caring enough to keep trying to make it better, and healthier! I like having good quality software to work with :)