5 ms·

Maintainers and repo owners will get where they want to go the fastest by not referring to what/who "generated" code in a PR. Discussions about AI/LLM code bei

by djoldman 11mo ago

Maintainers and repo owners will get where they want to go the fastest by not referring to what/who "generated" code in a PR. Discussions about AI/LLM code being a problem solely because AI/LLM is not generally a productive conversation. Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc. Additionally, if there isn't a code of conduct, AI policy, or, perhaps most importantly, a policy on how to submit PRs and which are acceptable, it's a huge weakness in a project. In this case, clearly some feathers were ruffled but cool heads prevailed. Well done in the end..

stefantalpalaru 11mo ago
[dead]
rogerrogerr 11mo ago
AI/LLMs are a problem because they create plausible looking code that can pass any review I have time to do, but doesn’t have a brain behind it that can be accountable for the code later. As a maintainer, it used to be I could merge code that “looked good”, and if it did something subtly goofy later I could look in the blame, ping the guy who wrote it, and get a “oh yeah, I did that to flobberate the bazzle. Didn’t think about when the bazzle comes from the shintlerator and is already flobbed” response. People who wrote plausible looking code were usually decent software people. Now, I would get “You’re absolutely right! I implemented this incorrectly. Here’s a completely different set of changes I should have sent instead. Hope this helps!”
- chii 11mo ago
  > doesn’t have a brain behind it that can be accountable for the code later. the submitter could also bail just as easily. Having an AI make the PR or not makes zero difference for this accountability. Ultimately, the maintainer pressing the merge button is accountable. What else would your value be as a maintainer, if all you did was a surface look, press merge, then find blame later when shit hits the fan?
  - rogerrogerr 11mo ago
    I don’t accept giant contributions from people who don’t have track records of sticking around. It’s faster for me to write something myself than review huge quantities of outsider code as a zero-trust artifact.
  - ares623 11mo ago
    If I had a magic wand I would wish for 2 parallel open source communities diverging from today. One path continues on the track it has always been on, human written and maintained. The other is fully on the AI track. Massive PRs with reviewers rubber stamping them. I’d love to see which track comes out ahead. Edit: in fact, perhaps there are open source projects already fully embracing AI authored contributions?
    - ctenb 11mo ago
      I agree. It would also work out like a long term supervised learning process though. Humans showing how it's really done, and AI companies taking that as a gold standard for training and development of AI.
      - ares623 11mo ago
        I'm not so sure. There's already decades of data available for the existing process.
        ctenb 11mo ago
        That is true, but it doesn't help for new languages, frameworks, etc
    - jebarker 11mo ago
      How would you define “ahead”?
      - forgetfulness 11mo ago
        Able to make changes preserving correctness over time Vibecoding reminds me sharply of the height of the Rails hype, products quickly rushed to market off the backs of a slurry of gems and autoimports inserted on generated code, the original authors dipping and teams of maintainers then screeching into a halt Here the bots will pigheadedly heap one 9000 lines PR onto another, shredding the code base to bits but making it look like a lot of work in the process
snickerbockers 11mo ago
I don't suppose you saw the post where OP asked claude to explain why this patch was not plagiarized? It's pretty damning.
- lambda_foo 11mo ago
  Why have the OP in the loop at all if he’s just sending prompts to AI? Surely it’s a wonderful piece of performance art.
  - footy 11mo ago
    it reads like humiliation fetish material honestly. I'd delete my account but he just doubles down.
    - pluc 11mo ago
      He's doing it elsewhere too: https://github.com/rerun-io/rerun/pull/11900#issuecomment-3563604774 https://github.com/rerun-io/rerun/pull/11900#issuecomment-35... https://github.com/ocaml/dune/issues/12731 https://github.com/ocaml/dune/issues/12731 https://github.com/tshort/StaticCompiler.jl/pull/180 https://github.com/tshort/StaticCompiler.jl/pull/180 Seems he's just on a rampage of "fixing" issues for trendy packages to get some attention.
      - footy 11mo ago
        > I like a tough challenge and I was hoping to attract your attention. thanks for the comedy material.
      - joelreymont 11mo ago
        I had $1000 in Claude credits to spend for the greater good.
        pluc 11mo ago
        You didn't
deleted 11mo ago
[deleted]
armchairhacker 11mo ago
I agree, but @gasche brings up real points in https://github.com/ocaml/ocaml/pull/14369#issuecomment-3556593972 https://github.com/ocaml/ocaml/pull/14369#issuecomment-35565.... In particular I found these important: - Copyright issues. Even among LLM-generated code, this PR is particularly suspicious, because some files begin with the comment “created by [someone’s name]” - No proposal. Maybe the feature isn’t useful enough to be worth the tech debt, maybe the design doesn’t follow conventions and/or adds too much tech debt - Not enough tests - The PR is overwhelmingly big, too big for the small core team that maintains OCaml - People are already working on this. They’ve brainstormed the design, they’re breaking the task into smaller reviewable parts, and the code they write is trusted more than LLM-generated code Later, @bluddy mentions a design issue: https://github.com/ocaml/ocaml/pull/14369#issuecomment-3556864936 https://github.com/ocaml/ocaml/pull/14369#issuecomment-35568...
williamdclt 11mo ago
> Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc. They did: the main point being made is "I'm not reading 13k LOCs when there's been no proposal and discussion that this is something we might want, and how we might want to have it implemented". Which is an absolutely fair point (there's no other possible answer really, unless you have days to waste) whether the code is AI-written or human-written.
- Anamon 11mo ago
  Exactly, this seems a bit overlooked in this discussion. A PR like this would NOT have been okay even if there was no LLM involved. It reminds me of a PR I once saw (don't remember which project) in which a first-time contributor opened a PR rewriting the project's entire website in their favourite new framework. The maintainers calmly replied to the effect of, before putting in the work, it might have been best to quickly check if we even want this. The contributor liked the framework so much that I'm sure they believed it was an improvement. But it's the same tone-deafness I now see in many vibe coders who don't seem to understand that OSS projects involve other people and demand some level of consensus and respect.
  - pepoluan 11mo ago
    I am one of the maintainers of aiosmtpd [1], and the largest PR I ever made was migrating the library's tests from nosetest to pytest. Before doing that, though, I discussed with the other maintainers if such a migration is welcome. And after getting support from them, I made the changes with gusto. It took weeks, even months to complete and the PR is massive [2] But still the crux of the matter is: Massive changes require buy-in from other maintainers BEFORE the changes even start. [1] https://github.com/aio-libs/aiosmtpd https://github.com/aio-libs/aiosmtpd [2] https://github.com/aio-libs/aiosmtpd/pull/202 https://github.com/aio-libs/aiosmtpd/pull/202
abathologist 11mo ago
For example "cites a different person as an author, who happened to have done all the substantive work on a related code base". ;)
shizzy0 11mo ago
I think it's deeply disadvantageous and legally dubious to accept code for which you don't know its provenance.

stefantalpalaru 11mo ago

[dead]

rogerrogerr 11mo ago

AI/LLMs are a problem because they create plausible looking code that can pass any review I have time to do, but doesn’t have a brain behind it that can be accountable for the code later. As a maintainer, it used to be I could merge code that “looked good”, and if it did something subtly goofy later I could look in the blame, ping the guy who wrote it, and get a “oh yeah, I did that to flobberate the bazzle. Didn’t think about when the bazzle comes from the shintlerator and is already flobbed” response. People who wrote plausible looking code were usually decent software people. Now, I would get “You’re absolutely right! I implemented this incorrectly. Here’s a completely different set of changes I should have sent instead. Hope this helps!”

chii 11mo ago
> doesn’t have a brain behind it that can be accountable for the code later. the submitter could also bail just as easily. Having an AI make the PR or not makes zero difference for this accountability. Ultimately, the maintainer pressing the merge button is accountable. What else would your value be as a maintainer, if all you did was a surface look, press merge, then find blame later when shit hits the fan?
- rogerrogerr 11mo ago
  I don’t accept giant contributions from people who don’t have track records of sticking around. It’s faster for me to write something myself than review huge quantities of outsider code as a zero-trust artifact.
- ares623 11mo ago
  If I had a magic wand I would wish for 2 parallel open source communities diverging from today. One path continues on the track it has always been on, human written and maintained. The other is fully on the AI track. Massive PRs with reviewers rubber stamping them. I’d love to see which track comes out ahead. Edit: in fact, perhaps there are open source projects already fully embracing AI authored contributions?
  - ctenb 11mo ago
    I agree. It would also work out like a long term supervised learning process though. Humans showing how it's really done, and AI companies taking that as a gold standard for training and development of AI.
    - ares623 11mo ago
      I'm not so sure. There's already decades of data available for the existing process.
      - ctenb 11mo ago
        That is true, but it doesn't help for new languages, frameworks, etc
  - jebarker 11mo ago
    How would you define “ahead”?
    - forgetfulness 11mo ago
      Able to make changes preserving correctness over time Vibecoding reminds me sharply of the height of the Rails hype, products quickly rushed to market off the backs of a slurry of gems and autoimports inserted on generated code, the original authors dipping and teams of maintainers then screeching into a halt Here the bots will pigheadedly heap one 9000 lines PR onto another, shredding the code base to bits but making it look like a lot of work in the process

snickerbockers 11mo ago

I don't suppose you saw the post where OP asked claude to explain why this patch was not plagiarized? It's pretty damning.

lambda_foo 11mo ago
Why have the OP in the loop at all if he’s just sending prompts to AI? Surely it’s a wonderful piece of performance art.
- footy 11mo ago
  it reads like humiliation fetish material honestly. I'd delete my account but he just doubles down.
  - pluc 11mo ago
    He's doing it elsewhere too: https://github.com/rerun-io/rerun/pull/11900#issuecomment-3563604774 https://github.com/rerun-io/rerun/pull/11900#issuecomment-35... https://github.com/ocaml/dune/issues/12731 https://github.com/ocaml/dune/issues/12731 https://github.com/tshort/StaticCompiler.jl/pull/180 https://github.com/tshort/StaticCompiler.jl/pull/180 Seems he's just on a rampage of "fixing" issues for trendy packages to get some attention.
    - footy 11mo ago
      > I like a tough challenge and I was hoping to attract your attention. thanks for the comedy material.
    - joelreymont 11mo ago
      I had $1000 in Claude credits to spend for the greater good.
      - pluc 11mo ago
        You didn't

deleted 11mo ago

[deleted]

armchairhacker 11mo ago

I agree, but @gasche brings up real points in https://github.com/ocaml/ocaml/pull/14369#issuecomment-3556593972 https://github.com/ocaml/ocaml/pull/14369#issuecomment-35565.... In particular I found these important: - Copyright issues. Even among LLM-generated code, this PR is particularly suspicious, because some files begin with the comment “created by [someone’s name]” - No proposal. Maybe the feature isn’t useful enough to be worth the tech debt, maybe the design doesn’t follow conventions and/or adds too much tech debt - Not enough tests - The PR is overwhelmingly big, too big for the small core team that maintains OCaml - People are already working on this. They’ve brainstormed the design, they’re breaking the task into smaller reviewable parts, and the code they write is trusted more than LLM-generated code Later, @bluddy mentions a design issue: https://github.com/ocaml/ocaml/pull/14369#issuecomment-3556864936 https://github.com/ocaml/ocaml/pull/14369#issuecomment-35568...

williamdclt 11mo ago

> Better is to critique the actual PR itself. For example, needs more tests, needs to be broken up, doesn't follow our protocols for merging/docs, etc. They did: the main point being made is "I'm not reading 13k LOCs when there's been no proposal and discussion that this is something we might want, and how we might want to have it implemented". Which is an absolutely fair point (there's no other possible answer really, unless you have days to waste) whether the code is AI-written or human-written.

Anamon 11mo ago
Exactly, this seems a bit overlooked in this discussion. A PR like this would NOT have been okay even if there was no LLM involved. It reminds me of a PR I once saw (don't remember which project) in which a first-time contributor opened a PR rewriting the project's entire website in their favourite new framework. The maintainers calmly replied to the effect of, before putting in the work, it might have been best to quickly check if we even want this. The contributor liked the framework so much that I'm sure they believed it was an improvement. But it's the same tone-deafness I now see in many vibe coders who don't seem to understand that OSS projects involve other people and demand some level of consensus and respect.
- pepoluan 11mo ago
  I am one of the maintainers of aiosmtpd [1], and the largest PR I ever made was migrating the library's tests from nosetest to pytest. Before doing that, though, I discussed with the other maintainers if such a migration is welcome. And after getting support from them, I made the changes with gusto. It took weeks, even months to complete and the PR is massive [2] But still the crux of the matter is: Massive changes require buy-in from other maintainers BEFORE the changes even start. [1] https://github.com/aio-libs/aiosmtpd https://github.com/aio-libs/aiosmtpd [2] https://github.com/aio-libs/aiosmtpd/pull/202 https://github.com/aio-libs/aiosmtpd/pull/202

abathologist 11mo ago

For example "cites a different person as an author, who happened to have done all the substantive work on a related code base". ;)

shizzy0 11mo ago

I think it's deeply disadvantageous and legally dubious to accept code for which you don't know its provenance.