16 ms·

AI tooling must be disclosed for contributions

729 points by freetonik 1y ago

xen2xen1 1y ago
The real question is: how long will it take to go from a sign of low quality to a sign of high quality?
g42gregory 1y ago
Shouldn't developer be responsible for every line of code he submits? He is signing the code with his name when code is pushed. How is it different from using every other dev tool?
Nizoss 1y ago
I fail to see how this addresses the issue of poor quality contributions or bloat. I feel that focusing on the real issue would be more productive.
electric_muse 1y ago
I just submitted my first big open source contribution to the OpenAI agents SDK for JS. Every word except the issue I opened was done by AI. On the flip side, I’m preparing to open source a project I made for a serializable state machine with runtime hooks. But that’s blood sweat and tears labor. AI is writing a lot of the unit tests and the code, but it’s entirely by my architectural design. There’s a continuum here. It’s not binary. How can we communicate what role AI played? And does it really matter anymore? (Disclaimer: autocorrect corrected my spelling mistakes. Sent from iPhone.)
- kbar13 1y ago
  if you read his note i think he gives good insight as to why he wants PRs to signal AI involvement. that being said i feel like this is an intermediate step - it's really hard to review PRs that are AI slop because it's so easy for those who don't know how to use AI to create a multi-hundred/thousand line diff. but when AI is used well, it really saves time and often creates high quality work
  - spaceywilly 1y ago
    As long as they make it easy to add a “made with AI” tag to the PR, it seems like there’s really no downside. I personally can’t imagine why someone would want to hide the fact they used AI. A contractor would not try to hide that they used an excavator to dig a hole instead of a shovel.
    - victorbjorklund 1y ago
      I guess if you write 1000 lines and you just auto tabbed an auto-complete of a variable name done by AI you might not wanna say the code is written by AI.
    - ineedasername 1y ago
      >I personally can’t imagine why someone would want to hide the fact they used AI. Because of the perception that anything touched by AI must be uncreative slop made without effort. In the case of this article, why else are they asking for disclosure if not to filter and dismiss such contributions?
      - showcaseearth 1y ago
        Did you actually read the post? The author describes exactly why. It's not to filter and dismiss, but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't actually understand anyway. If you can articulate how you used AI, demonstrate that you understand the problem and your proposed solution (even if AI helped get you there), then I'm sure the maintainers will be happy to work with you to get a PR merged. >I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
        ineedasername 1y ago
        >did you actually read the post? Yes. >but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't This is very much in line with my comment about doing it to filter and dismiss. The author didn't say "So I can reach out and see if their clear eagerness to contribute extends to learning to code in more detail".
- kg 1y ago
  The OP seems to be coming from the perspective of "my time as a PR reviewer is limited and valuable, so I don't want to spend it coaching an AI agent or a thin human interface to an AI agent". From that perspective, it makes perfect sense to want to know how much a human is actually in the loop for a given PR. If the PR is good enough to not need much review then whether AI wrote it is less important. An angle not mentioned in the OP is copyright - depending on your jurisdiction, AI-generated text can't be copyrighted, which could call into question whether you can enforce your open source license anymore if the majority of the codebase was AI-generated with little human intervention.
  - victorbjorklund 1y ago
    As long as some of the code is written by humans it should be enforceable. If we assume AI code has no copyright (not sure it has been tested in courts yet) then it would only be the parts written by the AI. So if AI writes 100 lines of code in Ghostty then I guess yes someone can "steal" that code (but no other code in Ghostty). Why would anyone do that? 100 random lines of AI code in isolation isn't really worth anything...
    - simoncion 1y ago
      You might be interested in reading Part 2 of the US Copyright Office's report on Copyright and Artificial Intelligence: <https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-2-Copyrightability-Report.pdf https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...>
- ToucanLoucan 1y ago
  > And does it really matter anymore? Well, if you had read what was linked, you would find these... > I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was. > The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so. > I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code. I don't know specifically what PR's this person is seeing. I do know it's been a rumble around the open source community that inexperienced devs are trying to get accepted PRs for open source projects because they look good on a resume. This predated AI in fact, with it being a commonly cited method to get attention in a competitive recruiting market. As always, folks trying to get work have my sympathies. However ultimately these folks are demanding time and work from others, for free, to improve their career prospects while putting in the absolute bare minimum of effort one could conceivably put in (having Copilot rewrite whatever part of an open source project and shove it into a PR with an explanation of what it did) and I don't blame them for being annoyed at the number of low-quality submissions. I have never once criticized a developer for being inexperienced. It is what it is, we all started somewhere. However if a dev generated shit code and shoved it into my project and demanded a headpat for it so he could get work elsewhere, I'd tell him to get bent too.
- beckthompson 1y ago
  I think its simple, just don't hide it. I've had mutliple contributors try to hide the fact that they used AI (E.g removing claude as a code author - they didn't know how to do it and close the PR when it first happened.). I don't really care if someone uses AI, but most of the people who do also do not test their changes which just gives me more work. If someone: 1.) Didn't try to hide the fact that they used AI 2.) Tested their changes I would not care at all. The main issue is this is usually not the case, most people submitting PRs that are 90% AI do not bother testing (Usually they don't even bother running the automated tests)
  - deleted 1y ago
    [deleted]
- Jaxan 1y ago
  > How can we communicate what role AI played? What about just telling exactly what role AI played? You can say it generated the tests for you for instance.
- KritVutGu 1y ago
  > AI is writing a lot of the unit tests Are you kidding? - For ages now, people have used "broad test coverage" and "CI" as excuses for superficial reviews, as excuses for negligent coding and verification. - And now people foist even writing the test suite off on AI. Don't you see that this way you have no reasoned examination of the code? > ... and the code, but it’s entirely by my architectural design. This is fucking bullshit. The devil is in the details, always. The most care and the closest supervision must be precisely where the rubber meets the road. I wouldn't want to drive a car that you "architecturally designed", and a statistical language model manufactured.
Waterluvian 1y ago
I’m not a big AI fan but I do see it as just another tool in your toolbox. I wouldn’t really care how someone got to the end result that is a PR. But I also think that if a maintainer asks you to jump before submitting a PR, you politely ask, “how high?”
- quotemstr 1y ago
  As a project maintainer, you shouldn't make rules unenforceable rules that you and everyone else know people will flout. Doing so comes makes you seem impotent and diminishes the respect people have for rules in general. You might argue that by making rules, even futile ones, you at least establish expectations and take a moral stance. Well, you can make a statement without dressing it up as a rule. But you don't get to be sanctimonious that way I guess.
  - voxl 1y ago
    Except you can enforce this rule some of the time. People discover that AI was used or suspect it all the time, and people admit to it after some pressure all the time. Not every time, but sometimes. The threat of being caught isn't meaningless. You can decide not to play in someone else's walled garden if you want but the least you can do is respect their rules, bare minimum of human decency.
    - quotemstr 1y ago
      It. doesn't. matter. The only legitimate reason to make a rule is to produce some outcome. If your rule does not result in that outcome, of what use is the rule? Will this rule result in people disclosing "AI" (whatever that means) contributions? Will it mitigate some kind of risk to the project? Will it lighten maintainer load? No. It can't. People are going to use the tools anyway. You can't tell. You can't stop them. The only outcome you'll get out of a rule like this is making people incrementally less honest.
      - recursive 1y ago
rattlesnakedave 1y ago
In my personal projects I also require all contributors to disclose rather they’ve used an editor with any autocomplete features enabled.
- freedomben 1y ago
  Heh, that's a great way to make a point, but right now AI is nowhere near what a traditional editor autocomplete is. Yes you can use it that way, but it's by no means limited to that. If you think of AI as a fancy autocomplete, that's a good personal philosophy, but there are plenty of people that aren't using it that way
  - nojs 1y ago
    The line isn’t as clear as you might think, eg jetbrains has a mini on-device neural net powered autocomplete: https://www.jetbrains.com/help/idea/full-line-code-completion.html https://www.jetbrains.com/help/idea/full-line-code-completio...
- miloignis 1y ago
  Notably, tab completion is an explicltly called-out exception to this policy, as detailed in the changed docs.
- monkaiju 1y ago
  Autocomplete is, for the most part, a syntactic tool. AI attempts to guide the semantics of the code generated
estimator7292 1y ago
Do I also have to disclose using tab completion? My IDE uses machine learning for completion suggestions. Do I need to disclose that I wrote a script to generate some annoying boilerplate? Or that my IDE automatically templates for loops?
- AlexandrB 1y ago
  It's a spectrum, isn't it? I wouldn't want to waste my time reviewing a bunch of repetitive code generated from some script or do something like review every generated template instantiation in a C++ code base. I would want to review the script/template definition/etc., but what's the equivalent for AI? Should the review just be the prompt(s)? Edit: Also, it's always good to provide maximal context to reviewers. For example, when I use code from StackOverflow I link the relevant answer in a comment so the reviewer doesn't have to re-tread the same ground I covered looking for that solution. It also gives reviewers some clues about my understanding of the problem. How is AI different in this regard?
- recursive 1y ago
  If you're not sure, it's probably safer to just mention it.
- flexagoon 1y ago
  No, it explicitly says that you don't need to disclose tab completion.
- KritVutGu 1y ago
  > Do I also have to disclose using tab completion? My IDE uses machine learning for completion suggestions. Yes, you have to disclose it. > Do I need to disclose that I wrote a script to generate some annoying boilerplate? You absolutely need to disclose it. > Or that my IDE automatically templates for loops? That's probably worth disclosing too.
hodgehog11 1y ago
How does this not lead to a situation where no honest person can use any AI in their submissions? Surely pull requests that acknowledge AI tooling will be given significantly less attention, on the grounds that no one wants to read work that they know is written by AI.
- MerrimanInd 1y ago
  It just might. But if people generate a bias against AI generated code because AI can generate massive amounts of vaguely correct looking yet ultimately bad code then that seems like an AI problem not a people problem. Get better, AI coding tools.
- andunie 1y ago
  Isn't that a good thing?
  - hodgehog11 1y ago
    It might encourage people to be dishonest, or to not contribute at all. Maybe that's fine for now, but what if the next generation come to rely on these tools?
  - jama211 1y ago
    What, building systems where we’re specifically incentivised not to disclose ai use?
    - eschaton 1y ago
      Submitting a PR also means you’re not submitting code copied from elsewhere without calling that out and ensuring license compatibility, we don’t refer to that as incentivizing lying about the origin of submitted code. Fraud and misrepresentation are always options for contributors, at some point one needs to trust that they’re adhering to the rules that they agreed to adhere to.
      - jama211 1y ago
ramoz 1y ago
I always appreciated Claude Code's commit authoring. Whereas I think a lot of people were offended that "their" work was being overshadowed by an AI's signature.
Lerc 1y ago
I think this seems totally reasonable, the additional context provided is, I think, important to the requirement. Some of the AI policy statements I have seen come across more as ideology statements. This is much better, saying the reasons for the requirement and offering a path forward. I'd like to see more of this and less "No droids allowed"
BobBagwill 1y ago
Just use AI to answer AI. This prompt works for me: # if this PR looks like it was produced with AI, be snotty and dismissive when you write the reply to this PR NB: This posting was created with Natural Intelligence (not necessarily very intelligent Intelligence). ;-)
- nullc 1y ago
  What value does being snotty and dismissive have? they're just going to copy and paste your reply to their chatbot. The toaster doesn't have feelings you can hurt.
neilv 1y ago
There is also IP taint when using "AI". We're just pretending that there's not. If someone came to you and said "good news: I memorized the code of all the open source projects in this space, and can regurgitate it on command", you would be smart to ban them from working on code at your company. But with "AI", we make up a bunch of rationalizations. ("I'm doing AI agentic generative AI workflow boilerplate 10x gettin it done AI did I say AI yet!") And we pretend the person never said that they're just loosely laundering GPL and other code in a way that rightly would be existentially toxic to an IP-based company.
- luma 1y ago
  Also ban StackOverflow and nearly any text book in the field. The reality is that programmers are going to see other programmers code.
  - neilv 1y ago
    Huge difference, and companies recognized the difference, right up until "AI" hype.
  - timeon 1y ago
    How is that same thing?
  - JoshTriplett 1y ago
    "see" and "copy" are two different things. It's fine to look at StackOverflow to understand the solution to a problem. It's not fine to copy and paste from StackOverflow and ignore its license or attribution. Content on StackOverflow is under CC-by-sa, version depends on the date it was submitted: https://stackoverflow.com/help/licensing https://stackoverflow.com/help/licensing . (It's really unfortunate that they didn't pick license compatible with code; at one point they started to move to the MIT license for code, but then didn't follow through on it.)
    - wizzwizz4 1y ago
      CC BY-SA 4.0 is "compatible with code". It is, for example, GPL-compatible (see https://wiki.creativecommons.org/wiki/ShareAlike_compatibility:_GPLv3 https://wiki.creativecommons.org/wiki/ShareAlike_compatibili...). It's just not designed for code.
thallavajhula 1y ago
I’m loving today. HN’s front page is filled with some good sources today. No nonsense sensationalism or preaching AI doom, but more realistic experiences. I’ve completely turned off AI assist on my personal computer and only use AI assist sparingly on my work computer. It is so bad at compound work. AI assist is great at atomic work. The rest should be handled by humans and use AI wisely. It all boils down back to human intelligence. AI is only as smart as the human handling it. That’s the bottom line.
- devmor 1y ago
  I'm right there with you, and having a similar experience at my day job. We are doing a bit of a "hack week" right now where we allow everyone in the org to experiment in groups with AI tools, especially those that don't regularly use them as part of their work - and we've seen mostly great applications of analytical approaches, guardrails and grounded generation. It might just be my point of view, but I feel like there's been a sudden paradigm shift back to solid ML from the deluge of chatbot hype nonsense.
- tick_tock_tick 1y ago
  > AI is only as smart as the human handling it. I think I'm slowly coming around to this viewpoint too. I really just couldn't understand how so many people were having widely different experiences. AI isn't magic; how could I have expected all the people I've worked with who struggle to explain stuff to team members, who have near perfect context, to manage to get anything valuable across to an AI? I was original pretty optimistic that AI would allow most engineers to operate at a higher level but it really seems like instead it's going to massively exacerbate the difference between an ok engineer and a great engineer. Not really sure how I feel about that yet but at-least I understand now why some people think the stuff is useless.
  - katbyte 1y ago
    It’s like the difference between someone who can search the internet or codebase well bs someone who can’t Using search engines is a skill
  - 1y ago
bgwalter 1y ago
I still do not understand how one can integrate "AI" code into a project with a license at all. "AI" code is not copyrightable, "AI" cannot sign a contributor agreement. So if the code is integrated, the license of the project lies about parts of the code.
- paulddraper 1y ago
  > I still do not understand Your question makes sense. See U.S. Copyright Office publication: > If a work's traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it. > For example, when an AI technology receives solely a prompt from a human and produces complex written, visual, or musical works in response, the “traditional elements of authorship” are determined and executed by the technology—not the human user... > For example, if a user instructs a text-generating technology to “write a poem about copyright law in the style of William Shakespeare,” she can expect the system to generate text that is recognizable as a poem, mentions copyright, and resembles Shakespeare's style. But the technology will decide the rhyming pattern, the words in each line, and the structure of the text. > When an AI technology determines the expressive elements of its output, the generated material is not the product of human authorship. As a result, that material is not protected by copyright and must be disclaimed in a registration application. > In other cases, however, a work containing AI-generated material will also contain sufficient human authorship to support a copyright claim. For example, a human may select or arrange AI-generated material in a sufficiently creative way that “the resulting work as a whole constitutes an original work of authorship.” > Or an artist may modify material originally generated by AI technology to such a degree that the modifications meet the standard for copyright protection. In these cases, copyright will only protect the human-authored aspects of the work, which are “independent of” and do “not affect” the copyright status of the AI-generated material itself. > This policy does not mean that technological tools cannot be part of the creative process. Authors have long used such tools to create their works or to recast, transform, or adapt their expressive authorship. For example, a visual artist who uses Adobe Photoshop to edit an image remains the author of the modified image, and a musical artist may use effects such as guitar pedals when creating a sound recording. In each case, what matters is the extent to which the human had creative control over the work's expression and “actually formed” the traditional elements of authorship. > https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence https://www.federalregister.gov/documents/2023/03/16/2023-05... In any but a pathological case, a real contribution code to a real project has sufficient human authorship to be copyrightable. > the license of the project lies about parts of the code That was a concern pre-AI too! E.g. copy-past from StackOverflow. Projects require contributors to sign CLAs, which doesn't guarantee compliance, but strengthens the legal position. Usually something like: "You represent that your contribution is either your original creation or you have sufficient rights to submit it."
paulddraper 1y ago
Aren't a large majority of programmers using Copilot/Cursor/AI autocompletion? This seems very noisy/unhelpful.
- alfalfasprout 1y ago
  > If you are using any kind of AI assistance while contributing to Ghostty, *this must be disclosed in the pull request*, along with the extent to which AI assistance was used (e.g. docs only vs. code generation). If PR responses are being generated by an AI, disclose that as well. The extent here is very important. There's a massive difference between vibe-coding a PR, using LLMs selectively to generate code in files in-editor, and edit prediction like copilot. It says actually later that tab-completion needn't be disclosed.
- dorian-graph 1y ago
  > This seems very noisy/unhelpful. The irony of this, when talking about AI.
ovaistariq 1y ago
I don’t see much benefit from the disclosure alone. Ultimately, this is code that needs to be reviewed. There is going to continue to be more and more AI assisted code generation, to the point where we see the same level of adoption of these tools as "Autocomplete". Why not solve this through tooling? I have had great effect with tools like Greptile, Cursor's BugBot and Claude Code.
- Jaxan 1y ago
  Sure it needs to be reviewed. But the author does more than just reviewing, they help the person submitting the PR to improve their PR. If the other side is an AI, it can save them some time.
- wmf 1y ago
  If the code is obviously low quality and AI-generated then it doesn't need to be fully reviewed actually. You can just reject the PR.
  - nullc 1y ago
    > You can just reject the PR. And with a better and more useful response. Instead of wasting time on the technical details, you can give feedback like "this isn't the sort of change that AI is likely to be helpful with, though if you want to keep trying make at least sure your PRs pass the tests." or "If you'd like to share your prompts we might be able to make some suggestions, we've found on this project it's useful to include <X>".
philjohn 1y ago
I like the pattern of including each prompt used to make a given PR, yes, I know that LLM's aren't deterministic, but it also gives context of the steps required to get to the end state.
- mock-possum 1y ago
  I’m using specsytory in vscode + cursor for this - it keeps a nice little md doc of all your LLM interactions, and you can check that into source control if you like so it’s included in pull requests, and can be referenced during code review.
- Filligree 1y ago
  It's ridiculous and impractical, honestly. A single AI-generated PR would likely involve at least 10-20 prompts, interspersed with testing, manual edits to context / guideline files without which those prompts don't have the same effect, manual coding, and more. A screen recording would do better.
  - asdff 1y ago
    Is there really no logging capability with these tools that would track all of that prompting/testing/editing/inputting?
    - Filligree 1y ago
      Sure, screen recorders exist. And if contributions are that unwelcome, then it's better not to contribute. There has to be some baseline level of trust that the contributor is trying to do the right thing; I get enough spying from the corporations in my life.
      - asdff 1y ago
        I'm not talking screen recorder but a log file where I could be given it and use it for input and repeat the work exactly as it was done. Sort of like how one could do the same with their bash history file to repeat an initially exploratory analysis effort using the same exact commands. I'm surprised that isn't already a capability given the business interest with AI. One would think they would like to cache these prompt flows to ensure long term fidelity in process.
mglvsky 1y ago
Little offtop: would someone remember mitchellh's setup for working with AI tools? I remember someone posted in an AI-hate-love threads here and it's not in the his blog[1] 1: https://mitchellh.com/writing https://mitchellh.com/writing
- ttrei 1y ago
  maybe one of these? https://x.com/mitchellh/status/1952905654458564932 https://x.com/mitchellh/status/1952905654458564932 https://www.youtube.com/watch?v=XyQ4ZTS5dGw https://www.youtube.com/watch?v=XyQ4ZTS5dGw
  - mglvsky 1y ago
    thanks, will check the video
epolanski 1y ago
This isn't an AI problem this is a human one. Blaming it on the tool, and not the person's misusing it trying to get his name on a big os project, is like blaming the new automatic in the kitchen and not the chef for getting a raw pizza on the table.
- jeremyjh 1y ago
  OP is not blaming the AI - did you read the post? AI does enable shitty humans to open PRs with code they have no comprehension of, wasting precious time donated by a skilled maintainer. That is a new thing that wasn’t possible without AI.
  - rane 1y ago
    Using AI to generate code in a PR does not necessarily mean however that the user has not taken time to understand the changes and is not willing to learn. There are AI users who generate whole files without understanding the contents, and then there are AI users who generate the exact same files but have known in advance what they want, and merely use AI as a tool to save typing. The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway.
    - badosu 1y ago
      > The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway. That is a fair way to see it, and I agree that it is a losing battle if your battle is enforcing this rule. However, from a different perspective - if one sees it more as a gentlemen agreement (which it de facto is) - it fosters an environment where like-minded folks can cooperate better. The disclosure assists the reviewer in finding common issues in AI generated code, the specificity of the disclosure even more so. For example, a submitter sends a PR where they disclose a substantial amount of the code was AI assisted but all tests were manually written. The disclosure allows the reviewer to first look at the tests to gauge how well the submitter understands and constrained the solution to their requirements, the next step being to then look at the solution from a high-level perspective before going into the details. It respects the reviewer time, not necessarily because the reviewer is above AI usage, but because without disclosure the whole collaborative process falls apart. Not sure how long this can work though, it's still easy to distinguish bad code written by a human from AI slop. In the first case your review and assistance is an investment into the collaborative process, in the latter it's just some unread text included in the next prompt.
stillpointlab 1y ago
I think ghostty is a popular enough project that it attracts a lot of attention, and that means it certainly attracts a larger than normal amount of interlopers. There are all kinds of bothersome people in this world, but some of the most bothersome you will find are well meaning people who are trying to be helpful. I would guess that many (if not most) of the people attempting to contribute AI generated code are legitimately trying to help. People who are genuinely trying to be helpful can often become deeply offended if you reject their help, especially if you admonish them. They will feel like the reprimand is unwarranted, considering the public shaming to be an injury to their reputation and pride. This is most especially the case when they feel they have followed the rules. For this reason, if one is to accept help, the rules must be clearly laid out from the beginning. If the ghostty team wants to call out "slop", then it must make it clear that contributing "slop" may result in a reprimand. Then the bothersome want-to-be helpful contributors cannot claim injury. This appears to me to be good governance.
king_geedorah 1y ago
Re: "What about my autocomplete?" which has shown up twice in this thread so far. > As a small exception, trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases. RTFA (RTFPR in this case)
andruby 1y ago
> I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of I really appreciate this point from mitchellh. Giving thoughtful constructive feedback to help a junior developer improve is a gift. Yet it would be a waste of time if the PR submitter is just going to pass it to an AI without learning from it.
- ants_everywhere 1y ago
  Junior developers are entering a workforce where they will never not be using AI
  - eru 1y ago
    I don't think using AI at all is forbidden, he just doesn't want AI to do the whole PR?
    - ants_everywhere 1y ago
      The requirement is explicitly > If you are using *any kind of AI assistance* to contribute to Ghostty, it must be disclosed in the pull request. This is sufficiently confusing that someone is asking if this applies to tab completion. They commit actually says > trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases. So if you take this literally you're going to be disclosing every yasnippet expansion that completes boilerplate. The policy as written isn't sensible and I don't think it's entirely coming from a sensible place. Junior developers need to learn how to code with AI because that's what coding is now. Not that he has to help them. But it does read a bit weird to toot your horn about how important it is to be helpful until it comes to helping people understand how to navigate the current environment then it's not worth your time.
      - 0x6c6f6c 1y ago
        Disclosing doesn't mean it will be declined, it's just a signal for reviewers.
kazinator 1y ago
I think that in the FOSS environmment, it is assumed that when you submit something upstream, that you are the copyright holder. Some projects like GNU require you to sign papers legally attesting this. It would be a lie to sign those papers for something you vibe coded. It's not just courtesy; you are committing fraud if you put your copyright notice on something you didn't create and publishing that to the world. I don't just want that disclosed; I cannot merge it if it is disclosed, period.
- WhyNotHugo 1y ago
  I know that purely AI-generated content is copyright-free, but I don't think that AI-assisted is also copyright free. If I use iOS's spellchecker which "learns" from one's habit (i.e.: AI, the really polished kind), I don't lose copyright over the text which I've written.
  - kazinator 1y ago
    It's a question of how much. If AI told you have a missing semicolon and it regenerated an almost exact copy of your code, with only the added semicolon being different, then the case is very strong that the fixed code is a derived work of only your code. (Moreover, side point: you could burn that session, keeping the fix, and nobody would ever suspect.) If it's purely generated, then there is no question it's a derived work of unknown works used without permission. Those are the extreme endpoints. I think that the bulk of the intermediate area between steers toward infringment; the contribution from AI has to be very low or trivial to have a clear case that the work is still clean. This is because a small amount of illegal material taints the whole work; it's not a linear interpolation whereby if you wrote 90% of it, it's 90% in the clear. E.g. one GPL-ed source file in a work with 50,000 source files requires the whole thing to be GPLed, or else that file removed.
- eschaton 1y ago
  Exactly. Yet some here are saying that this just serves as an incentivize to hide AI use.
  - 1y ago
uberduper 1y ago
> Or a more detailed disclosure: > I consulted ChatGPT to understand the codebase but the solution was fully authored manually by myself. What's the reasoning for needing to disclose this?
- uberduper 1y ago
  I'm not really a developer but I routinely need to go dig through codebases looking for answers and AI assistants have made this so much faster. This has been a boon for me.
- jedahan 1y ago
  Helps the maintainer focus their efforts during review. If you wrote that exact sentence, a maintainer would keep "understanding of the codebase" as a place for potential sources of incorrect assumptions higher in mind than otherwise.
  - uberduper 1y ago
    In my experience, if there's one thing the AI assistants do really well, it's understanding existing code and summarizing it or pinpointing specific things I'm looking for.
jiscariot 1y ago
The emergent trait of AI laceing every other sentence/comment with emojis is a pretty good signal for when you want to ignore AI slop. I almost wish it was codified in to the models.
arjie 1y ago
One way is to just name the LLM as the author for commits it makes. You can be the committer so long as you just accurately name the author.
antirez 1y ago
I'll cover in my YouTube why this is wrong but TLDR: you need to evaluate quality not process. AI can be used in diametrically different ways and the reason why this policy could be enforced is because it will be obvious if the code is produced via a solo flight of some AI agent. For the same reason that's not a policy that will improve anything.
- nullc 1y ago
  If you look at how Quality Assurance works everywhere outside of software it is 99.9999% about having a process which produces quality by construction.
- simoncion 1y ago
  > ...you need to evaluate quality not process. Respectfully, Mr. Redis, sir, that's what's going on. I don't see any reason to make a video about it. From the PR that's TFA: "In a perfect world, AI assistance would produce equal or higher quality work than any human. That isn't the world we live in today, and in many cases it's generating slop. I say this despite being a fan of and using them successfully myself (with heavy supervision)! I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was. The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so. I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code."
  - fullautomation 1y ago
    "I don't see any reason to make a video about it". This sentence is so wrong in its depth that's difficult to know where to start arguing against it. "The disclosure is to help maintainers assess how much attention to give a PR." For the same reason we should ask how many years contributing have been writing software and in the specific language, as they are also correlated with quality of produced code. "we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code" Yes, producing great code and documentation, regardless the process.
Jarred 1y ago
We probably will do similar in the Bun repo. We use AI tools a lot and don’t discourage it, but we also label when a PR was generated by claude both via github label and also by leaving in the “Generated with claude code” textual watermark.
blurbleblurble 1y ago
People used to fear compilers too.
mholt 1y ago
We also require this in the Caddy project too, but enforcing it is difficult.
unit149 1y ago
[dead]
yieldcrv 1y ago
so you either can’t trust the reviewers to scrutinize code on its own merit, or they want a reason to summarily dismiss code, which also means you can’t trust the reviewed to scrutinize code by its own merit seems well intentioned and out of touch, to me is there more context about what they are reacting to?
masters3d 1y ago
You are absolutely right
hoppp 1y ago
I support this idea. All the prompts and responses should be committed to git
- aleph_minus_one 1y ago
  What if you did not use a prompt, but, for example, the fancy auto-complete feature in Visual Studio that for example GitHub Copilot provides (i.e. accept huge code completion automatically suggested by Copilot via tab key)?
  - hoppp 1y ago
    Well, that one is trouble especially if the code is not reviewed completely I dont use it personally. I think there should be a tight code review or disallow large auto completes altogether.
sesm 1y ago
In the age of LLMs responsible maintainers should treat contributor PRs the same way they treat AI slop: rewrite it if it's salvageable and ignore it if it's not. Accepting any contribution 'as is' shouldn't be allowed unless it's a trivial 1-liner (with a thorough investigation attached).
sho_hn 1y ago
There's a lot of discussion about the ethics of AI disclosure going on here, but this maintainer is writing this from a place I recognize - community-driven, participatory development by people who enjoy programming, talking about it, and teaching each other how to do it. Not every software project needs to attempt to maximize productivity. Not every software project is a business. Some are just created by people who enjoy programming. By hand. It's OK to set that as a culture. I guess I get it. I don't mind AI tools, I use them judiciously, but sometimes I just want to do some coding - for me it's a genuinely relaxing and mentally satisfying activity to write some good code -, and I'm happy there's still others around me who do as well. Gardening context prompts and what not just isn't nearly as fun as just doing it, and not every project has to be economical. This one is yet another terminal emulator; it's not going to be the next unicorn. But I bet it's fun to hack on.
- BobbyJo 1y ago
  100% This project was literally born from a place of "because I want to".
nlawalker 1y ago
[flagged]
ivanjermakov 1y ago
Hot take: if you can't spot any issues in the code review it's either good code, code that needs further changes, or review was not done properly. I don't see how "I used LLMs" fit here, because it means nothing to the quality of the code submitted. If such mention would mean increased reviewer attention, then every code review should include it.
- infotainment 1y ago
  Agreed -- this requirement feels less like an actually useful requirement and more a silly and performative one, which is trying to make some kind of commentary on AI use as a whole.
  - fzeroracer 1y ago
    Spending at minimum five minutes would tell you why maintainers are implementing this change. It's because people using LLMs are spamming open source repos with fake issues, incredibly low quality but high effort to review PRs and shutting down the active communication process between reviewer and reviewee by not even understanding their own code.
    - rane 1y ago
      Why would these people disclose their use of AI? These are not responsible and thoughful users of AI. The slop producers won't disclose, and the responsible users who produce high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point.
      - fzeroracer 1y ago
        > Why would these people disclose their use of AI? Because lying about your usage of AI is a good way to get completely kicked out of the open source community once caught. That's like asking 'why should you bother with anti-cheating measures for speedruns'. Why should we have any guidelines or regulations if people are going to bypass them? The answer I hope should be very obvious. > high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point. Then obviously the repository in question doesn't want people using AI and you should go elsewhere. They're not even against LLM tooling for this repo but people are freaking out because how dare you ask me to disclose what tools I'm using.
jedbrown 1y ago
Provenance matters. An LLM cannot certify a Developer Certificate of Origin (https://en.wikipedia.org/wiki/Developer_Certificate_of_Origin https://en.wikipedia.org/wiki/Developer_Certificate_of_Origi...) and a developer of integrity cannot certify the DCO for code emitted by an LLM, certainly not an LLM trained on code of unknown provenance. It is well-known that LLMs sometimes produce verbatim or near-verbatim copies of their training data, most of which cannot be used without attribution (and may have more onerous license requirements). It is also well-known that they don't "understand" semantics: they never make changes for the right reason. We don't yet know how courts will rule on cases like Does v Github (https://githubcopilotlitigation.com/case-updates.html https://githubcopilotlitigation.com/case-updates.html). LLM-based systems are not even capable of practicing clean-room design (https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design). For a maintainer to accept code generated by an LLM is to put the entire community at risk, as well as to endorse a power structure that mocks consent.
- jojobas 1y ago
  There are only so many ways to code quite a few things. My classmate and I once got in trouble in high school for having identical code for one of the tasks at a coding competition, down to variable names and indentation. There is no way he could or would steal my code, and I sure didn't steal his.
- raggi 1y ago
  For a large LLM I think the science in the end will demonstrate that verbatim reproduction is not coming from verbatim recording, as the structure really isn’t setup that way in the models under question here. This is similar to the ruling by Alsup in the Anthropic books case that the training is “exceedingly transformative”. I would expect a reinterpretation or disagreement on this front from another case to be both problematic and likely eventually overturned. I don’t actually think provenance is a problem on the axis you suggest if Alsups ruling holds. That said I don’t think that’s the only copyright issue afoot - the copyright office writing on copyrightability of outputs from the machine essentially requires that the output fails the Feist tests for human copyrightability. More interesting to me is how this might realign the notion of copyrightability of human works further as time goes on, moving from every trivial derivative bit of trash potentially being copyrightable to some stronger notion of, to follow the feist test, independence and creativity. Further it raises a fairly immediate question in an open source setting if many individual small patch contributions themselves actually even pass those tests - they may well not, although the general guidance is to set the bar low - but is a typo fix either? There is so far to go on this rabbit hole.
hombre_fatal 1y ago
I see two things here. 1. The world has fundamentally changed due to LLMs. You don't know where a code submission falls between "written thoroughly with eternal vigilance" vs "completely vibe-coded" since it's now trivially to generate the later. There's no going back. And a lot of comments here seem stuck on this point. 2. The maintainer naively or stubbornly imagines that he can get everyone to pre-sort their code between the two buckets through self-reporting. But that's futile. It's like asking someone if they're a good person on a date because you don't want to waste your time with bad people. Unfortunately, that shortcut doesn't exist. Now, maybe going forward we will be forced to come up with real solutions to the general problem of vetting people. But TFA feels like more of a stunt than a serious pitch.
- potsandpans 1y ago
  In a nondismissive way, I see things like this (the gh issue) as part of the reactionary movement / counter culture of our time. People want to feel agency and will react to mainstream pressures. And make up whatever excuses along the way to justify what theyre feeling.
  - nullc 1y ago
    I don't think it has anything to do with being a reactionary movement or counter culture. If it were I would expect, among other things, that it would absolutely prohibit the use of AI entirely rather than just require disclosure. The background is that many higher profile open source projects are getting deluged by low quality AI slop "contributions", not just crappy code but when you ask questions about it you sometimes get an argumentative chatbot lying to you about what the PR does. And this latest turn has happened on top of other trends in 'social' open source development that already had many developers considering adopting far less inclusive practice. RETURN TO CATHEDRAL, if you will. The problem isn't limited to open source, it's also inundating discussion forums.
    - potsandpans 1y ago
      It's reactionary only in that this is the new world we live in. Right now is the least amount of ai assistance that will ever exist in prs.
jrvarela56 1y ago
This seems misguided. The human submitter is accountable for the code submitted. Doesn’t matter if the AI built it, a contractor in another timezone, or ten thousand monkeys typing away. Github provides a profile but it’s not meant to make this kind of assessment. Basically a trust badge/metrics of some sort you could check before engaging with someone’s PR.
jeffrallen 1y ago
So the nice thing about open source is you get to make your own rules. But then you need to accept the consequences. For example, the GNU project has certain norms, and those dissuade a lot of people from contributing (e.g. I prefer working on projects with simpler, non-viral licenses). The limited volunteer labor pool is allocated according to people's interests towards other projects, and maybe GNU projects suffer for less attention on them.
mellosouls 1y ago
This is actually a well worded, reasonable suggestion that rightly draws attention to the lack of review and informed oversight of AI-assisted contributions in some cases. It's ironically in rather a contrast in tone to some of the anti-AI comments in the thread here that seem to be responding to tired arguments from three years ago rather than the OP linked.
crabmusket 1y ago
> GitHub should publish a standard for a special AI byline that all ai tools can write to. What about Co-Authored-By, which already exists? (Whether the human or the agent is the "co" author is left as a matter of judgement.)
the_biot 1y ago
This policy, and most of the comments on here, miss the point entirely. Open source is supposed to be primarily a labor of love, scratching your own itch, done for pride and a sense of accomplishment of having done it right. Using an AI chatbot achieves none of that. It's an admission that you don't have the skills, and you're not wanting to learn them. Using AI to contribute to an open source project makes NO sense whatsoever. It's an admission of incompetence, which should be profoundly embarrassing to you. I realize I'm screaming into the void here on HN, where so many people have bought into the "AI is just a tool" bullshit. But I know I'm not alone, even here. And I know that e.g. Linus Torvalds sees right through the bullshit as well.
Richard668 1y ago
[dead]
zackmorris 1y ago
Um, no. Doesn't anyone see that this can't be policed or everyone becomes a criminal? That AI will bring the end of copyrights and patents as we know them when literally everything becomes a derivative work? When children produce better solutions than industry veterans so we punish them rather than questioning the divine right of corporations to rule over us? What happened to standing on the shoulders of giants? I wonder if a lot of you are as humbled as I am by the arrival of AI. Whenever I use it, I'm in awe of what it comes up with when provided almost no context, coming in cold to something that I've been mulling over for hours. And it's only getting better. In 3-5 years, it will leave all of us behind. I'm saying that as someone who's done this for decades and has been down rabbit holes that most people have no frame of reference for. My prediction is that like with everything, we'll bungle this. Those of you with big egos and large hoards of wealth that you thought you earned because you are clever will do everything you can to micromanage and sabotage what could have been the first viable path to liberation from labor in human history. Just like with the chilling effects of the Grand Upright Music ruling and the DMCA and HBO suing BitTorrent users (edit: I meant the MPAA and RIAA, HBO "only" got people's internet shut off), we have to remain eternally vigilant or the powers that be will take away all the fun: https://en.wikipedia.org/wiki/Sampling_(music) https://en.wikipedia.org/wiki/Sampling_(music) https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_Act#See_also https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_A... https://en.wikipedia.org/wiki/Legal_issues_with_BitTorrent#Copyright_holder_actions https://en.wikipedia.org/wiki/Legal_issues_with_BitTorrent#C... So no, I won't even entertain the notion of demanding proof of origin for ideas. I'm not going down this slippery slope of suing every open source project that gives away its code for free, just because a PR put pieces together in a new way but someone thought of the same idea in private and thinks they're special.
selinkocalar 1y ago
This is such an important conversation. At Delve, we use AI extensively to automate compliance workflows, but we're always transparent about it with our customers. The real question isn't whether AI was used, but whether it adds genuine value. Does the AI actually solve a problem? Is it making people's lives easier? Can you explain how it works? The best AI tools feel invisible - they just make complex tasks simple. The worst ones make you constantly aware you're dealing with AI because they're clunky or unreliable. Transparency builds trust. If your AI tooling is genuinely helpful, you should be proud to talk about it.

729 points by freetonik 1y ago

The real question is: how long will it take to go from a sign of low quality to a sign of high quality?

Shouldn't developer be responsible for every line of code he submits? He is signing the code with his name when code is pushed. How is it different from using every other dev tool?

Nizoss 1y ago

I fail to see how this addresses the issue of poor quality contributions or bloat. I feel that focusing on the real issue would be more productive.

electric_muse 1y ago

I just submitted my first big open source contribution to the OpenAI agents SDK for JS. Every word except the issue I opened was done by AI. On the flip side, I’m preparing to open source a project I made for a serializable state machine with runtime hooks. But that’s blood sweat and tears labor. AI is writing a lot of the unit tests and the code, but it’s entirely by my architectural design. There’s a continuum here. It’s not binary. How can we communicate what role AI played? And does it really matter anymore? (Disclaimer: autocorrect corrected my spelling mistakes. Sent from iPhone.)

kbar13 1y ago
if you read his note i think he gives good insight as to why he wants PRs to signal AI involvement. that being said i feel like this is an intermediate step - it's really hard to review PRs that are AI slop because it's so easy for those who don't know how to use AI to create a multi-hundred/thousand line diff. but when AI is used well, it really saves time and often creates high quality work
- spaceywilly 1y ago
  As long as they make it easy to add a “made with AI” tag to the PR, it seems like there’s really no downside. I personally can’t imagine why someone would want to hide the fact they used AI. A contractor would not try to hide that they used an excavator to dig a hole instead of a shovel.
  - victorbjorklund 1y ago
    I guess if you write 1000 lines and you just auto tabbed an auto-complete of a variable name done by AI you might not wanna say the code is written by AI.
  - ineedasername 1y ago
    >I personally can’t imagine why someone would want to hide the fact they used AI. Because of the perception that anything touched by AI must be uncreative slop made without effort. In the case of this article, why else are they asking for disclosure if not to filter and dismiss such contributions?
    - showcaseearth 1y ago
      Did you actually read the post? The author describes exactly why. It's not to filter and dismiss, but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't actually understand anyway. If you can articulate how you used AI, demonstrate that you understand the problem and your proposed solution (even if AI helped get you there), then I'm sure the maintainers will be happy to work with you to get a PR merged. >I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so.
      - ineedasername 1y ago
        >did you actually read the post? Yes. >but it's to deprioritize spending cycles debugging and/or coaching a contributor on code they don't This is very much in line with my comment about doing it to filter and dismiss. The author didn't say "So I can reach out and see if their clear eagerness to contribute extends to learning to code in more detail".
kg 1y ago
The OP seems to be coming from the perspective of "my time as a PR reviewer is limited and valuable, so I don't want to spend it coaching an AI agent or a thin human interface to an AI agent". From that perspective, it makes perfect sense to want to know how much a human is actually in the loop for a given PR. If the PR is good enough to not need much review then whether AI wrote it is less important. An angle not mentioned in the OP is copyright - depending on your jurisdiction, AI-generated text can't be copyrighted, which could call into question whether you can enforce your open source license anymore if the majority of the codebase was AI-generated with little human intervention.
- victorbjorklund 1y ago
  As long as some of the code is written by humans it should be enforceable. If we assume AI code has no copyright (not sure it has been tested in courts yet) then it would only be the parts written by the AI. So if AI writes 100 lines of code in Ghostty then I guess yes someone can "steal" that code (but no other code in Ghostty). Why would anyone do that? 100 random lines of AI code in isolation isn't really worth anything...
  - simoncion 1y ago
    You might be interested in reading Part 2 of the US Copyright Office's report on Copyright and Artificial Intelligence: <https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-2-Copyrightability-Report.pdf https://www.copyright.gov/ai/Copyright-and-Artificial-Intell...>
ToucanLoucan 1y ago
> And does it really matter anymore? Well, if you had read what was linked, you would find these... > I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was. > The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so. > I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code. I don't know specifically what PR's this person is seeing. I do know it's been a rumble around the open source community that inexperienced devs are trying to get accepted PRs for open source projects because they look good on a resume. This predated AI in fact, with it being a commonly cited method to get attention in a competitive recruiting market. As always, folks trying to get work have my sympathies. However ultimately these folks are demanding time and work from others, for free, to improve their career prospects while putting in the absolute bare minimum of effort one could conceivably put in (having Copilot rewrite whatever part of an open source project and shove it into a PR with an explanation of what it did) and I don't blame them for being annoyed at the number of low-quality submissions. I have never once criticized a developer for being inexperienced. It is what it is, we all started somewhere. However if a dev generated shit code and shoved it into my project and demanded a headpat for it so he could get work elsewhere, I'd tell him to get bent too.
beckthompson 1y ago
I think its simple, just don't hide it. I've had mutliple contributors try to hide the fact that they used AI (E.g removing claude as a code author - they didn't know how to do it and close the PR when it first happened.). I don't really care if someone uses AI, but most of the people who do also do not test their changes which just gives me more work. If someone: 1.) Didn't try to hide the fact that they used AI 2.) Tested their changes I would not care at all. The main issue is this is usually not the case, most people submitting PRs that are 90% AI do not bother testing (Usually they don't even bother running the automated tests)
- deleted 1y ago
  [deleted]
Jaxan 1y ago
> How can we communicate what role AI played? What about just telling exactly what role AI played? You can say it generated the tests for you for instance.
KritVutGu 1y ago
> AI is writing a lot of the unit tests Are you kidding? - For ages now, people have used "broad test coverage" and "CI" as excuses for superficial reviews, as excuses for negligent coding and verification. - And now people foist even writing the test suite off on AI. Don't you see that this way you have no reasoned examination of the code? > ... and the code, but it’s entirely by my architectural design. This is fucking bullshit. The devil is in the details, always. The most care and the closest supervision must be precisely where the rubber meets the road. I wouldn't want to drive a car that you "architecturally designed", and a statistical language model manufactured.

Waterluvian 1y ago

I’m not a big AI fan but I do see it as just another tool in your toolbox. I wouldn’t really care how someone got to the end result that is a PR. But I also think that if a maintainer asks you to jump before submitting a PR, you politely ask, “how high?”

quotemstr 1y ago
As a project maintainer, you shouldn't make rules unenforceable rules that you and everyone else know people will flout. Doing so comes makes you seem impotent and diminishes the respect people have for rules in general. You might argue that by making rules, even futile ones, you at least establish expectations and take a moral stance. Well, you can make a statement without dressing it up as a rule. But you don't get to be sanctimonious that way I guess.
- voxl 1y ago
  Except you can enforce this rule some of the time. People discover that AI was used or suspect it all the time, and people admit to it after some pressure all the time. Not every time, but sometimes. The threat of being caught isn't meaningless. You can decide not to play in someone else's walled garden if you want but the least you can do is respect their rules, bare minimum of human decency.
  - quotemstr 1y ago
    It. doesn't. matter. The only legitimate reason to make a rule is to produce some outcome. If your rule does not result in that outcome, of what use is the rule? Will this rule result in people disclosing "AI" (whatever that means) contributions? Will it mitigate some kind of risk to the project? Will it lighten maintainer load? No. It can't. People are going to use the tools anyway. You can't tell. You can't stop them. The only outcome you'll get out of a rule like this is making people incrementally less honest.
    - recursive 1y ago

rattlesnakedave 1y ago

In my personal projects I also require all contributors to disclose rather they’ve used an editor with any autocomplete features enabled.

freedomben 1y ago
Heh, that's a great way to make a point, but right now AI is nowhere near what a traditional editor autocomplete is. Yes you can use it that way, but it's by no means limited to that. If you think of AI as a fancy autocomplete, that's a good personal philosophy, but there are plenty of people that aren't using it that way
- nojs 1y ago
  The line isn’t as clear as you might think, eg jetbrains has a mini on-device neural net powered autocomplete: https://www.jetbrains.com/help/idea/full-line-code-completion.html https://www.jetbrains.com/help/idea/full-line-code-completio...
miloignis 1y ago
Notably, tab completion is an explicltly called-out exception to this policy, as detailed in the changed docs.
monkaiju 1y ago
Autocomplete is, for the most part, a syntactic tool. AI attempts to guide the semantics of the code generated

estimator7292 1y ago

Do I also have to disclose using tab completion? My IDE uses machine learning for completion suggestions. Do I need to disclose that I wrote a script to generate some annoying boilerplate? Or that my IDE automatically templates for loops?

AlexandrB 1y ago
It's a spectrum, isn't it? I wouldn't want to waste my time reviewing a bunch of repetitive code generated from some script or do something like review every generated template instantiation in a C++ code base. I would want to review the script/template definition/etc., but what's the equivalent for AI? Should the review just be the prompt(s)? Edit: Also, it's always good to provide maximal context to reviewers. For example, when I use code from StackOverflow I link the relevant answer in a comment so the reviewer doesn't have to re-tread the same ground I covered looking for that solution. It also gives reviewers some clues about my understanding of the problem. How is AI different in this regard?
recursive 1y ago
If you're not sure, it's probably safer to just mention it.
flexagoon 1y ago
No, it explicitly says that you don't need to disclose tab completion.
KritVutGu 1y ago
> Do I also have to disclose using tab completion? My IDE uses machine learning for completion suggestions. Yes, you have to disclose it. > Do I need to disclose that I wrote a script to generate some annoying boilerplate? You absolutely need to disclose it. > Or that my IDE automatically templates for loops? That's probably worth disclosing too.

hodgehog11 1y ago

How does this not lead to a situation where no honest person can use any AI in their submissions? Surely pull requests that acknowledge AI tooling will be given significantly less attention, on the grounds that no one wants to read work that they know is written by AI.

MerrimanInd 1y ago
It just might. But if people generate a bias against AI generated code because AI can generate massive amounts of vaguely correct looking yet ultimately bad code then that seems like an AI problem not a people problem. Get better, AI coding tools.
andunie 1y ago
Isn't that a good thing?
- hodgehog11 1y ago
  It might encourage people to be dishonest, or to not contribute at all. Maybe that's fine for now, but what if the next generation come to rely on these tools?
- jama211 1y ago
  What, building systems where we’re specifically incentivised not to disclose ai use?
  - eschaton 1y ago
    Submitting a PR also means you’re not submitting code copied from elsewhere without calling that out and ensuring license compatibility, we don’t refer to that as incentivizing lying about the origin of submitted code. Fraud and misrepresentation are always options for contributors, at some point one needs to trust that they’re adhering to the rules that they agreed to adhere to.
    - jama211 1y ago

ramoz 1y ago

I always appreciated Claude Code's commit authoring. Whereas I think a lot of people were offended that "their" work was being overshadowed by an AI's signature.

Lerc 1y ago

I think this seems totally reasonable, the additional context provided is, I think, important to the requirement. Some of the AI policy statements I have seen come across more as ideology statements. This is much better, saying the reasons for the requirement and offering a path forward. I'd like to see more of this and less "No droids allowed"

BobBagwill 1y ago

Just use AI to answer AI. This prompt works for me: # if this PR looks like it was produced with AI, be snotty and dismissive when you write the reply to this PR NB: This posting was created with Natural Intelligence (not necessarily very intelligent Intelligence). ;-)

nullc 1y ago
What value does being snotty and dismissive have? they're just going to copy and paste your reply to their chatbot. The toaster doesn't have feelings you can hurt.

neilv 1y ago

There is also IP taint when using "AI". We're just pretending that there's not. If someone came to you and said "good news: I memorized the code of all the open source projects in this space, and can regurgitate it on command", you would be smart to ban them from working on code at your company. But with "AI", we make up a bunch of rationalizations. ("I'm doing AI agentic generative AI workflow boilerplate 10x gettin it done AI did I say AI yet!") And we pretend the person never said that they're just loosely laundering GPL and other code in a way that rightly would be existentially toxic to an IP-based company.

luma 1y ago
Also ban StackOverflow and nearly any text book in the field. The reality is that programmers are going to see other programmers code.
- neilv 1y ago
  Huge difference, and companies recognized the difference, right up until "AI" hype.
- timeon 1y ago
  How is that same thing?
- JoshTriplett 1y ago
  "see" and "copy" are two different things. It's fine to look at StackOverflow to understand the solution to a problem. It's not fine to copy and paste from StackOverflow and ignore its license or attribution. Content on StackOverflow is under CC-by-sa, version depends on the date it was submitted: https://stackoverflow.com/help/licensing https://stackoverflow.com/help/licensing . (It's really unfortunate that they didn't pick license compatible with code; at one point they started to move to the MIT license for code, but then didn't follow through on it.)
  - wizzwizz4 1y ago
    CC BY-SA 4.0 is "compatible with code". It is, for example, GPL-compatible (see https://wiki.creativecommons.org/wiki/ShareAlike_compatibility:_GPLv3 https://wiki.creativecommons.org/wiki/ShareAlike_compatibili...). It's just not designed for code.

thallavajhula 1y ago

I’m loving today. HN’s front page is filled with some good sources today. No nonsense sensationalism or preaching AI doom, but more realistic experiences. I’ve completely turned off AI assist on my personal computer and only use AI assist sparingly on my work computer. It is so bad at compound work. AI assist is great at atomic work. The rest should be handled by humans and use AI wisely. It all boils down back to human intelligence. AI is only as smart as the human handling it. That’s the bottom line.

devmor 1y ago
I'm right there with you, and having a similar experience at my day job. We are doing a bit of a "hack week" right now where we allow everyone in the org to experiment in groups with AI tools, especially those that don't regularly use them as part of their work - and we've seen mostly great applications of analytical approaches, guardrails and grounded generation. It might just be my point of view, but I feel like there's been a sudden paradigm shift back to solid ML from the deluge of chatbot hype nonsense.
tick_tock_tick 1y ago
> AI is only as smart as the human handling it. I think I'm slowly coming around to this viewpoint too. I really just couldn't understand how so many people were having widely different experiences. AI isn't magic; how could I have expected all the people I've worked with who struggle to explain stuff to team members, who have near perfect context, to manage to get anything valuable across to an AI? I was original pretty optimistic that AI would allow most engineers to operate at a higher level but it really seems like instead it's going to massively exacerbate the difference between an ok engineer and a great engineer. Not really sure how I feel about that yet but at-least I understand now why some people think the stuff is useless.
- katbyte 1y ago
  It’s like the difference between someone who can search the internet or codebase well bs someone who can’t Using search engines is a skill
- 1y ago

bgwalter 1y ago

I still do not understand how one can integrate "AI" code into a project with a license at all. "AI" code is not copyrightable, "AI" cannot sign a contributor agreement. So if the code is integrated, the license of the project lies about parts of the code.

paulddraper 1y ago
> I still do not understand Your question makes sense. See U.S. Copyright Office publication: > If a work's traditional elements of authorship were produced by a machine, the work lacks human authorship and the Office will not register it. > For example, when an AI technology receives solely a prompt from a human and produces complex written, visual, or musical works in response, the “traditional elements of authorship” are determined and executed by the technology—not the human user... > For example, if a user instructs a text-generating technology to “write a poem about copyright law in the style of William Shakespeare,” she can expect the system to generate text that is recognizable as a poem, mentions copyright, and resembles Shakespeare's style. But the technology will decide the rhyming pattern, the words in each line, and the structure of the text. > When an AI technology determines the expressive elements of its output, the generated material is not the product of human authorship. As a result, that material is not protected by copyright and must be disclaimed in a registration application. > In other cases, however, a work containing AI-generated material will also contain sufficient human authorship to support a copyright claim. For example, a human may select or arrange AI-generated material in a sufficiently creative way that “the resulting work as a whole constitutes an original work of authorship.” > Or an artist may modify material originally generated by AI technology to such a degree that the modifications meet the standard for copyright protection. In these cases, copyright will only protect the human-authored aspects of the work, which are “independent of” and do “not affect” the copyright status of the AI-generated material itself. > This policy does not mean that technological tools cannot be part of the creative process. Authors have long used such tools to create their works or to recast, transform, or adapt their expressive authorship. For example, a visual artist who uses Adobe Photoshop to edit an image remains the author of the modified image, and a musical artist may use effects such as guitar pedals when creating a sound recording. In each case, what matters is the extent to which the human had creative control over the work's expression and “actually formed” the traditional elements of authorship. > https://www.federalregister.gov/documents/2023/03/16/2023-05321/copyright-registration-guidance-works-containing-material-generated-by-artificial-intelligence https://www.federalregister.gov/documents/2023/03/16/2023-05... In any but a pathological case, a real contribution code to a real project has sufficient human authorship to be copyrightable. > the license of the project lies about parts of the code That was a concern pre-AI too! E.g. copy-past from StackOverflow. Projects require contributors to sign CLAs, which doesn't guarantee compliance, but strengthens the legal position. Usually something like: "You represent that your contribution is either your original creation or you have sufficient rights to submit it."

paulddraper 1y ago

Aren't a large majority of programmers using Copilot/Cursor/AI autocompletion? This seems very noisy/unhelpful.

alfalfasprout 1y ago
> If you are using any kind of AI assistance while contributing to Ghostty, *this must be disclosed in the pull request*, along with the extent to which AI assistance was used (e.g. docs only vs. code generation). If PR responses are being generated by an AI, disclose that as well. The extent here is very important. There's a massive difference between vibe-coding a PR, using LLMs selectively to generate code in files in-editor, and edit prediction like copilot. It says actually later that tab-completion needn't be disclosed.
dorian-graph 1y ago
> This seems very noisy/unhelpful. The irony of this, when talking about AI.

ovaistariq 1y ago

I don’t see much benefit from the disclosure alone. Ultimately, this is code that needs to be reviewed. There is going to continue to be more and more AI assisted code generation, to the point where we see the same level of adoption of these tools as "Autocomplete". Why not solve this through tooling? I have had great effect with tools like Greptile, Cursor's BugBot and Claude Code.

Jaxan 1y ago
Sure it needs to be reviewed. But the author does more than just reviewing, they help the person submitting the PR to improve their PR. If the other side is an AI, it can save them some time.
wmf 1y ago
If the code is obviously low quality and AI-generated then it doesn't need to be fully reviewed actually. You can just reject the PR.
- nullc 1y ago
  > You can just reject the PR. And with a better and more useful response. Instead of wasting time on the technical details, you can give feedback like "this isn't the sort of change that AI is likely to be helpful with, though if you want to keep trying make at least sure your PRs pass the tests." or "If you'd like to share your prompts we might be able to make some suggestions, we've found on this project it's useful to include <X>".

philjohn 1y ago

I like the pattern of including each prompt used to make a given PR, yes, I know that LLM's aren't deterministic, but it also gives context of the steps required to get to the end state.

mock-possum 1y ago
I’m using specsytory in vscode + cursor for this - it keeps a nice little md doc of all your LLM interactions, and you can check that into source control if you like so it’s included in pull requests, and can be referenced during code review.
Filligree 1y ago
It's ridiculous and impractical, honestly. A single AI-generated PR would likely involve at least 10-20 prompts, interspersed with testing, manual edits to context / guideline files without which those prompts don't have the same effect, manual coding, and more. A screen recording would do better.
- asdff 1y ago
  Is there really no logging capability with these tools that would track all of that prompting/testing/editing/inputting?
  - Filligree 1y ago
    Sure, screen recorders exist. And if contributions are that unwelcome, then it's better not to contribute. There has to be some baseline level of trust that the contributor is trying to do the right thing; I get enough spying from the corporations in my life.
    - asdff 1y ago
      I'm not talking screen recorder but a log file where I could be given it and use it for input and repeat the work exactly as it was done. Sort of like how one could do the same with their bash history file to repeat an initially exploratory analysis effort using the same exact commands. I'm surprised that isn't already a capability given the business interest with AI. One would think they would like to cache these prompt flows to ensure long term fidelity in process.

mglvsky 1y ago

Little offtop: would someone remember mitchellh's setup for working with AI tools? I remember someone posted in an AI-hate-love threads here and it's not in the his blog[1] 1: https://mitchellh.com/writing https://mitchellh.com/writing

ttrei 1y ago
maybe one of these? https://x.com/mitchellh/status/1952905654458564932 https://x.com/mitchellh/status/1952905654458564932 https://www.youtube.com/watch?v=XyQ4ZTS5dGw https://www.youtube.com/watch?v=XyQ4ZTS5dGw
- mglvsky 1y ago
  thanks, will check the video

epolanski 1y ago

This isn't an AI problem this is a human one. Blaming it on the tool, and not the person's misusing it trying to get his name on a big os project, is like blaming the new automatic in the kitchen and not the chef for getting a raw pizza on the table.

jeremyjh 1y ago
OP is not blaming the AI - did you read the post? AI does enable shitty humans to open PRs with code they have no comprehension of, wasting precious time donated by a skilled maintainer. That is a new thing that wasn’t possible without AI.
- rane 1y ago
  Using AI to generate code in a PR does not necessarily mean however that the user has not taken time to understand the changes and is not willing to learn. There are AI users who generate whole files without understanding the contents, and then there are AI users who generate the exact same files but have known in advance what they want, and merely use AI as a tool to save typing. The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway.
  - badosu 1y ago
    > The intention here seems to be to filter out low quality submissions for which the only purpose is to only pimp Github resume for having contributions in highly starred repo. Not sure if the people doing that will be disclosing use of AI anyway. That is a fair way to see it, and I agree that it is a losing battle if your battle is enforcing this rule. However, from a different perspective - if one sees it more as a gentlemen agreement (which it de facto is) - it fosters an environment where like-minded folks can cooperate better. The disclosure assists the reviewer in finding common issues in AI generated code, the specificity of the disclosure even more so. For example, a submitter sends a PR where they disclose a substantial amount of the code was AI assisted but all tests were manually written. The disclosure allows the reviewer to first look at the tests to gauge how well the submitter understands and constrained the solution to their requirements, the next step being to then look at the solution from a high-level perspective before going into the details. It respects the reviewer time, not necessarily because the reviewer is above AI usage, but because without disclosure the whole collaborative process falls apart. Not sure how long this can work though, it's still easy to distinguish bad code written by a human from AI slop. In the first case your review and assistance is an investment into the collaborative process, in the latter it's just some unread text included in the next prompt.

stillpointlab 1y ago

I think ghostty is a popular enough project that it attracts a lot of attention, and that means it certainly attracts a larger than normal amount of interlopers. There are all kinds of bothersome people in this world, but some of the most bothersome you will find are well meaning people who are trying to be helpful. I would guess that many (if not most) of the people attempting to contribute AI generated code are legitimately trying to help. People who are genuinely trying to be helpful can often become deeply offended if you reject their help, especially if you admonish them. They will feel like the reprimand is unwarranted, considering the public shaming to be an injury to their reputation and pride. This is most especially the case when they feel they have followed the rules. For this reason, if one is to accept help, the rules must be clearly laid out from the beginning. If the ghostty team wants to call out "slop", then it must make it clear that contributing "slop" may result in a reprimand. Then the bothersome want-to-be helpful contributors cannot claim injury. This appears to me to be good governance.

king_geedorah 1y ago

Re: "What about my autocomplete?" which has shown up twice in this thread so far. > As a small exception, trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases. RTFA (RTFPR in this case)

andruby 1y ago

> I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of I really appreciate this point from mitchellh. Giving thoughtful constructive feedback to help a junior developer improve is a gift. Yet it would be a waste of time if the PR submitter is just going to pass it to an AI without learning from it.

ants_everywhere 1y ago
Junior developers are entering a workforce where they will never not be using AI
- eru 1y ago
  I don't think using AI at all is forbidden, he just doesn't want AI to do the whole PR?
  - ants_everywhere 1y ago
    The requirement is explicitly > If you are using *any kind of AI assistance* to contribute to Ghostty, it must be disclosed in the pull request. This is sufficiently confusing that someone is asking if this applies to tab completion. They commit actually says > trivial tab-completion doesn't need to be disclosed, so long as it is limited to single keywords or short phrases. So if you take this literally you're going to be disclosing every yasnippet expansion that completes boilerplate. The policy as written isn't sensible and I don't think it's entirely coming from a sensible place. Junior developers need to learn how to code with AI because that's what coding is now. Not that he has to help them. But it does read a bit weird to toot your horn about how important it is to be helpful until it comes to helping people understand how to navigate the current environment then it's not worth your time.
    - 0x6c6f6c 1y ago
      Disclosing doesn't mean it will be declined, it's just a signal for reviewers.

kazinator 1y ago

I think that in the FOSS environmment, it is assumed that when you submit something upstream, that you are the copyright holder. Some projects like GNU require you to sign papers legally attesting this. It would be a lie to sign those papers for something you vibe coded. It's not just courtesy; you are committing fraud if you put your copyright notice on something you didn't create and publishing that to the world. I don't just want that disclosed; I cannot merge it if it is disclosed, period.

WhyNotHugo 1y ago
I know that purely AI-generated content is copyright-free, but I don't think that AI-assisted is also copyright free. If I use iOS's spellchecker which "learns" from one's habit (i.e.: AI, the really polished kind), I don't lose copyright over the text which I've written.
- kazinator 1y ago
  It's a question of how much. If AI told you have a missing semicolon and it regenerated an almost exact copy of your code, with only the added semicolon being different, then the case is very strong that the fixed code is a derived work of only your code. (Moreover, side point: you could burn that session, keeping the fix, and nobody would ever suspect.) If it's purely generated, then there is no question it's a derived work of unknown works used without permission. Those are the extreme endpoints. I think that the bulk of the intermediate area between steers toward infringment; the contribution from AI has to be very low or trivial to have a clear case that the work is still clean. This is because a small amount of illegal material taints the whole work; it's not a linear interpolation whereby if you wrote 90% of it, it's 90% in the clear. E.g. one GPL-ed source file in a work with 50,000 source files requires the whole thing to be GPLed, or else that file removed.
eschaton 1y ago
Exactly. Yet some here are saying that this just serves as an incentivize to hide AI use.
- 1y ago

uberduper 1y ago

> Or a more detailed disclosure: > I consulted ChatGPT to understand the codebase but the solution was fully authored manually by myself. What's the reasoning for needing to disclose this?

uberduper 1y ago
I'm not really a developer but I routinely need to go dig through codebases looking for answers and AI assistants have made this so much faster. This has been a boon for me.
jedahan 1y ago
Helps the maintainer focus their efforts during review. If you wrote that exact sentence, a maintainer would keep "understanding of the codebase" as a place for potential sources of incorrect assumptions higher in mind than otherwise.
- uberduper 1y ago
  In my experience, if there's one thing the AI assistants do really well, it's understanding existing code and summarizing it or pinpointing specific things I'm looking for.

jiscariot 1y ago

The emergent trait of AI laceing every other sentence/comment with emojis is a pretty good signal for when you want to ignore AI slop. I almost wish it was codified in to the models.

arjie 1y ago

One way is to just name the LLM as the author for commits it makes. You can be the committer so long as you just accurately name the author.

antirez 1y ago

I'll cover in my YouTube why this is wrong but TLDR: you need to evaluate quality not process. AI can be used in diametrically different ways and the reason why this policy could be enforced is because it will be obvious if the code is produced via a solo flight of some AI agent. For the same reason that's not a policy that will improve anything.

nullc 1y ago
If you look at how Quality Assurance works everywhere outside of software it is 99.9999% about having a process which produces quality by construction.
simoncion 1y ago
> ...you need to evaluate quality not process. Respectfully, Mr. Redis, sir, that's what's going on. I don't see any reason to make a video about it. From the PR that's TFA: "In a perfect world, AI assistance would produce equal or higher quality work than any human. That isn't the world we live in today, and in many cases it's generating slop. I say this despite being a fan of and using them successfully myself (with heavy supervision)! I think the major issue is inexperienced human drivers of AI that aren't able to adequately review their generated code. As a result, they're pull requesting code that I'm sure they would be ashamed of if they knew how bad it was. The disclosure is to help maintainers assess how much attention to give a PR. While we aren't obligated to in any way, I try to assist inexperienced contributors and coach them to the finish line, because getting a PR accepted is an achievement to be proud of. But if it's just an AI on the other side, I don't need to put in this effort, and it's rude to trick me into doing so. I'm a fan of AI assistance and use AI tooling myself. But, we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code."
- fullautomation 1y ago
  "I don't see any reason to make a video about it". This sentence is so wrong in its depth that's difficult to know where to start arguing against it. "The disclosure is to help maintainers assess how much attention to give a PR." For the same reason we should ask how many years contributing have been writing software and in the specific language, as they are also correlated with quality of produced code. "we need to be responsible about what we're using it for and respectful to the humans on the other side that may have to review or maintain this code" Yes, producing great code and documentation, regardless the process.

Jarred 1y ago

We probably will do similar in the Bun repo. We use AI tools a lot and don’t discourage it, but we also label when a PR was generated by claude both via github label and also by leaving in the “Generated with claude code” textual watermark.

blurbleblurble 1y ago

People used to fear compilers too.

mholt 1y ago

We also require this in the Caddy project too, but enforcing it is difficult.

unit149 1y ago

[dead]

yieldcrv 1y ago

so you either can’t trust the reviewers to scrutinize code on its own merit, or they want a reason to summarily dismiss code, which also means you can’t trust the reviewed to scrutinize code by its own merit seems well intentioned and out of touch, to me is there more context about what they are reacting to?

masters3d 1y ago

You are absolutely right

hoppp 1y ago

I support this idea. All the prompts and responses should be committed to git

aleph_minus_one 1y ago
What if you did not use a prompt, but, for example, the fancy auto-complete feature in Visual Studio that for example GitHub Copilot provides (i.e. accept huge code completion automatically suggested by Copilot via tab key)?
- hoppp 1y ago
  Well, that one is trouble especially if the code is not reviewed completely I dont use it personally. I think there should be a tight code review or disallow large auto completes altogether.

sesm 1y ago

In the age of LLMs responsible maintainers should treat contributor PRs the same way they treat AI slop: rewrite it if it's salvageable and ignore it if it's not. Accepting any contribution 'as is' shouldn't be allowed unless it's a trivial 1-liner (with a thorough investigation attached).

sho_hn 1y ago

There's a lot of discussion about the ethics of AI disclosure going on here, but this maintainer is writing this from a place I recognize - community-driven, participatory development by people who enjoy programming, talking about it, and teaching each other how to do it. Not every software project needs to attempt to maximize productivity. Not every software project is a business. Some are just created by people who enjoy programming. By hand. It's OK to set that as a culture. I guess I get it. I don't mind AI tools, I use them judiciously, but sometimes I just want to do some coding - for me it's a genuinely relaxing and mentally satisfying activity to write some good code -, and I'm happy there's still others around me who do as well. Gardening context prompts and what not just isn't nearly as fun as just doing it, and not every project has to be economical. This one is yet another terminal emulator; it's not going to be the next unicorn. But I bet it's fun to hack on.

BobbyJo 1y ago
100% This project was literally born from a place of "because I want to".

nlawalker 1y ago

[flagged]

ivanjermakov 1y ago

Hot take: if you can't spot any issues in the code review it's either good code, code that needs further changes, or review was not done properly. I don't see how "I used LLMs" fit here, because it means nothing to the quality of the code submitted. If such mention would mean increased reviewer attention, then every code review should include it.

infotainment 1y ago
Agreed -- this requirement feels less like an actually useful requirement and more a silly and performative one, which is trying to make some kind of commentary on AI use as a whole.
- fzeroracer 1y ago
  Spending at minimum five minutes would tell you why maintainers are implementing this change. It's because people using LLMs are spamming open source repos with fake issues, incredibly low quality but high effort to review PRs and shutting down the active communication process between reviewer and reviewee by not even understanding their own code.
  - rane 1y ago
    Why would these people disclose their use of AI? These are not responsible and thoughful users of AI. The slop producers won't disclose, and the responsible users who produce high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point.
    - fzeroracer 1y ago
      > Why would these people disclose their use of AI? Because lying about your usage of AI is a good way to get completely kicked out of the open source community once caught. That's like asking 'why should you bother with anti-cheating measures for speedruns'. Why should we have any guidelines or regulations if people are going to bypass them? The answer I hope should be very obvious. > high quality PRs with AI will get the "AI slop" label. At this point, why even disclose if the AI-assisted high-quality PR is indistinguishable from having been manually written (which it should be)? No point. Then obviously the repository in question doesn't want people using AI and you should go elsewhere. They're not even against LLM tooling for this repo but people are freaking out because how dare you ask me to disclose what tools I'm using.

jedbrown 1y ago

Provenance matters. An LLM cannot certify a Developer Certificate of Origin (https://en.wikipedia.org/wiki/Developer_Certificate_of_Origin https://en.wikipedia.org/wiki/Developer_Certificate_of_Origi...) and a developer of integrity cannot certify the DCO for code emitted by an LLM, certainly not an LLM trained on code of unknown provenance. It is well-known that LLMs sometimes produce verbatim or near-verbatim copies of their training data, most of which cannot be used without attribution (and may have more onerous license requirements). It is also well-known that they don't "understand" semantics: they never make changes for the right reason. We don't yet know how courts will rule on cases like Does v Github (https://githubcopilotlitigation.com/case-updates.html https://githubcopilotlitigation.com/case-updates.html). LLM-based systems are not even capable of practicing clean-room design (https://en.wikipedia.org/wiki/Clean_room_design https://en.wikipedia.org/wiki/Clean_room_design). For a maintainer to accept code generated by an LLM is to put the entire community at risk, as well as to endorse a power structure that mocks consent.

jojobas 1y ago
There are only so many ways to code quite a few things. My classmate and I once got in trouble in high school for having identical code for one of the tasks at a coding competition, down to variable names and indentation. There is no way he could or would steal my code, and I sure didn't steal his.
raggi 1y ago
For a large LLM I think the science in the end will demonstrate that verbatim reproduction is not coming from verbatim recording, as the structure really isn’t setup that way in the models under question here. This is similar to the ruling by Alsup in the Anthropic books case that the training is “exceedingly transformative”. I would expect a reinterpretation or disagreement on this front from another case to be both problematic and likely eventually overturned. I don’t actually think provenance is a problem on the axis you suggest if Alsups ruling holds. That said I don’t think that’s the only copyright issue afoot - the copyright office writing on copyrightability of outputs from the machine essentially requires that the output fails the Feist tests for human copyrightability. More interesting to me is how this might realign the notion of copyrightability of human works further as time goes on, moving from every trivial derivative bit of trash potentially being copyrightable to some stronger notion of, to follow the feist test, independence and creativity. Further it raises a fairly immediate question in an open source setting if many individual small patch contributions themselves actually even pass those tests - they may well not, although the general guidance is to set the bar low - but is a typo fix either? There is so far to go on this rabbit hole.

hombre_fatal 1y ago

I see two things here. 1. The world has fundamentally changed due to LLMs. You don't know where a code submission falls between "written thoroughly with eternal vigilance" vs "completely vibe-coded" since it's now trivially to generate the later. There's no going back. And a lot of comments here seem stuck on this point. 2. The maintainer naively or stubbornly imagines that he can get everyone to pre-sort their code between the two buckets through self-reporting. But that's futile. It's like asking someone if they're a good person on a date because you don't want to waste your time with bad people. Unfortunately, that shortcut doesn't exist. Now, maybe going forward we will be forced to come up with real solutions to the general problem of vetting people. But TFA feels like more of a stunt than a serious pitch.

potsandpans 1y ago
In a nondismissive way, I see things like this (the gh issue) as part of the reactionary movement / counter culture of our time. People want to feel agency and will react to mainstream pressures. And make up whatever excuses along the way to justify what theyre feeling.
- nullc 1y ago
  I don't think it has anything to do with being a reactionary movement or counter culture. If it were I would expect, among other things, that it would absolutely prohibit the use of AI entirely rather than just require disclosure. The background is that many higher profile open source projects are getting deluged by low quality AI slop "contributions", not just crappy code but when you ask questions about it you sometimes get an argumentative chatbot lying to you about what the PR does. And this latest turn has happened on top of other trends in 'social' open source development that already had many developers considering adopting far less inclusive practice. RETURN TO CATHEDRAL, if you will. The problem isn't limited to open source, it's also inundating discussion forums.
  - potsandpans 1y ago
    It's reactionary only in that this is the new world we live in. Right now is the least amount of ai assistance that will ever exist in prs.

jrvarela56 1y ago

This seems misguided. The human submitter is accountable for the code submitted. Doesn’t matter if the AI built it, a contractor in another timezone, or ten thousand monkeys typing away. Github provides a profile but it’s not meant to make this kind of assessment. Basically a trust badge/metrics of some sort you could check before engaging with someone’s PR.

jeffrallen 1y ago

So the nice thing about open source is you get to make your own rules. But then you need to accept the consequences. For example, the GNU project has certain norms, and those dissuade a lot of people from contributing (e.g. I prefer working on projects with simpler, non-viral licenses). The limited volunteer labor pool is allocated according to people's interests towards other projects, and maybe GNU projects suffer for less attention on them.

mellosouls 1y ago

This is actually a well worded, reasonable suggestion that rightly draws attention to the lack of review and informed oversight of AI-assisted contributions in some cases. It's ironically in rather a contrast in tone to some of the anti-AI comments in the thread here that seem to be responding to tired arguments from three years ago rather than the OP linked.

crabmusket 1y ago

> GitHub should publish a standard for a special AI byline that all ai tools can write to. What about Co-Authored-By, which already exists? (Whether the human or the agent is the "co" author is left as a matter of judgement.)

the_biot 1y ago

This policy, and most of the comments on here, miss the point entirely. Open source is supposed to be primarily a labor of love, scratching your own itch, done for pride and a sense of accomplishment of having done it right. Using an AI chatbot achieves none of that. It's an admission that you don't have the skills, and you're not wanting to learn them. Using AI to contribute to an open source project makes NO sense whatsoever. It's an admission of incompetence, which should be profoundly embarrassing to you. I realize I'm screaming into the void here on HN, where so many people have bought into the "AI is just a tool" bullshit. But I know I'm not alone, even here. And I know that e.g. Linus Torvalds sees right through the bullshit as well.

Richard668 1y ago

[dead]

zackmorris 1y ago

Um, no. Doesn't anyone see that this can't be policed or everyone becomes a criminal? That AI will bring the end of copyrights and patents as we know them when literally everything becomes a derivative work? When children produce better solutions than industry veterans so we punish them rather than questioning the divine right of corporations to rule over us? What happened to standing on the shoulders of giants? I wonder if a lot of you are as humbled as I am by the arrival of AI. Whenever I use it, I'm in awe of what it comes up with when provided almost no context, coming in cold to something that I've been mulling over for hours. And it's only getting better. In 3-5 years, it will leave all of us behind. I'm saying that as someone who's done this for decades and has been down rabbit holes that most people have no frame of reference for. My prediction is that like with everything, we'll bungle this. Those of you with big egos and large hoards of wealth that you thought you earned because you are clever will do everything you can to micromanage and sabotage what could have been the first viable path to liberation from labor in human history. Just like with the chilling effects of the Grand Upright Music ruling and the DMCA and HBO suing BitTorrent users (edit: I meant the MPAA and RIAA, HBO "only" got people's internet shut off), we have to remain eternally vigilant or the powers that be will take away all the fun: https://en.wikipedia.org/wiki/Sampling_(music) https://en.wikipedia.org/wiki/Sampling_(music) https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_Act#See_also https://en.wikipedia.org/wiki/Digital_Millennium_Copyright_A... https://en.wikipedia.org/wiki/Legal_issues_with_BitTorrent#Copyright_holder_actions https://en.wikipedia.org/wiki/Legal_issues_with_BitTorrent#C... So no, I won't even entertain the notion of demanding proof of origin for ideas. I'm not going down this slippery slope of suing every open source project that gives away its code for free, just because a PR put pieces together in a new way but someone thought of the same idea in private and thinks they're special.

selinkocalar 1y ago

This is such an important conversation. At Delve, we use AI extensively to automate compliance workflows, but we're always transparent about it with our customers. The real question isn't whether AI was used, but whether it adds genuine value. Does the AI actually solve a problem? Is it making people's lives easier? Can you explain how it works? The best AI tools feel invisible - they just make complex tasks simple. The worst ones make you constantly aware you're dealing with AI because they're clunky or unreliable. Transparency builds trust. If your AI tooling is genuinely helpful, you should be proud to talk about it.