3 ms·

As helpful as it may be, AI can emit verbatim training data where the training data for the particular issue is scarce. I have, unfortunately, seen it myself in

As helpful as it may be, AI can emit verbatim training data where the training data for the particular issue is scarce. I have, unfortunately, seen it myself in an interview take-home assignment where the solution for a particular sub-problem was 1:1 copied from a seemingly obscure chinese blog and had a completely different coding style than the rest. (The rest also had numerous inconsistencies and the solution had AI written all over it.) For an open source project inadvertently accepting something like that may spell legal trouble long after the original contributor disappeared, so until the legal status of AI-generated code is clarified in court, there are good reasons to ask contributors to limit themselves or completely forego using an AI assistant.

barfbagginus 2y ago
I'd be curious about the problem and context that elicited the verbatim output. Did it have detailed descriptions of a real system in production that you were supposed to extend, including API docs and other system specific elements? Or was it a generic homework assignment? If it's emitting verbatim solutions in my context, that is surprising. I write up custom requirements, have a conversation with a two sided critique and requirement negotiation and gap analysis vs the exist system. I then generate and critique a test plan, generate stub tests and stub implementations in line with existing code, then implement and test the stories by hand on top of that. Nothing that "has AI written all over it" gets generated, since we're having a custom conversation and arriving at custom solutions. If it does copy public but not open code, it's tiny generic snippets or, design patterns, rather than, say, sophisticated algorithms.. not anything likely to cause damage to another org or it's market position. I'm prepared to field a cease and desist or put down a lawsuit by yanking affected code. And I'm only going to do this for projects that I own, or whose authors are taking a permissive stance. I'm boycotting any project taking a hard line against AI. My projects are affected in the case of a full retroactive AI ban that makes all AI generated codes unlawful. However, due to the transformative nature of processes like mine, and the vast majority of AI output, I really doubt that'll happen. I'll grant that for a Linux distribution, the risk is much higher - Linux most definitely impacts market positions, and yoinking someone's public blog code could lead to rework after a cease and desist. But even there, I feel the legal risk is symbolic rather than existential or even eminent. Legal risk won't be a driver for most projects, and might not be the driver for gentoo. It'll be prejudice against people who can't easily contribute without AI assistance. There's certainly an element of that in the gentoo discussion, with AI contributions being misrepresented, discounted and dismissed outright. That'll prove short-sighted, and drive contribution to inclusive projects with strong controls ensuring their AI use is responsible and transformative. I wish Gentoo would choose to be an example of that. And I'm sad to say I plan to boycott it until it changes stance. But I am sure other distributions will step up here and work to mitigate the legal risks in return for the obvious accessibility benefits.
- janosdebugs 2y ago
  I found that LLMs tends to emit training data when you ask it simple questions, such as "connect a MySQL server". In one of my experiments it emitted a StackOverflow example 1:1, including the mistakes that were made there (no/faulty error handling). Regardless if it is legal to use code in such fashion, more experienced coders tend to rewrite the code an LLM emits, so it's less likely to lead to actual problems down the line. I've seen a lot of less experienced people take LLM-generated code and use it as-is, which can lead to problems. In case of an open source project one of the main problems with LLMs is that if a contributor does contribute infringing material, inadvertently or not, the project will be left holding the bag for it. The contributor may disappear because all that's known about them may just be an alias and an email address. This is especially true if a project doesn't have the backing of a large, well-funded organization that can fight out legal battles on behalf of the contributors. The legal situation on Internet-trained LLM usage isn't clear and I can totally understand a small, underfunded project not wanting to go up against large corporations. I think respecting the project maintainers wishes in this case is the right thing to do. I've seen several examples where an LLM was used as a tool for accessibility (vision assistance, etc) and I do think that that's going to be one of the areas where it will be most valuable to society as a whole. Unfortunately, the legal system moves slowly and it will be a few more years until this mess gets cleaned up. I also think you are doing the right thing and I wish more people would follow your example. If a particular project deems the risk of using an LLM too high, contributors should follow the wishes of the maintainers and not contribute to that project with the usage of an LLM. Depending on the country, petitioning lawmakers may also help move the needle in the desired direction.

barfbagginus 2y ago

I'd be curious about the problem and context that elicited the verbatim output. Did it have detailed descriptions of a real system in production that you were supposed to extend, including API docs and other system specific elements? Or was it a generic homework assignment? If it's emitting verbatim solutions in my context, that is surprising. I write up custom requirements, have a conversation with a two sided critique and requirement negotiation and gap analysis vs the exist system. I then generate and critique a test plan, generate stub tests and stub implementations in line with existing code, then implement and test the stories by hand on top of that. Nothing that "has AI written all over it" gets generated, since we're having a custom conversation and arriving at custom solutions. If it does copy public but not open code, it's tiny generic snippets or, design patterns, rather than, say, sophisticated algorithms.. not anything likely to cause damage to another org or it's market position. I'm prepared to field a cease and desist or put down a lawsuit by yanking affected code. And I'm only going to do this for projects that I own, or whose authors are taking a permissive stance. I'm boycotting any project taking a hard line against AI. My projects are affected in the case of a full retroactive AI ban that makes all AI generated codes unlawful. However, due to the transformative nature of processes like mine, and the vast majority of AI output, I really doubt that'll happen. I'll grant that for a Linux distribution, the risk is much higher - Linux most definitely impacts market positions, and yoinking someone's public blog code could lead to rework after a cease and desist. But even there, I feel the legal risk is symbolic rather than existential or even eminent. Legal risk won't be a driver for most projects, and might not be the driver for gentoo. It'll be prejudice against people who can't easily contribute without AI assistance. There's certainly an element of that in the gentoo discussion, with AI contributions being misrepresented, discounted and dismissed outright. That'll prove short-sighted, and drive contribution to inclusive projects with strong controls ensuring their AI use is responsible and transformative. I wish Gentoo would choose to be an example of that. And I'm sad to say I plan to boycott it until it changes stance. But I am sure other distributions will step up here and work to mitigate the legal risks in return for the obvious accessibility benefits.

janosdebugs 2y ago
I found that LLMs tends to emit training data when you ask it simple questions, such as "connect a MySQL server". In one of my experiments it emitted a StackOverflow example 1:1, including the mistakes that were made there (no/faulty error handling). Regardless if it is legal to use code in such fashion, more experienced coders tend to rewrite the code an LLM emits, so it's less likely to lead to actual problems down the line. I've seen a lot of less experienced people take LLM-generated code and use it as-is, which can lead to problems. In case of an open source project one of the main problems with LLMs is that if a contributor does contribute infringing material, inadvertently or not, the project will be left holding the bag for it. The contributor may disappear because all that's known about them may just be an alias and an email address. This is especially true if a project doesn't have the backing of a large, well-funded organization that can fight out legal battles on behalf of the contributors. The legal situation on Internet-trained LLM usage isn't clear and I can totally understand a small, underfunded project not wanting to go up against large corporations. I think respecting the project maintainers wishes in this case is the right thing to do. I've seen several examples where an LLM was used as a tool for accessibility (vision assistance, etc) and I do think that that's going to be one of the areas where it will be most valuable to society as a whole. Unfortunately, the legal system moves slowly and it will be a few more years until this mess gets cleaned up. I also think you are doing the right thing and I wish more people would follow your example. If a particular project deems the risk of using an LLM too high, contributors should follow the wishes of the maintainers and not contribute to that project with the usage of an LLM. Depending on the country, petitioning lawmakers may also help move the needle in the desired direction.