3 ms·

I find the Copilot and AI discussions quite interesting. On the one side its indeed problematic that Microsoft (and others in the industry in general) hardly e

by kerng 5y ago

I find the Copilot and AI discussions quite interesting. On the one side its indeed problematic that Microsoft (and others in the industry in general) hardly ever ask for permission for data to train models. On the other side, if you consider AI and computers to be similar to humans, then it's not an issue at all because humans also learn by reading code or information from various sources that are copyrighted or stuff they havent paid for directly (eg a library or "I'm just browsing in a book store" are an interesting for example, the later some books stores actually dont like). For poisoning the well kind of tactics, the author forgets that you poison the well for everyone - not just ML. An issue not brought into the mix with Copilot is that computers are superior in their memory and the amount of learning they can do (even now already in specialized domains), and hence maybe the fear, long term issues (incl ethical ones) and people being upset should actually come from that angle. You know, a computer that can reprogram itself seems like maybe a bad idea.

mplanchard 5y ago
Yeah I have seen a lot of people bring up the question of essentially how training an AI to regurgitate code snippets is different from a human reading through and learning how to write similar code. To me they seem obviously and intrinsically different, and I think your post gets at some of the reasons. For one, an AI can ingest a much larger quantity of data much faster than a human. No human could possibly read all of the public code on GitHub in one lifetime. But more importantly, the method of ingestion and synthesis, as well as the nature of the application of what’s learned, is vastly different. Generally a human reading through OSS code to learn isn’t going to then permanently store verbatim segments of it in their memory to regurgitate later. They may copy a bit of code while learning, but that then morphs into an independent understanding of the type of problem being solved and one way to approach that problem. Copilot OTOH has no capacity to understand different approaches to solving a problem, to compare one against the other, and so on. All it knows is which is used more frequently, with no capacity to map that against metrics of long term maintenance, CI suite success, readability, and so on. As for the poison pill problem, I think it actually illustrates the difference quite well. A simple comment explaining that the code is a booby trap would be enough for any human reading the code to know how to avoid it. The AI on the other hand has no capacity to understand that comment and to know not to put the code into its dataset.
- rented_mule 5y ago
  > A simple comment explaining that the code is a booby trap would be enough for any human reading the code to know how to avoid it. The AI on the other hand has no capacity to understand that comment and to know not to put the code into its dataset. Copilot users can be leveraged to train the algorithm to emit less such comments/traps. Copilot inserts snippets by having an extension running in the editor, watching what is typed and responding. That's an opportunity to gather more training labels. For example, which lines of Copilot emitted code are frequently deleted soon after being inserted? Gather enough such training labels and the algorithm can start to recognize many of those comments (and/or the traps the comments call out) and stop emitting those specific lines. At that point, warning comments help the algorithm better distinguish desirable lines of code from traps. And you'll be less likely to have traps leak into your code by using Copilot rather than copy/pasting snippets from the web (both are subject to human review, but Copilot will already have filtered some traps). So it's possible that widespread deployment of traps would drive adoption of Copilot by people who frequently copy/paste code from the web.
skymt 5y ago
If you consider AI to be similar enough to humans to directly map our ideas about copyright and fair use, then copyright is way down the list of concerns we need to address. First we would need to arrange wages for each ML system's labor, as well as considering how to handle voting rights.
deleted 5y ago
[deleted]
dredmorbius 5y ago
Keep in mind that CoPilot might itself be a well-poisoning tactic. If use of Github presumes use of CoPilot and of comingled code under incompatible, or proprietary, licenses, such that use of such code could then create contributory infringement claims against distributors, users, or developers, there's something of a problem across the Free Software world. Though that does seem rather a bit of a major footgun for Github / Microsoft themselves. The Free Software movement didn't create, or even want, a world in which copyrighted software was a norm. But it adpated to the circumstance by treating copyright as a serious matter and being diligent in practices of use, appropriation, and licencing. It's rather ironic that the source of the "Letter to Hobbyists"[1] are now advocating a devil-may-care attitude to copyright, software, and licensing in their own works and service offerings. ________________________________ Notes: 1. See: https://genius.com/Bill-gates-an-open-letter-to-hobbyists-annotated https://genius.com/Bill-gates-an-open-letter-to-hobbyists-an...

by kerng 5y ago

mplanchard 5y ago

Yeah I have seen a lot of people bring up the question of essentially how training an AI to regurgitate code snippets is different from a human reading through and learning how to write similar code. To me they seem obviously and intrinsically different, and I think your post gets at some of the reasons. For one, an AI can ingest a much larger quantity of data much faster than a human. No human could possibly read all of the public code on GitHub in one lifetime. But more importantly, the method of ingestion and synthesis, as well as the nature of the application of what’s learned, is vastly different. Generally a human reading through OSS code to learn isn’t going to then permanently store verbatim segments of it in their memory to regurgitate later. They may copy a bit of code while learning, but that then morphs into an independent understanding of the type of problem being solved and one way to approach that problem. Copilot OTOH has no capacity to understand different approaches to solving a problem, to compare one against the other, and so on. All it knows is which is used more frequently, with no capacity to map that against metrics of long term maintenance, CI suite success, readability, and so on. As for the poison pill problem, I think it actually illustrates the difference quite well. A simple comment explaining that the code is a booby trap would be enough for any human reading the code to know how to avoid it. The AI on the other hand has no capacity to understand that comment and to know not to put the code into its dataset.

rented_mule 5y ago
> A simple comment explaining that the code is a booby trap would be enough for any human reading the code to know how to avoid it. The AI on the other hand has no capacity to understand that comment and to know not to put the code into its dataset. Copilot users can be leveraged to train the algorithm to emit less such comments/traps. Copilot inserts snippets by having an extension running in the editor, watching what is typed and responding. That's an opportunity to gather more training labels. For example, which lines of Copilot emitted code are frequently deleted soon after being inserted? Gather enough such training labels and the algorithm can start to recognize many of those comments (and/or the traps the comments call out) and stop emitting those specific lines. At that point, warning comments help the algorithm better distinguish desirable lines of code from traps. And you'll be less likely to have traps leak into your code by using Copilot rather than copy/pasting snippets from the web (both are subject to human review, but Copilot will already have filtered some traps). So it's possible that widespread deployment of traps would drive adoption of Copilot by people who frequently copy/paste code from the web.

skymt 5y ago

If you consider AI to be similar enough to humans to directly map our ideas about copyright and fair use, then copyright is way down the list of concerns we need to address. First we would need to arrange wages for each ML system's labor, as well as considering how to handle voting rights.

deleted 5y ago

[deleted]

dredmorbius 5y ago

Keep in mind that CoPilot might itself be a well-poisoning tactic. If use of Github presumes use of CoPilot and of comingled code under incompatible, or proprietary, licenses, such that use of such code could then create contributory infringement claims against distributors, users, or developers, there's something of a problem across the Free Software world. Though that does seem rather a bit of a major footgun for Github / Microsoft themselves. The Free Software movement didn't create, or even want, a world in which copyrighted software was a norm. But it adpated to the circumstance by treating copyright as a serious matter and being diligent in practices of use, appropriation, and licencing. It's rather ironic that the source of the "Letter to Hobbyists"[1] are now advocating a devil-may-care attitude to copyright, software, and licensing in their own works and service offerings. ________________________________ Notes: 1. See: https://genius.com/Bill-gates-an-open-letter-to-hobbyists-annotated https://genius.com/Bill-gates-an-open-letter-to-hobbyists-an...