Yeah I have seen a lot of people bring up the question of essentially how training an AI to regurgitate code snippets is different from a human reading through

Yeah I have seen a lot of people bring up the question of essentially how training an AI to regurgitate code snippets is different from a human reading through and learning how to write similar code. To me they seem obviously and intrinsically different, and I think your post gets at some of the reasons. For one, an AI can ingest a much larger quantity of data much faster than a human. No human could possibly read all of the public code on GitHub in one lifetime. But more importantly, the method of ingestion and synthesis, as well as the nature of the application of what’s learned, is vastly different. Generally a human reading through OSS code to learn isn’t going to then permanently store verbatim segments of it in their memory to regurgitate later. They may copy a bit of code while learning, but that then morphs into an independent understanding of the type of problem being solved and one way to approach that problem. Copilot OTOH has no capacity to understand different approaches to solving a problem, to compare one against the other, and so on. All it knows is which is used more frequently, with no capacity to map that against metrics of long term maintenance, CI suite success, readability, and so on. As for the poison pill problem, I think it actually illustrates the difference quite well. A simple comment explaining that the code is a booby trap would be enough for any human reading the code to know how to avoid it. The AI on the other hand has no capacity to understand that comment and to know not to put the code into its dataset.