> Computers do have such accurate memories, and machine learning systems, which are optimized for things like maximum likelihood estimation, can and do reproduc

> Computers do have such accurate memories, and machine learning systems, which are optimized for things like maximum likelihood estimation, can and do reproduce both copyrightable and non-copyrightable elements of works that they've been trained on. After all, the maximum likelihood continuation of a fragment of a text or a song is ... the complete original work. And the ability to reproduce the complete original work would, other things being equal, reduce loss in training. After all, that's something someone might specifically ask for, and if the system could oblige, it would be doing a better job of providing what the user wanted. This actually depends on how you train the model. Techniques such as using unlikelihood to penalize plagiarizing models exists. Microsoft/OpenAI are of course aware of those techniques but have chosen not to use them. The reason why is not difficult to figure out. Because the model hasn't learned how to implement sparse matrix multiplication in C, it has learned how to spit out someone else's code with a few variable names changed. Not unlike how many CS students not cut out for software development try to pass their entry-level programming courses. Professors use anti-cheating software to catch cheating students. Such software would catch Codex too and expose it as incompetent. Hence why it is not used.