3 ms·

Of course the trained models, being derived from GPL code, become derivative work, and by extension - also GPL? Right?

compiler-guy 2mo ago
Maybe. The courts haven't exactly ruled yet. And it is the risk here that the GNU project wants to avoid. Will things probably be OK? Sure. Probably. But GNU is particularly risk averse when it comes to licensing.
jimmaswell 2mo ago
No, you can't copyright an idea, only an expression of an idea, and LLMs operate at the level of ideas. They don't literally stitch together code from training.
- bulder 2mo ago
  LLMs very much operate on the level of text. That text can represent ideas, but it can also directly be a copyrighted work. I am however very interested in the novel interpretation of copyright that says that you can do whatever as long as your compression is lossy.
- alerighi 2mo ago
  Well, not exactly. A LLM is still a computer, doesn't have an intelligence (beside being called AI). That means that their output is a mere computation of their input data, and their input data it's the stuff that was used for the training. If you imagine it as a "box" you feed into it material and a prompt and it spits out the same material rearranged to do what you did ask for. It does nothing more than a permutation of their input data, as does any computer program, of course in extremely complex and obscure way, but if you reason it abstractly it's the same things Turing theorized almost a century years ago, input -> BOX -> output. So *of course* the output *is* a derived work of the input, and thus a GPL code should not really used as a training set.
  - jimmaswell 2mo ago
    I'm pretty familiar with the technical details of how an LLM works. They absolutely operate more on a level of abstract concepts than acting like some kind of text-Frankenstein as you're suggesting (as if the model is Frankenstein and its output is his monster). Here's a very basic example: if you have access to a typical language model's weights, you can subtract the embedding for "man" from the embedding for "king", add the embedding for "woman", and land somewhere very close to the embedding for "queen". Why is "intelligence", whatever that means, a prerequisite for a machine to process ideas in the abstract?

Maybe. The courts haven't exactly ruled yet. And it is the risk here that the GNU project wants to avoid. Will things probably be OK? Sure. Probably. But GNU is particularly risk averse when it comes to licensing.

jimmaswell 2mo ago

No, you can't copyright an idea, only an expression of an idea, and LLMs operate at the level of ideas. They don't literally stitch together code from training.

bulder 2mo ago
LLMs very much operate on the level of text. That text can represent ideas, but it can also directly be a copyrighted work. I am however very interested in the novel interpretation of copyright that says that you can do whatever as long as your compression is lossy.
alerighi 2mo ago
Well, not exactly. A LLM is still a computer, doesn't have an intelligence (beside being called AI). That means that their output is a mere computation of their input data, and their input data it's the stuff that was used for the training. If you imagine it as a "box" you feed into it material and a prompt and it spits out the same material rearranged to do what you did ask for. It does nothing more than a permutation of their input data, as does any computer program, of course in extremely complex and obscure way, but if you reason it abstractly it's the same things Turing theorized almost a century years ago, input -> BOX -> output. So *of course* the output *is* a derived work of the input, and thus a GPL code should not really used as a training set.
- jimmaswell 2mo ago
  I'm pretty familiar with the technical details of how an LLM works. They absolutely operate more on a level of abstract concepts than acting like some kind of text-Frankenstein as you're suggesting (as if the model is Frankenstein and its output is his monster). Here's a very basic example: if you have access to a typical language model's weights, you can subtract the embedding for "man" from the embedding for "king", add the embedding for "woman", and land somewhere very close to the embedding for "queen". Why is "intelligence", whatever that means, a prerequisite for a machine to process ideas in the abstract?