6 ms·

Another factor to consider is that neural nets can function as lossy compression, which becomes extremely evident when using models that are overfit. Sometimes

Another factor to consider is that neural nets can function as lossy compression, which becomes extremely evident when using models that are overfit. Sometimes they're so overfit that the compression isn't even lossy, and the data is encoded verbatim in the NN.

TeMPOraL 3y ago
Yes, but this then hits against learning/understanding and compression being fundamentally the same thing. I can't think of a better way to argue in favor of "it's fine if human does it, therefore it's fine if LLM does it", than from the "lossy compression" angle.
- RandomLensman 3y ago
  We can have different rules for humans than for machines. In fact, that happens all the time.
- cyborgx7 3y ago
  It's fine for a human to remember it. It's not fine for a human redistribute it for money (legally speaking). That's copyright infringement.
  - Robotbeat 3y ago
    Correct, just like it’s infringement to reproduce an article from memory using pen and paper intentionally. The person deciding to do that bears responsibility. OpenAI would be liable IFF they were intentionally facilitating that, instead of it being an undesired artifact from overfitting.
    - cyborgx7 3y ago
      I'm pretty sure if you reproduce a work from memory by accident, because you didn't notice your subconscious had just stored the entire article and is now reproducing it word for word, you'd still be guilty of copyright infringement.
      - yencabulator 3y ago
        The music business is full of examples of that.
        freejazz 3y ago
        It's super obnoxious when people who have no understanding of the law, point to industry patterns or behaviors as examples of what is legal, not knowing the law and not knowing whether or not the thing they are pointing to is legal. The music business is also full of copyright infringement litigation. You also are not taking into account whether what is copied by an artist is covered by copyright when you made your statement. Do you know what's covered in music copyrights, such that your statement ever had any value for anyone else here?
      - Robotbeat 3y ago
        Or the person doing the prompt hacking intending to produce this result from you would be.
    - freejazz 3y ago
      That's not true at all. Copyright infringement is a strict liability offense with no inquiry in to the state of the mind of the infringer from a liability perspective. The state of mind of the infringer is only relevant to the issue of willful infringement.
      - anticensor 3y ago
        Willful vs less-than-willful infringement are definitely two separate types of offences, as indicated by the difference of penalty.
        freejazz 3y ago
        It's just "infringement" and "willful infringement" there is no "less-than-willful infringement". Willful infringement is punitive with increased damages and increased burden to show - it's in the freakin' statute.
- heavyset_go 3y ago
  It's not okay for a human to pirate, plagiarize, violate IP rights and laws, etc. But I disagree with the underlying assumption that you can anthropomorphize LLMs. Gradient descent and backpropagation don't take place in the brain. LLMs "learn" in the same way that Excel sheets "learn". Humans are living beings with needs and rights. A person being able to legally squat in a home doesn't mean that a drone occupying property for some amount of time also has squatter's rights, even though you could easily and affordably automate and scale the deployment of drones to live and hide away on properties long enough to attain rights regarding properties all over the country.
  - sgt101 3y ago
    also if I write and article and quote some "text like this" [1] then that's not plagerism, but if my arguement is that the underlying assumption that you can anthropomorphize LLMs. Gradient descent and backpropagation don't take place in the brain. LLMs "learn" in the same way that Excel sheets "learn". Well, that's plagiarism and it's not allowed and people will get peeved and my career might get damaged. I await the HN ban with fear.. [1] I'm not even doing referencing - so I am surely an LLM.
  - pas 3y ago
    sure, but if I use an LLM to write a novel/article, I can be sued in civil court not the LLM. but, more importantly, OpenAI can also be sued for tortious interference? (basically the civil equivalent of accessory)
    - heavyset_go 3y ago
      Whoever operates the LLM, in this case OpenAI, engaged in copyright infringement through the unauthorized modification, reproduction and distribution of content to you.
      - Robotbeat 3y ago
        The person doing the requesting did.
- wokwokwok 3y ago
  Is there some LLM meta where understanding and compression are argued to be the same thing I’m not aware of? Anyone got more details on this? Superficially it sounds like total BS; a highly compressed zip file does not exhibit any characteristics of learning. Algorithmically derived highly compressed video streams do not exhibit characteristics of learning. ? I’ve vaguely heard the learning can be considered to exhibit the characteristics of compression in that understanding of content (eg. segmentation of video content resulting in more highly compressed videos) can lead to better compression schemes. …but saying you can “do a with b” and “a and b are fundamentally the same thing” seems like a leap…? It seems self evident you can have compression without comprehension.
  - adroniser 3y ago
    Suppose you wanted to train an LLM to do addition. An LLM has limited parameters. If an LLM had infinite parameters it could just memorize the results of every single addition question in existence and could not claim to have understood anything. Because it has finite parameters, if an LLM wants to get a lower loss on all addition questions, it needs to come up with a general algorithm to perform addition. Indeed, Neel Nanda trained a transformer to do addition mod 113 on relatively few examples, and it eventually learned some cursed Fourier transform mumbo jumbo to get 0 loss https://twitter.com/robertskmiles/status/1663534255249453056 https://twitter.com/robertskmiles/status/1663534255249453056. And the fact it has developed this "understanding" as an ability to learn a general pattern in the training data enables it to compress. I claim that the number of bits required to encode the general algorithm is fewer than the number of bits required to memorize every single example. If it weren't then the transformer would simply memorize every single example. But if it doesn't have space then it is forced to try to compress by developing a general model. And the ability to compress enables you to construct a language model. Essentially, the more things compress, the higher the likelihood you assign them. Given a sequence of tokens say "the cat sat on the", we should expect "the cat sat on the mat" to compress into fewer bits than "the cat sat on the door". This is because the latter is far more common and intuitively more common sequences should compress more. You can then look at the number of bits used for every single choice of token following "the cat sat on the" and thus develop a probability distribution for the next token. The exact details of this I'm unclear on. https://www.hendrik-erz.de/post/why-gzip-just-beat-a-large-language-model https://www.hendrik-erz.de/post/why-gzip-just-beat-a-large-l... this gives a good summary.
- dns_snek 3y ago
  > fundamentally the same thing I fundamentally disagree. That's not some established fact, just a narrative used by those who wish to plagiarize using "AI".
- devsda 3y ago
  Humans are defined not just by their abilities but by their limitations too. We celebrate our achievements because sometimes they surpass the limitations of an average human. Our collective human limitations(physical, mental and temporal) are sort of invisible implicit rules that we all follow in one way or the other. If an entity is not bound by those rules then I don't see why that entity should be treated the same as a human. Companies already make this differentiation. For example take captcha and bot detection. Some of the heuristics are based on inherent human limitations like response time, click time, mouse acceleration etc. I doubt youtube or any other streaming service will be happy if you want to stream all their videos to train a hypothetical human like AI(which views and prepares notes like a human) at a hugely accelerated speed compared to a regular human. You can guess how quickly they will cite fair usage policies. What I want to say is there are fundamental differences between a human and an AI. So, we should not be quick to dismiss any concerns just because AI can "mimic" humans in certain areas.
- anileated 3y ago
  I can’t think of a better way to argue in favor of “LLMs are copyright laundering machines” than from the humanness angle. Humans have rights, software tools don’t. If you grant an LLM the full set of human rights, then it can consume information, regurgitate copyrighted works, and use it to generate money for itself. However, considering blatantly obvious theft as “homage” goes hand in hand with free will, agency, being in control of yourself, not being enslaved and abused, etc. Pondering various scenarios along those lines really gets to the heart of why an LLM is so very much not a human, and how subjecting it to the same treatment as humans is a ridiculous notion. If you don’t grant LLM human rights, then ClosedAI’s stance is basically that pirating works is OK because they pass them through a black box of if conditions and it leads to results that they can monetize. That’s such a solid argument, it’ll surely play well in the court of law. Training data is not an “LLM does it”; first because “it” here is not “learning” or understanding in human sense (otherwise you would have to presume that an LLM is a human), and second because a software tool doesn’t have agency and it’s really just Microsoft using a tool based on copyrighted works to generate profit.
  - kelseyfrog 3y ago
    Humans don't exactly have the greatest track record of granting other humans rights. I don't presume they'll get it any better with AI. What I expect to happen is whoever has the most influence and power will get what they want and we'll end up raising a generation with the implicit understanding of "that's just how things are," natural order, truth, reality, and all that jazz. The only thing that ever changes outcomes is if the contradiction status quo is incapable of being managed.
    - anileated 3y ago
      I can’t argue for or against whether LLMs should have rights or not… I can only point out the hypocrisy of claiming LLMs are “like human” enough and independent enough that their operators-become-slaveowners cannot be held to account on any copyright matters, but also claiming that LLMs are not like human at all lest someone demans them to have rights and nukes the industry.
accrual 3y ago
> Sometimes they're so overfit that the compression isn't even lossy, and the data is encoded verbatim in the NN. Here's an article from November 2023 that discusses this: https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html https://not-just-memorization.github.io/extracting-training-...

TeMPOraL 3y ago

Yes, but this then hits against learning/understanding and compression being fundamentally the same thing. I can't think of a better way to argue in favor of "it's fine if human does it, therefore it's fine if LLM does it", than from the "lossy compression" angle.

RandomLensman 3y ago
We can have different rules for humans than for machines. In fact, that happens all the time.
cyborgx7 3y ago
It's fine for a human to remember it. It's not fine for a human redistribute it for money (legally speaking). That's copyright infringement.
- Robotbeat 3y ago
  Correct, just like it’s infringement to reproduce an article from memory using pen and paper intentionally. The person deciding to do that bears responsibility. OpenAI would be liable IFF they were intentionally facilitating that, instead of it being an undesired artifact from overfitting.
  - cyborgx7 3y ago
    I'm pretty sure if you reproduce a work from memory by accident, because you didn't notice your subconscious had just stored the entire article and is now reproducing it word for word, you'd still be guilty of copyright infringement.
    - yencabulator 3y ago
      The music business is full of examples of that.
      - freejazz 3y ago
        It's super obnoxious when people who have no understanding of the law, point to industry patterns or behaviors as examples of what is legal, not knowing the law and not knowing whether or not the thing they are pointing to is legal. The music business is also full of copyright infringement litigation. You also are not taking into account whether what is copied by an artist is covered by copyright when you made your statement. Do you know what's covered in music copyrights, such that your statement ever had any value for anyone else here?
    - Robotbeat 3y ago
      Or the person doing the prompt hacking intending to produce this result from you would be.
  - freejazz 3y ago
    That's not true at all. Copyright infringement is a strict liability offense with no inquiry in to the state of the mind of the infringer from a liability perspective. The state of mind of the infringer is only relevant to the issue of willful infringement.
    - anticensor 3y ago
      Willful vs less-than-willful infringement are definitely two separate types of offences, as indicated by the difference of penalty.
      - freejazz 3y ago
        It's just "infringement" and "willful infringement" there is no "less-than-willful infringement". Willful infringement is punitive with increased damages and increased burden to show - it's in the freakin' statute.
heavyset_go 3y ago
It's not okay for a human to pirate, plagiarize, violate IP rights and laws, etc. But I disagree with the underlying assumption that you can anthropomorphize LLMs. Gradient descent and backpropagation don't take place in the brain. LLMs "learn" in the same way that Excel sheets "learn". Humans are living beings with needs and rights. A person being able to legally squat in a home doesn't mean that a drone occupying property for some amount of time also has squatter's rights, even though you could easily and affordably automate and scale the deployment of drones to live and hide away on properties long enough to attain rights regarding properties all over the country.
- sgt101 3y ago
  also if I write and article and quote some "text like this" [1] then that's not plagerism, but if my arguement is that the underlying assumption that you can anthropomorphize LLMs. Gradient descent and backpropagation don't take place in the brain. LLMs "learn" in the same way that Excel sheets "learn". Well, that's plagiarism and it's not allowed and people will get peeved and my career might get damaged. I await the HN ban with fear.. [1] I'm not even doing referencing - so I am surely an LLM.
- pas 3y ago
  sure, but if I use an LLM to write a novel/article, I can be sued in civil court not the LLM. but, more importantly, OpenAI can also be sued for tortious interference? (basically the civil equivalent of accessory)
  - heavyset_go 3y ago
    Whoever operates the LLM, in this case OpenAI, engaged in copyright infringement through the unauthorized modification, reproduction and distribution of content to you.
    - Robotbeat 3y ago
      The person doing the requesting did.
wokwokwok 3y ago
Is there some LLM meta where understanding and compression are argued to be the same thing I’m not aware of? Anyone got more details on this? Superficially it sounds like total BS; a highly compressed zip file does not exhibit any characteristics of learning. Algorithmically derived highly compressed video streams do not exhibit characteristics of learning. ? I’ve vaguely heard the learning can be considered to exhibit the characteristics of compression in that understanding of content (eg. segmentation of video content resulting in more highly compressed videos) can lead to better compression schemes. …but saying you can “do a with b” and “a and b are fundamentally the same thing” seems like a leap…? It seems self evident you can have compression without comprehension.
- adroniser 3y ago
  Suppose you wanted to train an LLM to do addition. An LLM has limited parameters. If an LLM had infinite parameters it could just memorize the results of every single addition question in existence and could not claim to have understood anything. Because it has finite parameters, if an LLM wants to get a lower loss on all addition questions, it needs to come up with a general algorithm to perform addition. Indeed, Neel Nanda trained a transformer to do addition mod 113 on relatively few examples, and it eventually learned some cursed Fourier transform mumbo jumbo to get 0 loss https://twitter.com/robertskmiles/status/1663534255249453056 https://twitter.com/robertskmiles/status/1663534255249453056. And the fact it has developed this "understanding" as an ability to learn a general pattern in the training data enables it to compress. I claim that the number of bits required to encode the general algorithm is fewer than the number of bits required to memorize every single example. If it weren't then the transformer would simply memorize every single example. But if it doesn't have space then it is forced to try to compress by developing a general model. And the ability to compress enables you to construct a language model. Essentially, the more things compress, the higher the likelihood you assign them. Given a sequence of tokens say "the cat sat on the", we should expect "the cat sat on the mat" to compress into fewer bits than "the cat sat on the door". This is because the latter is far more common and intuitively more common sequences should compress more. You can then look at the number of bits used for every single choice of token following "the cat sat on the" and thus develop a probability distribution for the next token. The exact details of this I'm unclear on. https://www.hendrik-erz.de/post/why-gzip-just-beat-a-large-language-model https://www.hendrik-erz.de/post/why-gzip-just-beat-a-large-l... this gives a good summary.
dns_snek 3y ago
> fundamentally the same thing I fundamentally disagree. That's not some established fact, just a narrative used by those who wish to plagiarize using "AI".
devsda 3y ago
Humans are defined not just by their abilities but by their limitations too. We celebrate our achievements because sometimes they surpass the limitations of an average human. Our collective human limitations(physical, mental and temporal) are sort of invisible implicit rules that we all follow in one way or the other. If an entity is not bound by those rules then I don't see why that entity should be treated the same as a human. Companies already make this differentiation. For example take captcha and bot detection. Some of the heuristics are based on inherent human limitations like response time, click time, mouse acceleration etc. I doubt youtube or any other streaming service will be happy if you want to stream all their videos to train a hypothetical human like AI(which views and prepares notes like a human) at a hugely accelerated speed compared to a regular human. You can guess how quickly they will cite fair usage policies. What I want to say is there are fundamental differences between a human and an AI. So, we should not be quick to dismiss any concerns just because AI can "mimic" humans in certain areas.
anileated 3y ago
I can’t think of a better way to argue in favor of “LLMs are copyright laundering machines” than from the humanness angle. Humans have rights, software tools don’t. If you grant an LLM the full set of human rights, then it can consume information, regurgitate copyrighted works, and use it to generate money for itself. However, considering blatantly obvious theft as “homage” goes hand in hand with free will, agency, being in control of yourself, not being enslaved and abused, etc. Pondering various scenarios along those lines really gets to the heart of why an LLM is so very much not a human, and how subjecting it to the same treatment as humans is a ridiculous notion. If you don’t grant LLM human rights, then ClosedAI’s stance is basically that pirating works is OK because they pass them through a black box of if conditions and it leads to results that they can monetize. That’s such a solid argument, it’ll surely play well in the court of law. Training data is not an “LLM does it”; first because “it” here is not “learning” or understanding in human sense (otherwise you would have to presume that an LLM is a human), and second because a software tool doesn’t have agency and it’s really just Microsoft using a tool based on copyrighted works to generate profit.
- kelseyfrog 3y ago
  Humans don't exactly have the greatest track record of granting other humans rights. I don't presume they'll get it any better with AI. What I expect to happen is whoever has the most influence and power will get what they want and we'll end up raising a generation with the implicit understanding of "that's just how things are," natural order, truth, reality, and all that jazz. The only thing that ever changes outcomes is if the contradiction status quo is incapable of being managed.
  - anileated 3y ago
    I can’t argue for or against whether LLMs should have rights or not… I can only point out the hypocrisy of claiming LLMs are “like human” enough and independent enough that their operators-become-slaveowners cannot be held to account on any copyright matters, but also claiming that LLMs are not like human at all lest someone demans them to have rights and nukes the industry.

accrual 3y ago

> Sometimes they're so overfit that the compression isn't even lossy, and the data is encoded verbatim in the NN. Here's an article from November 2023 that discusses this: https://not-just-memorization.github.io/extracting-training-data-from-chatgpt.html https://not-just-memorization.github.io/extracting-training-...