5 ms·

>It's 20PB of examples used to form the shape of a 20GB model. You could show it 5GB of training data or 500EB of training data and it would still be 20GB - bec

>It's 20PB of examples used to form the shape of a 20GB model. You could show it 5GB of training data or 500EB of training data and it would still be 20GB - because it is not a compression algo, it's a 20GB shape formed by external data. You can compress 20PB of text to 20Gb or even less, if input is super repetitive. So the same with images, if 50% of the images are cats then you learn how to represent the cat pixels with a few vectors and then you could represent all the cats int he world doing all possible cat actions. But please have the courage to respond to this, when the AI is caught regurgitating the exact text from a popular book, the exact verses from a poem, the exact code function from some code , then how can you defend that is not memorizing things? If a human uses my poem(after they read it) and signs his name under it would you defend them?

Workaccount2 1y ago
The point is that it isn't compression. Its molding a plain structure iteratively into a ultra complex one. The model starts and ends at 20GB. It might have features that are reminiscent of compression or act like it, but under the hood there is nothing like zip, rar, H.265, or JPEG going on. And yes LLMs can recall exact material, but it is excerpts and fragments. There is statistical significance to it's ordering. Humans readily do this too (excerpts and fragments), most artists can draw a batman symbol (but not an episode of batman). That doesn't in anyway mean that artists should not be allowed to ever see a batman symbol. It means that artists shouldn't be allowed to get paid to draw one. And they are not. And LLMs are not exempt either. But the fix is output filtering, just like everything else that can violate copyright. Which is already being done (albeit poorly, but way better than 2 years ago), the same as artists will not draw the batman symbol for you despite being able to.
- simion314 1y ago
  OK, so we name it something different, you transform inputs into smaller outputs. If I make a script without AI that transforms someones poems without permission so sometimes it outputs the exact poems but sometimes it does it wrong, when is my script fair to use and when what I did is illegal. say my script contains words and matrixes of numbers the original poems are not directly inside, the script transformed them into vectors. maybe even simpler, I create a zip format where I randomly replace words with their synonym, or group of words with something equivalent. Would you defend this as original ? why my random transformations are not original while mathematical transformations you will defend ? And how can you suggest putting output filters to protect only the giants for copyright and everyone else gets screwed.
  - Workaccount2 1y ago
    YouTube actively filters small channel copied content too. We should be building robust copyright filters and everyone should be able to contribute their work to it. but that is a different issue than whether or not an LLM is legally allowed to view a work that is publicly available. Again, pretty much every artist is capable of off-hand copyright violation on the spot. This has been true forever. We don't bar them from seeing art to prevent this.
    - simion314 1y ago
      >but that is a different issue than whether or not an LLM is legally allowed to view a work that is publicly available. We are not talking about the scenario where LLM does a web search and "transforms" your blog as a response to a user prompt. We are talking about people writing code to torrent content, including copyrighted content as books and newspaper articles then this same people create soem software/script/blackbox and they put this copyrighted stuff inside and this same people also put a filter in the output to attempt and block SOME of the copyrighted material to be spitted out(they did it for Dune, Harry Poter , probably other popular stuff but for sure they did not done it for Romanian copyrighted material or some small writer). If I publish a poem I do not give implicit permission to anyone to use it as they see fit, and have scripts/software/backboxes train on it and spit parts of my poem out.

Workaccount2 1y ago

The point is that it isn't compression. Its molding a plain structure iteratively into a ultra complex one. The model starts and ends at 20GB. It might have features that are reminiscent of compression or act like it, but under the hood there is nothing like zip, rar, H.265, or JPEG going on. And yes LLMs can recall exact material, but it is excerpts and fragments. There is statistical significance to it's ordering. Humans readily do this too (excerpts and fragments), most artists can draw a batman symbol (but not an episode of batman). That doesn't in anyway mean that artists should not be allowed to ever see a batman symbol. It means that artists shouldn't be allowed to get paid to draw one. And they are not. And LLMs are not exempt either. But the fix is output filtering, just like everything else that can violate copyright. Which is already being done (albeit poorly, but way better than 2 years ago), the same as artists will not draw the batman symbol for you despite being able to.

simion314 1y ago
OK, so we name it something different, you transform inputs into smaller outputs. If I make a script without AI that transforms someones poems without permission so sometimes it outputs the exact poems but sometimes it does it wrong, when is my script fair to use and when what I did is illegal. say my script contains words and matrixes of numbers the original poems are not directly inside, the script transformed them into vectors. maybe even simpler, I create a zip format where I randomly replace words with their synonym, or group of words with something equivalent. Would you defend this as original ? why my random transformations are not original while mathematical transformations you will defend ? And how can you suggest putting output filters to protect only the giants for copyright and everyone else gets screwed.
- Workaccount2 1y ago
  YouTube actively filters small channel copied content too. We should be building robust copyright filters and everyone should be able to contribute their work to it. but that is a different issue than whether or not an LLM is legally allowed to view a work that is publicly available. Again, pretty much every artist is capable of off-hand copyright violation on the spot. This has been true forever. We don't bar them from seeing art to prevent this.
  - simion314 1y ago
    >but that is a different issue than whether or not an LLM is legally allowed to view a work that is publicly available. We are not talking about the scenario where LLM does a web search and "transforms" your blog as a response to a user prompt. We are talking about people writing code to torrent content, including copyrighted content as books and newspaper articles then this same people create soem software/script/blackbox and they put this copyrighted stuff inside and this same people also put a filter in the output to attempt and block SOME of the copyrighted material to be spitted out(they did it for Dune, Harry Poter , probably other popular stuff but for sure they did not done it for Romanian copyrighted material or some small writer). If I publish a poem I do not give implicit permission to anyone to use it as they see fit, and have scripts/software/backboxes train on it and spit parts of my poem out.