7 ms·

Why does it have to be verbatim? Seriously, this I don't understand. If I produce a terrible shakycam recording of a film while sitting in a movie theater, it'

• ilikehurdles 1y ago
  If you read a book and later understand its plot but can only explain it in your own words, did you copy it? The model isn’t storing the book.
  • kxrm 1y ago
    The model doesn't "understand its plot". So I am not sure this is a good analogy.
    • ilikehurdles 1y ago
      To what extent connections in a neural network are analogous to connections between neurons in your brain is open to interpretation and study, but the point of the analogy is that in neither case is a copy being made.
      • jamiek88 1y ago
        Yeah but a copy IS made. A human just reads. The machine copies the full text then compresses a lossy copy in its weights. You keep dodging that with tortuous analogies of a human learning. I’m sure all these ‘clever’ questions would be useful if this trial was about humans but it’s not.
        • Workaccount2 1y ago
          Model training works roughly by feeding the model a text excerpt and then hiding the last word in the excerpt. The model is then asked to "guess" what the final word is. It will then move around it's weights until the guess sufficiently matches the actual token. Then the process repeats. The training material is used to play this guessing game to dial in it's weights. The training data is picked up, used as reference material for the game, and then discarded. It's hard to place this far from what humans do when reading, because both are using the information to mold their respective "brains" and both are doing an acquire, analyze, discard process. At no point is training data actually copied into the model itself, it's just run past the "eyes" of the model to play the training game.
          • __loam 1y ago
            This trial has nothing to with how the brain works and even if they did work the same, humans obviously have different legal rights than a computer.
      • amlib 1y ago
        I can arrange a series of bricks in many ways to try and build a wall but that doesn't mean I will automatically get a good result if my process (like a ML training algorithm) doesn't precisely arrange then in a manner that produces a rigid wall with the desired characteristics. In the same vein you can have a fancy neural network arranged by some fancy LLM training algorithm with gobs of data about a subject but current methods likely won't produce anything with the depth of "understanding" that a human can do. It's a crumbly wall that falls once you do any real inspection or put any real load into it.
  • mcny 1y ago
    > If you read a book and later understand its plot but can only explain it in your own words, did you copy it? I think that is the center of the conversation. What does it mean for a computer to "understand"? If I wrote some code that somehow transformed the text of the book and never return the verbatim text but somehow modified the output, I would likely not be spared because the ruling will likely be my transformation is "trivial". Personally, I think we have several fixes we need to make: 1. Abolish the CFAA. 2. Limit copyright to a maximum of 5 years from date of production with no extension possible for any reason. 3. Allow explicit carveout in copyright for transformational work. Explicitly allow format shifting, time shifting, yada yada. 4. Prohibit authors and publishers from including the now obviously false statements like "No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording" bla bla bla in their works. 5. I am sure I am missing some stuff here. For brand protection, we already have trademark law. Most readers here already know this but We really should severe the artificial ties we have created between patents, trademarks, and copyright. https://www.gnu.org/philosophy/not-ipr.en.html https://www.gnu.org/philosophy/not-ipr.en.html
  • waynesonfire 1y ago
    I just happen do read the Phoenix Technologies wikipedia page a few days ago. This company is known for developing BIOS software for computers. Maybe you've seen their logo when you first turn on your computer. In early computing, everything was closed sourced. Quoting the wikiepdia page, To develop a legal BIOS, Phoenix used a clean room design. Engineers read the BIOS source listings in the IBM PC Technical Reference Manual. They wrote technical specifications for the BIOS APIs for a single, separate engineer—one with experience programming the Texas Instruments TMS9900, not the Intel 8088 or 8086—who had not been exposed to IBM BIOS source code. The legal team at Phoenix deemed inappropriate to "recall source in their own words" for legal reasons. My non-legal intuition is that these companies training their models are violating copyright. But, the stakes are too high--it's too big to fail if you will. If we don't do it, then our competitors will destroy us. How do you reconcile that?
    • SoftTalker 1y ago
      Perhaps Phoenix just looked at the potential adversary (IBM) and decided to approach the project in an exceedingly cautious way, knowing that IBM could litigate it forever if there were any plausible argument that they "copied" even a line of code.
    • int_19h 1y ago
      You're right that if we want to have usable LLMs at all, there's no way around training them on copyrighted materials. So it has to be allowed, but in a way that compensates the original authors somehow. For example, every model provider has to publicly declare all works used for training, and then all inference providers offering that model have to collect a per-token tax that gets distributed to authors in proportion to their presence in the dataset (by the by, this could also be a way to fund websites like Wikipedia). But any such arrangement needs to be hammered out by the legislature. As laws are, I think it's pretty clear that infringement is happening.
  • triceratops 1y ago
    You didn't copy it because you're a human. A computer can't "read" without copying. It's how it works.
  • freejazz 1y ago
    Great question for a different litigation actually involving humans.
• nilsbunger 1y ago
  There’s something called a substantive transformation test in copyright law. When you write a summary of a book, you don’t infringe on copyright because it’s a “substantial transformation”. This goes along with the idea that you can copyright the text but not the ideas it expresses. When model training reads the text and creates weights internally, is that a substantial transformation? I think there’s a pretty strong argument that it is.
  • triceratops 1y ago
    The counterargument to that is model training is impossible without making copies. That's not true for humans.
    • realusername 1y ago
      It's also true for humans, you memorize only parts of what you read and see but you still had to view the whole thing first. The computer model is working differently of course but functionally it's the same idea.
      • __loam 1y ago
        God I hate this conversation so much. These cases have nothing to do with how the brain works.
        • realusername 1y ago
          That's why the keyword here is functionally.
    • Workaccount2 1y ago
      That's not really true. Models train (in greatly simplified way) by being shown an excerpt and being told to guess the next token from the excerpt. They push around their weights until the token they output matches the next token in the excerpt. Then the excerpt is no longer needed. You can think of it like the article is loaded, the LLM plays this token guessing game through it, then the article is discarded. On the face of it this is what happens, but it gets hairier depending on how exactly this process is done. But it is seemingly not far removed from how humans consume content (acquire, read, discard), hence the legal blur.
• meta_ai_x 1y ago
  completely different scenarios. A pirated movie is marketed/sold as a copy of something, which is not fair use. An LLM just remembers/get inspired by what it consumes
  • xyzzy_plugh 1y ago
    I don't believe that's correct. The existence of filters to block potentially copyrighted materials contained in LLM outputs proves that they don't just "get inspired." It seems like it is very much a matter of fidelity.
  • crystal_revenge 1y ago
    > An LLM just remembers/get inspired by what it consumes As mentioned in another comment, LLMs (and most popular machine learning algorithms) can be viewed, correctly, as compression algorithms which leverage lossy encoding + interpolation to force a kind of generalization. Your argument is that a video wouldn't count as pirated if the compression used for the pirated copy was lossy (or at least sufficiently lossy). The closest real world example would be the cases where someone records a the filming of a movie on their phone then uploads it. Such a copy is lossy enough that you can't produce anything really like the original, but my most definitions is still considered copyright.
    • Workaccount2 1y ago
      They are in no way compression algorithms. They can be modeled like that in the same way you can model humans as lossy compression algorithms. You would never use a human to backup your financial reports, but the human might be able to give a good overview. You would never use an LLM to backup your financial reports, but they might be able to give a good overview. AI training data is disposable. There is nothing that could be called a compression algorithm that disposes all of the data you put into it. AI uses training data as examples of what the next token in a token sequence is. The examples are disposable reference points, not the model itself. That's how you get image models that are 20GB in size despite training on 20PB of data. It's 20PB of examples used to form the shape of a 20GB model. You could show it 5GB of training data or 500EB of training data and it would still be 20GB - because it is not a compression algo, it's a 20GB shape formed by external data.
• realusername 1y ago
  It's the same as a book or a film review, you can't get the film or the book back from it but the original material is still needed to produce it. Needing the original material isn't enough for claiming copyright infringement as we have existing counter examples
  • __loam 1y ago
    Movie reviews are fair use because they don't compete with the original work.
    • realusername 1y ago
      The AI models also do not compete with the original work. Nobody is going to try to extract pages by pages a book from ChapGPT, let's be realistic. (and you can't anyways)
      • __loam 1y ago
        Half the point of this crap is labor automation. How does it not compete? If I write a bunch of books and then you use my work to make a machine that writes books in my style, you are using my labor to directly compete with me.
        • realusername 1y ago
          Is there even a documented case of it happening? I don't think the current state of LLM would be able to write 200 pages in a coherent manner anyways.
• stevenAthompson 1y ago
  The purpose of copyright is it progress the arts and sciences. Not to guarantee profit. Guaranteeing profit is just the way we encourage people to progress the arts and sciences. That is why so called derivative works are allowed (and even encouraged). If copyrighted material is ingested, modified or enhanced to add value, and then regurgitated that is legal, whereas copying it without adding value is not legal. If derivative works weren't deemed acceptable copyright would have the opposite of it's intended effect and become an impediment to progress.
  • throwaway290 1y ago
    Derivative works are not generally allowed in many jurisdictions. Try releasing a cover song without clearing it first etc. Even using a recognizable sample will bite you Derivative works are tolerated in some cases like some manga or fanfics but it is a gray area and whenever the author or publisher wants to pursue it it is their full right to do it. Many do pursue it (You can get inspired by something, and this is where some arguments can happen if you get inspired too mmmm literally, but no one will say with a straight face that inspiration is a thing that happens to software)
    • stevenAthompson 1y ago
      You are absolutely right. I should have phrased that differently. Derivative work is a legal term, but I misused it above. I should have either used another term or been clearer. If the work is "derivative" in the legal sense it is copyrighted, and you may not create derivative works without the copyright holders permission. What I should have said is that simply being inspired by a work or copying unprotectable elements (like facts or ideas) does not create a derivative work. For example, if ChatGPT were to generate Star Wars, except with Dookies instead of Wookies, that might be illegal. If it were to learn what a spaceship is from Star Wars and then create something substantially new it would not. The key is is that it must not be substantially similar to the original. You must add enough value that it becomes something new, not just rehash the original.
      • 1y ago
• amelius 1y ago
  > I'm legitimately curious what the test is here. The test is if a judge says it is fair use, nothing else. The judge will take into account the human factor in this matter, e.g. things like who did the actual work, and who just used an algorithm (which is not the hard part anymore, code can be obtained on the internet for free). And we all know that DL is nowhere without huge amounts data.
• johanyc 1y ago
  The keyword you’re looking for is “transformative”