3 ms·

It's quite different, is it not? I don't get the analogy. These models are scanning, storing, and ingesting more material than any one human could. Not only is

by bmitc 3y ago

It's quite different, is it not? I don't get the analogy. These models are scanning, storing, and ingesting more material than any one human could. Not only is the method completely different, the end goal and applications are as well. The analogy basically isn't one, at all. I'm pretty upset at companies using our personal data to make gobs of money off of. I'm also upset that they're now using our knowledge work to make even more money off of. We don't exist as computational nodes for them, a free resource to exhaust. It is a completely one way street with no consent. So I am in favor of all of these companies getting a reality check.

machdiamonds 3y ago
Consider a mega consulting firm with millions of von Neumann-like analysts. Together, they've processed the same vast data that LLMs have, but individually, none could. It's not an LLM, but its purpose is like ChatGPT: assisting clients with their tasks. If LLMs concern you due to their data processing, would a firm like this do the same?
- bmitc 3y ago
  Bringing up poor fitting analogies won't change my opinion.
  - sebzim4500 3y ago
    I normally consider these discussion to be more about the people reading the comments than the people writing them. You've clearly made up your mind, but others presumably haven't so I think it's good he makes these arguments, even if it looks like tilting at windmills to you.
    - bmitc 3y ago
      That's a good point, but I guess I was implicitly getting at that continually permutating poor analogies and hypotheticals isn't an interesting discussion. The fact of the matter here is that parties, such as OpenAI, are benefiting from others' knowledge work, protected or not, in a completely one-sided way and all for free. And I don't feel sorry for companies that need to build Trojan horse products, such as OpenAI and Google, in order to survive off of other people's data that they never compensate for.
      - paulmd 3y ago
        Why should you get to profit in the workplace from the mental models you built in college from a commercially sourced textbook? Doesn’t Pearson actually own the knowledge you’re using on the job at $BIGCORP? Why do you get to profit off diffusion based on their work? Someone else made the analogy - if you read a NYT article and then go do a stock trade based on what you read, didn’t you do that based on value generated by them, and why wouldn’t they own that too? Like if you don’t want to talk analogies then talk principles, and humans are diffusion machines. When you write a term paper from sources you are simply diffusing those words into a new arrangement, but it’s still fundamentally someone else’s work. Why do you get to profit off the model that results from someone else’s work? Copyright has mutated into this bizarre chimaera where people (like NYT) are essentially claiming ownership of ideas (and derivative works fall into a similar space) and that’s inherently in conflict with a system that is supposed to promote the creation of works. But it has turned into this bizarre shibboleth that if you came up with an idea it’s yours for life+75 years, completely yours and nobody else can work off it or remix it without crediting you. And that’s an unusual state, humanity hasn’t existed like this forever, the Berne convention is only 50 years old and already falling apart from unintended consequences. Anyway there’s no proof that copyright benefits the small guy more than corporations. Disney squashing someone for writing a Star Wars fan fiction happens a lot more than Disney ripping off someone’s fanfic character for their series. Like patents there’s this mythos of it benefiting the small guy and that’s absolutely not how it works in the real world. Also, NYT is a particularly egregious plaintiff here because they’re essentially just factual reporting of occurrences, which (like a phone book) are not really copyrightable in itself. You can copy a phone book without infringing copyright and you can train an AI model on a phonebook, and training an AI model on NYT in particular is basically doing that but for historical facts and occurrences. The fact that this costs money for NYT to generate is irrelevant, this is the “sweat of the brow” doctrine and was already swept aside by the phone book case. Just because you spent time/money making it doesn’t mean it’s copyrightable. A large amount of NYT comment is factual observation and tabulation and probably is not copyrightable in the first place. But separating that out is of course going to be challenging for NYT’s lawyers!
joncrocks 3y ago
The question I think is what is the scope of copyright? I think historically it's about copying wholesale and redistributing for profit. That doesn't seem to be what's happening here. If I read a publicly available article, and I create a summary, is that covered by copyright? If I get an AI to do that, is that somehow a 'special' type of summary that is covered? Can a provider of content somehow say "you may not use this for summarization"? Or apply other terms to my consumption if they are making it publicly available? I think the comparison is that if you ask a person something like: "Describe a power station?" They will likely have never been in a power station. They will be leaning heavily pieces of content they have consumed over the years, that presumably were copyrighted. If you create an article that describes a power station, have you breached copyright and should be sued by all the people over the years that have produced content about power stations? Is it suddenly different when a computer does it?
- bmitc 3y ago
  > Is it suddenly different when a computer does it? The answer is yes. Everyone here knows where this is heading, and yet people will sit here and defend these companies as if they're on some righteous path. Humans are already becoming disposable statistics and the engines of compute, all for free and all for the benefit of corporations who didn't pay for any of it and don't even contribute back taxes.
sebzim4500 3y ago
>It's quite different, is it not? I don't get the analogy. These models are scanning, storing, and ingesting more material than any one human could. Not only is the method completely different, the end goal and applications are as well. The analogy basically isn't one, at all. So you'd be fine with it if the model only ingested a humanly plausible amount of data? I suspect that would only make their legal issues worse, since the LLM would be much more likely to repeat tokens from the training set verbatim.
myaccountonhn 3y ago
Thank you for saying this. I can’t believe the amount of people calling for the end of copyright or saying that this is “holding back progress”. Why do we want big corporations to suck in all our work, put us out of work and not pay us anything? There’s nothing artificial about AIs like chatgpt, it’s all regurgitation of human knowledge.

by bmitc 3y ago

machdiamonds 3y ago

Consider a mega consulting firm with millions of von Neumann-like analysts. Together, they've processed the same vast data that LLMs have, but individually, none could. It's not an LLM, but its purpose is like ChatGPT: assisting clients with their tasks. If LLMs concern you due to their data processing, would a firm like this do the same?

bmitc 3y ago
Bringing up poor fitting analogies won't change my opinion.
- sebzim4500 3y ago
  I normally consider these discussion to be more about the people reading the comments than the people writing them. You've clearly made up your mind, but others presumably haven't so I think it's good he makes these arguments, even if it looks like tilting at windmills to you.
  - bmitc 3y ago
    That's a good point, but I guess I was implicitly getting at that continually permutating poor analogies and hypotheticals isn't an interesting discussion. The fact of the matter here is that parties, such as OpenAI, are benefiting from others' knowledge work, protected or not, in a completely one-sided way and all for free. And I don't feel sorry for companies that need to build Trojan horse products, such as OpenAI and Google, in order to survive off of other people's data that they never compensate for.
    - paulmd 3y ago
      Why should you get to profit in the workplace from the mental models you built in college from a commercially sourced textbook? Doesn’t Pearson actually own the knowledge you’re using on the job at $BIGCORP? Why do you get to profit off diffusion based on their work? Someone else made the analogy - if you read a NYT article and then go do a stock trade based on what you read, didn’t you do that based on value generated by them, and why wouldn’t they own that too? Like if you don’t want to talk analogies then talk principles, and humans are diffusion machines. When you write a term paper from sources you are simply diffusing those words into a new arrangement, but it’s still fundamentally someone else’s work. Why do you get to profit off the model that results from someone else’s work? Copyright has mutated into this bizarre chimaera where people (like NYT) are essentially claiming ownership of ideas (and derivative works fall into a similar space) and that’s inherently in conflict with a system that is supposed to promote the creation of works. But it has turned into this bizarre shibboleth that if you came up with an idea it’s yours for life+75 years, completely yours and nobody else can work off it or remix it without crediting you. And that’s an unusual state, humanity hasn’t existed like this forever, the Berne convention is only 50 years old and already falling apart from unintended consequences. Anyway there’s no proof that copyright benefits the small guy more than corporations. Disney squashing someone for writing a Star Wars fan fiction happens a lot more than Disney ripping off someone’s fanfic character for their series. Like patents there’s this mythos of it benefiting the small guy and that’s absolutely not how it works in the real world. Also, NYT is a particularly egregious plaintiff here because they’re essentially just factual reporting of occurrences, which (like a phone book) are not really copyrightable in itself. You can copy a phone book without infringing copyright and you can train an AI model on a phonebook, and training an AI model on NYT in particular is basically doing that but for historical facts and occurrences. The fact that this costs money for NYT to generate is irrelevant, this is the “sweat of the brow” doctrine and was already swept aside by the phone book case. Just because you spent time/money making it doesn’t mean it’s copyrightable. A large amount of NYT comment is factual observation and tabulation and probably is not copyrightable in the first place. But separating that out is of course going to be challenging for NYT’s lawyers!

joncrocks 3y ago

The question I think is what is the scope of copyright? I think historically it's about copying wholesale and redistributing for profit. That doesn't seem to be what's happening here. If I read a publicly available article, and I create a summary, is that covered by copyright? If I get an AI to do that, is that somehow a 'special' type of summary that is covered? Can a provider of content somehow say "you may not use this for summarization"? Or apply other terms to my consumption if they are making it publicly available? I think the comparison is that if you ask a person something like: "Describe a power station?" They will likely have never been in a power station. They will be leaning heavily pieces of content they have consumed over the years, that presumably were copyrighted. If you create an article that describes a power station, have you breached copyright and should be sued by all the people over the years that have produced content about power stations? Is it suddenly different when a computer does it?

bmitc 3y ago
> Is it suddenly different when a computer does it? The answer is yes. Everyone here knows where this is heading, and yet people will sit here and defend these companies as if they're on some righteous path. Humans are already becoming disposable statistics and the engines of compute, all for free and all for the benefit of corporations who didn't pay for any of it and don't even contribute back taxes.

sebzim4500 3y ago

>It's quite different, is it not? I don't get the analogy. These models are scanning, storing, and ingesting more material than any one human could. Not only is the method completely different, the end goal and applications are as well. The analogy basically isn't one, at all. So you'd be fine with it if the model only ingested a humanly plausible amount of data? I suspect that would only make their legal issues worse, since the LLM would be much more likely to repeat tokens from the training set verbatim.

myaccountonhn 3y ago

Thank you for saying this. I can’t believe the amount of people calling for the end of copyright or saying that this is “holding back progress”. Why do we want big corporations to suck in all our work, put us out of work and not pay us anything? There’s nothing artificial about AIs like chatgpt, it’s all regurgitation of human knowledge.