5 ms·

If I listen to 1000 pop songs and then write my own that's different from all of them, that's not copyright infringement. If I listen to 1000 pop songs and then

by feoren 3y ago

If I listen to 1000 pop songs and then write my own that's different from all of them, that's not copyright infringement. If I listen to 1000 pop songs and then write one that's nearly exactly the same as one of them, that absolutely is copyright infringement and I would most likely lose a lawsuit. The fact that I listened to 999 other songs does not absolve me of the infringement. Most modern neural nets, and certainly LLMs, are overparameterized, which means they are completely capable of regurgitating exact training instances. They can and do repeat copyrighted training data verbatim. It's absolutely copyright infringement when that happens. I anticipate AI researchers will soon come up with a way to ensure that outputs are a sufficient distance from all training inputs -- to essentially "block" direct regurgitation as valid output, with a tunable "max similarity" set to a safe distance. At that point we can start being more confident that our AI output is genuinely transformative and safe from (valid!) lawsuit. > The bitching from writers and musicians reflects not that they're being ripped off, but that they're being out-produced. This comes across as quite heartless. It will be interesting to see whether your view on this changes when AI starts making obsolete whatever you have spent your life specializing in.

toomuchtodo 3y ago
I do not mind being made redundant by AI. I'll go become an electrician, general contractor, or something else you're not going to automate in my lifetime. The problem isn't the automation, its what happens if the usual Capital subjects capture all the gains vs them being distributed as excess across the populace. EDIT: We should welcome automation and destruction of work where a machine can do it better. We fear it because of a broken socioeconomic system. Agree we should probably put breaks on anything so disruptive unless people impacted have some safeguards, because if you don't, what good is progress for progress' sake? It then becomes no different than bullshit jobs.
- eropple 3y ago
  This is a critical distinction. Like 'toomuchtodo, I can go pick up a nailer and build a house if the computery jobs go away. But I cannot effectively participate in the governance of the society of which I am in theory an equal member if the benefits of automation continue inexorably to accrue to a very small group. (e: and his edit says it as eloquently as I can!)
  - toomuchtodo 3y ago
    Highest praise. Appreciate the kind words.
- shadowgovt 3y ago
  This. My full-time job is automating myself out of a job. Arguably, that's one of the indicators of a quality software engineer versus one who can follow directions but not decrease the problem space. The problem is not automation... Automation is great. It frees up people's time to do more activities. The problem is how we distribute the spoils of that freed time.
- feoren 3y ago
  Your suggestion to someone whose multi-decade career was ended by AI is to play career whack-a-mole, try to guess which careers won't be next to be killed off, go back to school for several years, and pray?
  - deleted 3y ago
    [deleted]
  - worik 3y ago
    Yes. Telephone operators. Typing pools. Grooms. TV repair technicians. Weavers To this list we may be adding law clerks. Drafters. Copy writers. This is an ancient process. Some societies and cultures cope and maintain a civil society, some don't So do not play "whack'a'mole", campaign for social change
  - surgical_fire 3y ago
    I guess we should get rid of bulldozers and give more people shovels. While you're at it, take computers always from people. They automate way too much stuff. Btw, I am in one of those careers that people think is at the risk of being automated away. I don't really care about it. If I become obsolete, maybe my skills were not that useful. I'll try working in something else. And if all jobs are automated away, I'll just go starve with the rest of the peasantry. The last to starve will be the first to suffocate.
m00x 3y ago
Actually, a lot of songs are incredibly similar to older songs (see ed sheeran v marvin gaye), and they're not infringing. Things will look very similar because there are only so many things that sound good to the mass appeal. > I anticipate AI researchers will soon come up with a way to ensure that outputs are a sufficient distance from all training inputs This is already a thing.
- thomastjeffery 3y ago
  If you are basing your argument on a comparison between Ed Sheeran and Marvin Gaye, then you are talking about something completely different. A better example is that Vanilla Ice had to pay royalties to Queen and David Bowie for using the bass hook from Under Pressure in Ice Ice Baby.
- feoren 3y ago
  > a lot of songs are incredibly similar to older songs (see ed sheeran v marvin gaye), and they're not infringing They're really not that similar. The Marvin Gaye family lawsuits are absolutely frivolous and a stain on his legacy. Suing over the same chord progression (ignoring tons of small variations that actually matter) would be like suing an author because they also had a character who was an angsty teenager with overbearing parents. Suing over a mostly-kinda-similar riff is like suing an author because they also used the sentence "Terrified, he ran." The songs only sound similar when you ignore all elements of the whole song and focus exactly on a few ways in which they overlap out of context. They're nothing like GitHub CoPilot literally outputting comments that include copyright notifications from their original author. It's much more like suing someone who directly samples your song as a core element of their own (the Vanilla Ice example in the other comment).
moritzwarhier 3y ago
> 1000 pop songs and then write one that's nearly exactly the same as one of them Alright, but how do you define this when talking about most of modern "pop" music (using a broad definition that also includes things like techno or rock sub-genres)? Maybe even only consider what's commercially relevant. Copyright is famously about authorship as an ulterior value, governing and including other values such as monetization. What about this aspect? Creativity is also often about subtlety, e.g. polished songwriting and sound design. Already without AI, rent-seeking is prevalent in popular music, maybe it's even an integral part? For a stupid example of reusing themes in pop music, take this especially spicy example: https://en.wikipedia.org/wiki/Kernkraft_400 https://en.wikipedia.org/wiki/Kernkraft_400 Or listen to the radio maybe. As a 90s and 2000s dance music lover, there's plenty of rehashing old content, sometimes licensed, other times it's just what music is. Songs are memes after all
octacat 3y ago
yeah, basically write an AI which would remove watermarks before training on data. Or after, if you've already trained (sarcasm). Great programmatic solution indeed.
sn0wf1re 3y ago
Pop music is perhaps the worst example you could have chosen. It is highly derivative. https://www.youtube.com/watch?v=QpB_40hYjXU https://www.youtube.com/watch?v=QpB_40hYjXU
ta2234234242 3y ago
> They can and do repeat copyrighted training data verbatim. It's absolutely copyright infringement when that happens. My brain can do this too with music. In my 20's I used to be able to hear songs crystal clear in my head as if they were coming from the stereo itself. Did my brain copy the song (yes!). Did it receive copyright approval from the artist (no!).
- 3p40ureg0 3y ago
  If you put that information into a fixed medium of expression can you be found guilty of copyright infringement? (Yes!)
Animats 3y ago
> Most modern neural nets, and certainly LLMs, are overparameterized, which means they are completely capable of regurgitating exact training instances. No they're not. GPT-3: Model size: 175 GB, training set size 300 billion tokens. GPT-4: Not disclosed. LLaMA: Model size: 65GB, training set size, 1 trillion tokens. These things condense the data. They don't just index it, like a search engine. [1] https://lambdalabs.com/blog/demystifying-gpt-3 https://lambdalabs.com/blog/demystifying-gpt-3
- WithinReason 3y ago
  Also, having the same number of parameters as the training set doesn't mean you can memorize it

by feoren 3y ago

toomuchtodo 3y ago

I do not mind being made redundant by AI. I'll go become an electrician, general contractor, or something else you're not going to automate in my lifetime. The problem isn't the automation, its what happens if the usual Capital subjects capture all the gains vs them being distributed as excess across the populace. EDIT: We should welcome automation and destruction of work where a machine can do it better. We fear it because of a broken socioeconomic system. Agree we should probably put breaks on anything so disruptive unless people impacted have some safeguards, because if you don't, what good is progress for progress' sake? It then becomes no different than bullshit jobs.

eropple 3y ago
This is a critical distinction. Like 'toomuchtodo, I can go pick up a nailer and build a house if the computery jobs go away. But I cannot effectively participate in the governance of the society of which I am in theory an equal member if the benefits of automation continue inexorably to accrue to a very small group. (e: and his edit says it as eloquently as I can!)
- toomuchtodo 3y ago
  Highest praise. Appreciate the kind words.
shadowgovt 3y ago
This. My full-time job is automating myself out of a job. Arguably, that's one of the indicators of a quality software engineer versus one who can follow directions but not decrease the problem space. The problem is not automation... Automation is great. It frees up people's time to do more activities. The problem is how we distribute the spoils of that freed time.
feoren 3y ago
Your suggestion to someone whose multi-decade career was ended by AI is to play career whack-a-mole, try to guess which careers won't be next to be killed off, go back to school for several years, and pray?
- deleted 3y ago
  [deleted]
- worik 3y ago
  Yes. Telephone operators. Typing pools. Grooms. TV repair technicians. Weavers To this list we may be adding law clerks. Drafters. Copy writers. This is an ancient process. Some societies and cultures cope and maintain a civil society, some don't So do not play "whack'a'mole", campaign for social change
- surgical_fire 3y ago
  I guess we should get rid of bulldozers and give more people shovels. While you're at it, take computers always from people. They automate way too much stuff. Btw, I am in one of those careers that people think is at the risk of being automated away. I don't really care about it. If I become obsolete, maybe my skills were not that useful. I'll try working in something else. And if all jobs are automated away, I'll just go starve with the rest of the peasantry. The last to starve will be the first to suffocate.

m00x 3y ago

Actually, a lot of songs are incredibly similar to older songs (see ed sheeran v marvin gaye), and they're not infringing. Things will look very similar because there are only so many things that sound good to the mass appeal. > I anticipate AI researchers will soon come up with a way to ensure that outputs are a sufficient distance from all training inputs This is already a thing.

thomastjeffery 3y ago
If you are basing your argument on a comparison between Ed Sheeran and Marvin Gaye, then you are talking about something completely different. A better example is that Vanilla Ice had to pay royalties to Queen and David Bowie for using the bass hook from Under Pressure in Ice Ice Baby.
feoren 3y ago
> a lot of songs are incredibly similar to older songs (see ed sheeran v marvin gaye), and they're not infringing They're really not that similar. The Marvin Gaye family lawsuits are absolutely frivolous and a stain on his legacy. Suing over the same chord progression (ignoring tons of small variations that actually matter) would be like suing an author because they also had a character who was an angsty teenager with overbearing parents. Suing over a mostly-kinda-similar riff is like suing an author because they also used the sentence "Terrified, he ran." The songs only sound similar when you ignore all elements of the whole song and focus exactly on a few ways in which they overlap out of context. They're nothing like GitHub CoPilot literally outputting comments that include copyright notifications from their original author. It's much more like suing someone who directly samples your song as a core element of their own (the Vanilla Ice example in the other comment).

moritzwarhier 3y ago

> 1000 pop songs and then write one that's nearly exactly the same as one of them Alright, but how do you define this when talking about most of modern "pop" music (using a broad definition that also includes things like techno or rock sub-genres)? Maybe even only consider what's commercially relevant. Copyright is famously about authorship as an ulterior value, governing and including other values such as monetization. What about this aspect? Creativity is also often about subtlety, e.g. polished songwriting and sound design. Already without AI, rent-seeking is prevalent in popular music, maybe it's even an integral part? For a stupid example of reusing themes in pop music, take this especially spicy example: https://en.wikipedia.org/wiki/Kernkraft_400 https://en.wikipedia.org/wiki/Kernkraft_400 Or listen to the radio maybe. As a 90s and 2000s dance music lover, there's plenty of rehashing old content, sometimes licensed, other times it's just what music is. Songs are memes after all

octacat 3y ago

yeah, basically write an AI which would remove watermarks before training on data. Or after, if you've already trained (sarcasm). Great programmatic solution indeed.

sn0wf1re 3y ago

Pop music is perhaps the worst example you could have chosen. It is highly derivative. https://www.youtube.com/watch?v=QpB_40hYjXU https://www.youtube.com/watch?v=QpB_40hYjXU

ta2234234242 3y ago

> They can and do repeat copyrighted training data verbatim. It's absolutely copyright infringement when that happens. My brain can do this too with music. In my 20's I used to be able to hear songs crystal clear in my head as if they were coming from the stereo itself. Did my brain copy the song (yes!). Did it receive copyright approval from the artist (no!).

3p40ureg0 3y ago
If you put that information into a fixed medium of expression can you be found guilty of copyright infringement? (Yes!)

Animats 3y ago

> Most modern neural nets, and certainly LLMs, are overparameterized, which means they are completely capable of regurgitating exact training instances. No they're not. GPT-3: Model size: 175 GB, training set size 300 billion tokens. GPT-4: Not disclosed. LLaMA: Model size: 65GB, training set size, 1 trillion tokens. These things condense the data. They don't just index it, like a search engine. [1] https://lambdalabs.com/blog/demystifying-gpt-3 https://lambdalabs.com/blog/demystifying-gpt-3

WithinReason 3y ago
Also, having the same number of parameters as the training set doesn't mean you can memorize it