11 ms·

I didn't realize that the llama license forbids you from using its outputs to train other models. That's essentially a dealbreaker, synthetic data is going to b

by Taek 3y ago

I didn't realize that the llama license forbids you from using its outputs to train other models. That's essentially a dealbreaker, synthetic data is going to be the most important type of training data from here on out. Any model that prohibits use of synthetic data to train new models is crippled.

SanderNL 3y ago
Good luck enforcing that, though. How would they ever know?
- denlekke 3y ago
  i wonder if they could include some marker prompt and response that wouldn't occur "naturally" from any other model or training data
  - ortusdux 3y ago
    https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
    - mcny 3y ago
      Level1Techs "link show" (because we can't call it news anymore) kind of touched this topic. I would like to read what you guys make of this: > Supreme Court rejects Genius lawsuit claiming Google stole song lyrics SCOTUS won't overturn ruling that US copyright law preempts Genius' claim. > The song lyrics website Genius' allegations that Google "stole" its work in violation of a contract will not be heard by the US Supreme Court. The top US court denied Genius' petition for certiorari in an order list issued today, leaving in place lower-court rulings that went in Google's favor. > Genius previously lost rulings in US District Court for the Eastern District of New York and the US Court of Appeals for the 2nd Circuit. In August 2020, US District Judge Margo Brodie ruled that Genius' claim is preempted by the US Copyright Act. The appeals court upheld the ruling in March 2022. > "Plaintiff's argument is, in essence, that it has created a derivative work of the original lyrics in applying its own labor and resources to transcribe the lyrics, and thus, retains some ownership over and has rights in the transcriptions distinct from the exclusive rights of the copyright owners... Plaintiff likely makes this argument without explicitly referring to the lyrics transcriptions as derivative works because the case law is clear that only the original copyright owner has exclusive rights to authorize derivative works," Brodie wrote in the August 2020 ruling. > Google search results routinely display song lyrics via the service LyricFind. Genius alleged that LyricFind copied Genius transcriptions and licensed them to Google. > Brodie found that Genius' claim must fail even if one accepts the argument that it "added a separate and distinct value to the lyrics by transcribing them such that the lyrics are essentially derivative works." Since Genius "does not allege that it received an assignment of the copyright owners' rights in the lyrics displayed on its website, Plaintiff's claim is preempted by the Copyright Act because, at its core, it is a claim that Defendants created an unauthorized reproduction of Plaintiff's derivative work, which is itself conduct that violates an exclusive right of the copyright owner under federal copyright law," Brodie wrote. https://arstechnica.com/tech-policy/2023/06/supreme-court-rejects-genius-lawsuit-claiming-google-stole-song-lyrics/ https://arstechnica.com/tech-policy/2023/06/supreme-court-re...
      - rcxdude 3y ago
        The basic idea is whether an unauthorised derivative work is itself entitled to copyright protection: could the creator of the derivative work prevent copying by the original creator (or anyone else) of the work on which it is based, even though they themselves have no permission to distribute it? (if the work is authorised, this is generally considered to be the case). It looks like from this the conclusion is 'no', at the very least in this case. I'm not sure this matches most people's moral intuitions: every now and again a big company includes some fan art in their own official release without permission (usually not as a result of a general policy, but because of someone getting lazy and the rest of the system failing to catch it), and generally speaking the reaction is negative.
        joshuaissac 3y ago
        > whether an unauthorised derivative work is itself entitled to copyright protection That is not what this court case was about. Genius had already settled the case of unauthorised transcriptions and had bought licences for its lyrics after a lawsuit 2014, so its own work was no longer unauthorised. In the case cited above, Genius was trying to enforce its claims against Google via contract law rather than copyright law. The court ruled that the alleged violations were covered by copyright law, so they could only pursued via copyright law, and that only the copyright holder (or assignee) of the lyrics that were copied could sue Google under it.
    - nsplayer 3y ago
      They could have picked up the LLM equivalent from LLM generated posts online however. How do you prove they didn't?
      - denlekke 3y ago
        as a layman, i imagine for someone at the scale required it may not be worth the risk or the added effort vs paying or using a different model but it'd be funny if we see companies creating a subsidiary that just acts as a web-passthrough to "legalize" llama2 output as training data
- criddell 3y ago
  Disgruntled current or former employee turning in their employer for the reward? That’s how Microsoft and the BSA used to bust people before the days of always online software.
danShumway 3y ago
I don't see how this would be enforceable in law without killing almost every AI company on the market today. The whole legal premise of these models is that training on copyrighted material is fair use. If it's not, then... I mean is Facebook trying to claim that including copyrighted material in a dataset isn't fair use regardless of the author's wishes? Because I have bad news for LLaMA then. "You need permission to train on this" is an interesting legal stance for any AI company to take.
- doctorpangloss 3y ago
  > The whole legal premise of these models is that training on copyrighted material is fair use. Not to diminish the conversation here, but not even a Supreme Court Justice knows what the legality is. You’d have to be a whole 9 person Supreme Court to make an accurate statement here. I don’t think anyone really knows how Congress meant today’s laws to work in this scenario.
  - mschuster91 3y ago
    > I don’t think anyone really knows how Congress meant today’s laws to work in this scenario. Congress, or more accurate, the drafters of the Constitution, intended that Congress would work to keep the Constitution updated to match the needs of modern times. Instead, Congress ossified to the point it's unable to pass basic laws because a bunch of far right morons hold the House GQP hostage and an absurd amount of leverage was passed to the executive and the Supreme Court as a result - with the active aid of both parties by the way, who didn't even think of passing actual laws to codify something as important as equitable access to elections, fair elections, or the right to have an abortion or to smoke weed when they held majorities. And on top of that your Supreme Court and many Federal court picks were hand-selected from a society that prefers a literal viewpoint of the constitution. But fear not, y'all are not alone in this kind of idiocy, just look at us Germans and how we're still running on fax machines.
    - deleted 3y ago
      [deleted]
redox99 3y ago
It's so hypocritical, it's insane. "Yes, we train our models on a good chunk of the internet without asking permission, but don't you dare train on our models' output without our permission!" And OpenAI also has a similar restriction.
- alerighi 3y ago
  In fact they can't (both Facebook and OpenAI) train their models without asking permission. Just wait for someone to start raising this concern. The EU is working on regulating these kind of aspects, for example this is not compliant at all with the GDPR (unless you train only on data that doesn't contain personal data, that is more rare than you would think).
6gvONxR4sf7o 3y ago
It's hilarious that big players in this space seem to think these are consistent views: - It's okay to train a model on arbitrary internet data without permission/license just because you can access it - It's not okay train a model on our model
- whatshisface 3y ago
  The belief that makes them consistent is that the authors of a million Reddit posts have no way to assert their rights while the big company that trained a Redditor model does.
  - LastTrain 3y ago
    Sure they do, albeit a shitty one: it's called a class-action.
- rahkiin 3y ago
  Like google is allowed to scrape the whole internet but you’re not allowed to scrape google. Rules for thee but not for me
  - kgwgk 3y ago
    What rules? Google won’t scrape your part of the internet if you don’t allow it, right?
    - makeitdouble 3y ago
      Google respects the "robot.txt" and asks you to use it to opt out of their crawling. Parent's point is if your own scaping army respects the "scaping.txt" and goes down on Google as they don't opt-out in their scraping.txt, it probably wouldn't fly.
      - kgwgk 3y ago
        I don't understand. What does "Rules for thee but not for me" mean if "google is allowed to scrape" whatever people allows Google to scrape but "you’re not allowed to scrape google" because using the same rules google.com/robots.txt says User-agent: * Disallow: /search ....
moffkalast 3y ago
I'm not sure why anyone would even do that in the first place, LLama doesn't generate synthetic data that would be even remotely good enough. Even GPT 3.5 and 4 are already very borderline for it, with lots of wrong and censored answers. And at best you make a model that's as good as LLama is, i.e. not very.
- jstarfish 3y ago
  Instruction-tuning is the obvious use case. That much has nothing to do with subjectivity, alignment or censorship, it's will-you-actually-show-this-as-JSON-if-asked.
  - moffkalast 3y ago
    That's tuning llama which is allowed from what I understand. Otherwise why release it at all, it's not very functional in its initial state anyway. What that applies to is using llama outputs to train a completely new base model which makes no practical sense. As for generating jsons, that's more of a inference runtime thing, since you need to pick the top tokens that result a valid json instead of just hoping it returns something that can be parsed. On top of extensive tuning of course.
zarzavat 3y ago
A contract ordinarily has to have consideration. Since LLaMa weights are not copyrightable by Meta and are freely available, what exactly is the consideration? The bandwidth they provide?
objektif 3y ago
I played with Llama2 for a bit and for a lot of the questions I asked I got complete made up garbage stuff. Why would you want to train on it?
lolinder 3y ago
Not that it's okay for this to be in the license, but I'm curious: what is the use case for synthetic data? Most of the discussion I've seen has been about how to avoid accidentally using LLM-generated data.
- lmeyerov 3y ago
  Tuning a tiny classifier
- int_19h 3y ago
  You can use synthetic data produced by more complex models to finetune smaller ones to be better.
Der_Einzige 3y ago
It's exactly the opposite. We have better ways to combine the knowledge of several models together than sampling them. (i.e. mixture of experts, model merges, etc) Relying on synthetic data from one LLM to train another LLM is in general a terrible idea and will lead to a race to the bottom.
tasubotadas 3y ago
Generate data using ai, save it, it cannot be copyrighted or anything, data isn't a model, use it as much as you want for training. Ezpz
concinds 3y ago
Fundamentally untrue, and disheartening that it's the top comment. You can't use a model's output to train another model, it leads to complete gibberish (termed "model collapse"). https://arxiv.org/abs/2305.17493v2 https://arxiv.org/abs/2305.17493v2 And the Llama 2 license allows users to train derivative models, which is what people really care about. https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE
- rgoldste 3y ago
  The truth is between these two. You can use a model’s output to train another model, but it has drawbacks, including model collapse.
- int_19h 3y ago
  Some of the best LLaMA finetunes today are trained on GPT-4 output. Yes, you cannot do this kind of thing indefinitely and expect endless improvements from "endless training set". But that's a very different problem.
dheera 3y ago
> forbids you from using its outputs to train other models. I don't know how one can even forbid this. As a human, I'm a walking neural net, and I train myself on everything that I see, without a choice. The only difference is I'm a carbon-based neural net.
renewiltord 3y ago
I would just do it anyway. In fact, I can release a suitably laundered version and you'd never know. If I release a few million, each with slight variation, there's no way provenance can be established. And then we're home-free.

by Taek 3y ago

SanderNL 3y ago

Good luck enforcing that, though. How would they ever know?

denlekke 3y ago
i wonder if they could include some marker prompt and response that wouldn't occur "naturally" from any other model or training data
- ortusdux 3y ago
  https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
  - mcny 3y ago
    Level1Techs "link show" (because we can't call it news anymore) kind of touched this topic. I would like to read what you guys make of this: > Supreme Court rejects Genius lawsuit claiming Google stole song lyrics SCOTUS won't overturn ruling that US copyright law preempts Genius' claim. > The song lyrics website Genius' allegations that Google "stole" its work in violation of a contract will not be heard by the US Supreme Court. The top US court denied Genius' petition for certiorari in an order list issued today, leaving in place lower-court rulings that went in Google's favor. > Genius previously lost rulings in US District Court for the Eastern District of New York and the US Court of Appeals for the 2nd Circuit. In August 2020, US District Judge Margo Brodie ruled that Genius' claim is preempted by the US Copyright Act. The appeals court upheld the ruling in March 2022. > "Plaintiff's argument is, in essence, that it has created a derivative work of the original lyrics in applying its own labor and resources to transcribe the lyrics, and thus, retains some ownership over and has rights in the transcriptions distinct from the exclusive rights of the copyright owners... Plaintiff likely makes this argument without explicitly referring to the lyrics transcriptions as derivative works because the case law is clear that only the original copyright owner has exclusive rights to authorize derivative works," Brodie wrote in the August 2020 ruling. > Google search results routinely display song lyrics via the service LyricFind. Genius alleged that LyricFind copied Genius transcriptions and licensed them to Google. > Brodie found that Genius' claim must fail even if one accepts the argument that it "added a separate and distinct value to the lyrics by transcribing them such that the lyrics are essentially derivative works." Since Genius "does not allege that it received an assignment of the copyright owners' rights in the lyrics displayed on its website, Plaintiff's claim is preempted by the Copyright Act because, at its core, it is a claim that Defendants created an unauthorized reproduction of Plaintiff's derivative work, which is itself conduct that violates an exclusive right of the copyright owner under federal copyright law," Brodie wrote. https://arstechnica.com/tech-policy/2023/06/supreme-court-rejects-genius-lawsuit-claiming-google-stole-song-lyrics/ https://arstechnica.com/tech-policy/2023/06/supreme-court-re...
    - rcxdude 3y ago
      The basic idea is whether an unauthorised derivative work is itself entitled to copyright protection: could the creator of the derivative work prevent copying by the original creator (or anyone else) of the work on which it is based, even though they themselves have no permission to distribute it? (if the work is authorised, this is generally considered to be the case). It looks like from this the conclusion is 'no', at the very least in this case. I'm not sure this matches most people's moral intuitions: every now and again a big company includes some fan art in their own official release without permission (usually not as a result of a general policy, but because of someone getting lazy and the rest of the system failing to catch it), and generally speaking the reaction is negative.
      - joshuaissac 3y ago
        > whether an unauthorised derivative work is itself entitled to copyright protection That is not what this court case was about. Genius had already settled the case of unauthorised transcriptions and had bought licences for its lyrics after a lawsuit 2014, so its own work was no longer unauthorised. In the case cited above, Genius was trying to enforce its claims against Google via contract law rather than copyright law. The court ruled that the alleged violations were covered by copyright law, so they could only pursued via copyright law, and that only the copyright holder (or assignee) of the lyrics that were copied could sue Google under it.
  - nsplayer 3y ago
    They could have picked up the LLM equivalent from LLM generated posts online however. How do you prove they didn't?
    - denlekke 3y ago
      as a layman, i imagine for someone at the scale required it may not be worth the risk or the added effort vs paying or using a different model but it'd be funny if we see companies creating a subsidiary that just acts as a web-passthrough to "legalize" llama2 output as training data
criddell 3y ago
Disgruntled current or former employee turning in their employer for the reward? That’s how Microsoft and the BSA used to bust people before the days of always online software.

danShumway 3y ago

I don't see how this would be enforceable in law without killing almost every AI company on the market today. The whole legal premise of these models is that training on copyrighted material is fair use. If it's not, then... I mean is Facebook trying to claim that including copyrighted material in a dataset isn't fair use regardless of the author's wishes? Because I have bad news for LLaMA then. "You need permission to train on this" is an interesting legal stance for any AI company to take.

doctorpangloss 3y ago
> The whole legal premise of these models is that training on copyrighted material is fair use. Not to diminish the conversation here, but not even a Supreme Court Justice knows what the legality is. You’d have to be a whole 9 person Supreme Court to make an accurate statement here. I don’t think anyone really knows how Congress meant today’s laws to work in this scenario.
- mschuster91 3y ago
  > I don’t think anyone really knows how Congress meant today’s laws to work in this scenario. Congress, or more accurate, the drafters of the Constitution, intended that Congress would work to keep the Constitution updated to match the needs of modern times. Instead, Congress ossified to the point it's unable to pass basic laws because a bunch of far right morons hold the House GQP hostage and an absurd amount of leverage was passed to the executive and the Supreme Court as a result - with the active aid of both parties by the way, who didn't even think of passing actual laws to codify something as important as equitable access to elections, fair elections, or the right to have an abortion or to smoke weed when they held majorities. And on top of that your Supreme Court and many Federal court picks were hand-selected from a society that prefers a literal viewpoint of the constitution. But fear not, y'all are not alone in this kind of idiocy, just look at us Germans and how we're still running on fax machines.
  - deleted 3y ago
    [deleted]

redox99 3y ago

It's so hypocritical, it's insane. "Yes, we train our models on a good chunk of the internet without asking permission, but don't you dare train on our models' output without our permission!" And OpenAI also has a similar restriction.

alerighi 3y ago
In fact they can't (both Facebook and OpenAI) train their models without asking permission. Just wait for someone to start raising this concern. The EU is working on regulating these kind of aspects, for example this is not compliant at all with the GDPR (unless you train only on data that doesn't contain personal data, that is more rare than you would think).

6gvONxR4sf7o 3y ago

It's hilarious that big players in this space seem to think these are consistent views: - It's okay to train a model on arbitrary internet data without permission/license just because you can access it - It's not okay train a model on our model

whatshisface 3y ago
The belief that makes them consistent is that the authors of a million Reddit posts have no way to assert their rights while the big company that trained a Redditor model does.
- LastTrain 3y ago
  Sure they do, albeit a shitty one: it's called a class-action.
rahkiin 3y ago
Like google is allowed to scrape the whole internet but you’re not allowed to scrape google. Rules for thee but not for me
- kgwgk 3y ago
  What rules? Google won’t scrape your part of the internet if you don’t allow it, right?
  - makeitdouble 3y ago
    Google respects the "robot.txt" and asks you to use it to opt out of their crawling. Parent's point is if your own scaping army respects the "scaping.txt" and goes down on Google as they don't opt-out in their scraping.txt, it probably wouldn't fly.
    - kgwgk 3y ago
      I don't understand. What does "Rules for thee but not for me" mean if "google is allowed to scrape" whatever people allows Google to scrape but "you’re not allowed to scrape google" because using the same rules google.com/robots.txt says User-agent: * Disallow: /search ....

moffkalast 3y ago

I'm not sure why anyone would even do that in the first place, LLama doesn't generate synthetic data that would be even remotely good enough. Even GPT 3.5 and 4 are already very borderline for it, with lots of wrong and censored answers. And at best you make a model that's as good as LLama is, i.e. not very.

jstarfish 3y ago
Instruction-tuning is the obvious use case. That much has nothing to do with subjectivity, alignment or censorship, it's will-you-actually-show-this-as-JSON-if-asked.
- moffkalast 3y ago
  That's tuning llama which is allowed from what I understand. Otherwise why release it at all, it's not very functional in its initial state anyway. What that applies to is using llama outputs to train a completely new base model which makes no practical sense. As for generating jsons, that's more of a inference runtime thing, since you need to pick the top tokens that result a valid json instead of just hoping it returns something that can be parsed. On top of extensive tuning of course.

zarzavat 3y ago

A contract ordinarily has to have consideration. Since LLaMa weights are not copyrightable by Meta and are freely available, what exactly is the consideration? The bandwidth they provide?

objektif 3y ago

I played with Llama2 for a bit and for a lot of the questions I asked I got complete made up garbage stuff. Why would you want to train on it?

lolinder 3y ago

Not that it's okay for this to be in the license, but I'm curious: what is the use case for synthetic data? Most of the discussion I've seen has been about how to avoid accidentally using LLM-generated data.

lmeyerov 3y ago
Tuning a tiny classifier
int_19h 3y ago
You can use synthetic data produced by more complex models to finetune smaller ones to be better.

Der_Einzige 3y ago

It's exactly the opposite. We have better ways to combine the knowledge of several models together than sampling them. (i.e. mixture of experts, model merges, etc) Relying on synthetic data from one LLM to train another LLM is in general a terrible idea and will lead to a race to the bottom.

tasubotadas 3y ago

Generate data using ai, save it, it cannot be copyrighted or anything, data isn't a model, use it as much as you want for training. Ezpz

concinds 3y ago

Fundamentally untrue, and disheartening that it's the top comment. You can't use a model's output to train another model, it leads to complete gibberish (termed "model collapse"). https://arxiv.org/abs/2305.17493v2 https://arxiv.org/abs/2305.17493v2 And the Llama 2 license allows users to train derivative models, which is what people really care about. https://github.com/facebookresearch/llama/blob/main/LICENSE https://github.com/facebookresearch/llama/blob/main/LICENSE

rgoldste 3y ago
The truth is between these two. You can use a model’s output to train another model, but it has drawbacks, including model collapse.
int_19h 3y ago
Some of the best LLaMA finetunes today are trained on GPT-4 output. Yes, you cannot do this kind of thing indefinitely and expect endless improvements from "endless training set". But that's a very different problem.

dheera 3y ago

> forbids you from using its outputs to train other models. I don't know how one can even forbid this. As a human, I'm a walking neural net, and I train myself on everything that I see, without a choice. The only difference is I'm a carbon-based neural net.

renewiltord 3y ago

I would just do it anyway. In fact, I can release a suitably laundered version and you'd never know. If I release a few million, each with slight variation, there's no way provenance can be established. And then we're home-free.