8 ms·

There is currently a project going on to create the pile v2 which has only permissively licensed data, because of all the bickering about copyright.

by artninja1988 3y ago

jeffrallen 3y ago
> because authors prefer to be paid for their labor FTFY.
- evilduck 3y ago
  I asked an AI tool to create a cheery poem about ringworms infecting kids from the 1600s and it created something that's never existed before. Which author gets paid for this labor they performed?
  - JetSpiegel 3y ago
    Is it labor if it automatic? Which worker gets paid when I turn on my oven to heat my food? I can see value, but no labor.
- zettabomb 3y ago
  This is pretty reductive - "FTFY" is rarely the witty response you think it is.
- ben_w 3y ago
  Naturally, but I wonder what writers are going to do when the AI trained purely on suitably licensed content is still good enough to make most redundant. (The authors in best seller's lists may well be immune for a bit longer than others writers, as they're necessarily the top 0.1% of writers, but not forever: nay-sayers claimed that AI could never beat humans at chess or go because the games required special human insight).
  - mejutoco 3y ago
    > The authors in best seller's lists may well be immune for a bit longer than others writers, as they're necessarily the top 0.1% of writers The top best selling. Only one of many possible reasons for that might be the quality.
    - ben_w 3y ago
      Quality is subjective, therefore I think it is reasonable to say the best are those most able to profit rather than, e.g. winners of the Nobel Prize in Literature, or the list of books people most pretend to have read.
      - mejutoco 3y ago
        I disagree. While quality is subjective popularity is different to quality. Otherwise we would only have marvel movies. Your argument assumes no marketing is manipulating the best selling lists, something known to happen in new york best selling and others. Regarding people pretending to read "fancy books" (my term) I think most people just dont read, but I find it annoying, for ex. when people see my shelves, that some people think that I buy those books to impress somebody. It is as if people that do not enjoy reading cannot conceive somebody enjoying it. I think it is slightly antiintellectual, and cynical. I have a better opinion of people in general.
        ben_w 3y ago
        > I disagree. While quality is subjective popularity is different to quality. Otherwise we would only have marvel movies. I disagree on two fronts. First, given I responded to "> because authors prefer to be paid for their labor", I think the economics are the key, rather than the artistic merits. The authors and artists suffer economically purely on the basis of the AI doing their work for less, not on the basis of actual artistic merit. Second, the subjectivity of artistic merit means different people like different things. From a market perspective, this is why horror films get made even though they disgust people like me, it's why kids films get made even though adults outnumber kids, and it's why RomComs exist despite the stereotype of men cringing at them. You are however correct that it assumes no marketing exists to manipulate the best seller lists. But the marketing is also being outsourced to AI, and I suspect there were more writers writing copy than writing novels and screenplays. Now? Now I'm not so sure, though I'd guess it's still true. I can sympathise with you about other people thinking you're just virtue signalling with your book collection. What I meant more along the lines of how War and Peace has a reputation for being a book that people like to claim to have read but actually have not, or how many loud atheists state that only atheists have read the bible and that's how they ended up being atheists, though in either case I don't know how accurate the reputations are.
- onion2k 3y ago
  If the data is available online for the pile, surely it's also publicly available to ordinary people in a way that means authors aren't getting any money.
  - sangnoir 3y ago
    What sort of defense is this? "Your honor, after someone broke in, they left the door open. Since the door was unlocked anyone could have committed the crime I'm accused of."
idle_zealot 3y ago
So a bunch of extra work to create a downgrade? I'm sure that's going to be very popular.
- arthurcolle 3y ago
  The training data distribution is the only thing that matters, not the actual content
  - observationist 3y ago
    Unless you want something like style from a range of authors, knowledge of a fictional universe or storyline, or other domain specific data or style characteristics. A blanket removal of copyrighted data would make a bot sterile, boring, unrelatable, and ignorant of culture and common memes. We have amazing AI technology. Let's lean into it and see where it goes.
    - __loam 3y ago
      By the violating the copyright hundreds of authors.
    - arthurcolle 3y ago
      I agree, hypothetically, if I were ever to have an opinion on the matter. Just playing to the audience and potential audiences if this ever gets read into evidence ;) Haha... just kidding... unless.. ?
chasd00 3y ago
if the pile contains the code to go from step 1 to step 2 and then to 3 then couldn't you just remove the parts you don't want from the raw dataset and re-run the code?

There is currently a project going on to create the pile v2 which has only permissively licensed data, because of all the bickering about copyright.

jeffrallen 3y ago

> because authors prefer to be paid for their labor FTFY.

evilduck 3y ago
I asked an AI tool to create a cheery poem about ringworms infecting kids from the 1600s and it created something that's never existed before. Which author gets paid for this labor they performed?
- JetSpiegel 3y ago
  Is it labor if it automatic? Which worker gets paid when I turn on my oven to heat my food? I can see value, but no labor.
zettabomb 3y ago
This is pretty reductive - "FTFY" is rarely the witty response you think it is.
ben_w 3y ago
Naturally, but I wonder what writers are going to do when the AI trained purely on suitably licensed content is still good enough to make most redundant. (The authors in best seller's lists may well be immune for a bit longer than others writers, as they're necessarily the top 0.1% of writers, but not forever: nay-sayers claimed that AI could never beat humans at chess or go because the games required special human insight).
- mejutoco 3y ago
  > The authors in best seller's lists may well be immune for a bit longer than others writers, as they're necessarily the top 0.1% of writers The top best selling. Only one of many possible reasons for that might be the quality.
  - ben_w 3y ago
    Quality is subjective, therefore I think it is reasonable to say the best are those most able to profit rather than, e.g. winners of the Nobel Prize in Literature, or the list of books people most pretend to have read.
    - mejutoco 3y ago
      I disagree. While quality is subjective popularity is different to quality. Otherwise we would only have marvel movies. Your argument assumes no marketing is manipulating the best selling lists, something known to happen in new york best selling and others. Regarding people pretending to read "fancy books" (my term) I think most people just dont read, but I find it annoying, for ex. when people see my shelves, that some people think that I buy those books to impress somebody. It is as if people that do not enjoy reading cannot conceive somebody enjoying it. I think it is slightly antiintellectual, and cynical. I have a better opinion of people in general.
      - ben_w 3y ago
        > I disagree. While quality is subjective popularity is different to quality. Otherwise we would only have marvel movies. I disagree on two fronts. First, given I responded to "> because authors prefer to be paid for their labor", I think the economics are the key, rather than the artistic merits. The authors and artists suffer economically purely on the basis of the AI doing their work for less, not on the basis of actual artistic merit. Second, the subjectivity of artistic merit means different people like different things. From a market perspective, this is why horror films get made even though they disgust people like me, it's why kids films get made even though adults outnumber kids, and it's why RomComs exist despite the stereotype of men cringing at them. You are however correct that it assumes no marketing exists to manipulate the best seller lists. But the marketing is also being outsourced to AI, and I suspect there were more writers writing copy than writing novels and screenplays. Now? Now I'm not so sure, though I'd guess it's still true. I can sympathise with you about other people thinking you're just virtue signalling with your book collection. What I meant more along the lines of how War and Peace has a reputation for being a book that people like to claim to have read but actually have not, or how many loud atheists state that only atheists have read the bible and that's how they ended up being atheists, though in either case I don't know how accurate the reputations are.
onion2k 3y ago
If the data is available online for the pile, surely it's also publicly available to ordinary people in a way that means authors aren't getting any money.
- sangnoir 3y ago
  What sort of defense is this? "Your honor, after someone broke in, they left the door open. Since the door was unlocked anyone could have committed the crime I'm accused of."

idle_zealot 3y ago

So a bunch of extra work to create a downgrade? I'm sure that's going to be very popular.

arthurcolle 3y ago
The training data distribution is the only thing that matters, not the actual content
- observationist 3y ago
  Unless you want something like style from a range of authors, knowledge of a fictional universe or storyline, or other domain specific data or style characteristics. A blanket removal of copyrighted data would make a bot sterile, boring, unrelatable, and ignorant of culture and common memes. We have amazing AI technology. Let's lean into it and see where it goes.
  - __loam 3y ago
    By the violating the copyright hundreds of authors.
  - arthurcolle 3y ago
    I agree, hypothetically, if I were ever to have an opinion on the matter. Just playing to the audience and potential audiences if this ever gets read into evidence ;) Haha... just kidding... unless.. ?

chasd00 3y ago

if the pile contains the code to go from step 1 to step 2 and then to 3 then couldn't you just remove the parts you don't want from the raw dataset and re-run the code?