4 ms·

The real tragedy is that people had to work so hard to figure out exactly what was in the dataset. I just never got around to releasing the metadata, which is h

The real tragedy is that people had to work so hard to figure out exactly what was in the dataset. I just never got around to releasing the metadata, which is here: https://battle.shawwn.com/books3-metadata.jsonl https://battle.shawwn.com/books3-metadata.jsonl Dozens of people have told me I belong in prison for making books3. And as far as I can tell, it’s had zero economic impact on authors. If I do end up in prison, I think I’ll join the ranks of "people we look back on throughout history with morbid curiosity". There are lots of examples of people being imprisoned for things we now consider harmless. The buggery act was used to purge the Knights Templar. See also https://annas-archive.org/llm https://annas-archive.org/llm for acquiring ~20M books. But at this point you’d better be willing to risk being despised at best. At this point I don’t know why I even did anything in AI. llama-dl was DMCA’ed, books3 is prison fodder, and the big players do as they please with very little risk. Even if the lawsuits are successful, they’ll just pay out damages from VC money. I love ML for the sake of ML. I wanted to prove that individual researchers can contribute in a way that matters. Hopefully they’ll continue to be able to. By the way, the reporter here, Alex Reisner, is a wonderful fellow, and a fellow hacker. He has a bunch of open source projects.

pavlov 3y ago
> “as far as I can tell, it’s had zero economic impact on authors” Why are you the right person in the world to make these decisions on the behalf of thousands of living authors? I can understand distributing old works and abandonware. Copyright protection is far too long. But I don’t understand taking the recent creative work of somebody who’s alive and trying to make a living, and packaging it so that trillion-dollar corporations have a plausible excuse to use the work without permission.
- anothernewdude 3y ago
  Who are publishers to do so? Because they're the ones that have decided for everyone right now. There will be zero economic impact on authors because they're already being screwed by publishers.
  - pavlov 3y ago
    That’s pointless whataboutism. The authors can decide for themselves if they want to be in a contract with a publisher.
    - anothernewdude 3y ago
      Sure they can. Ask any academic how they feel about publishers.
- jrflowers 3y ago
  > Why are you the right person in the world to make these decisions on the behalf of thousands of living authors? This is a good argument. Why would anyone focus on the fact that no harm has been caused when we could be talking about theoretical harms that we can’t describe or quantify??
  - pavlov 3y ago
    Because there is such a thing as moral rights? Can I start writing books in your name and selling them on Kindle using your reputation with made-up quotes from your professional associates? Why not? How can you quantify the harm?
    - jrflowers 3y ago
      > Because there is such a thing as moral rights? The inalienable right to protect us from a computer using statistical math to output the word “embiggens” when given a Simpsons prompt is as obvious as free speech and liberty. Edit >Can I start writing books in your name and selling them on Kindle using your reputation with made-up quotes from your professional associates? Why not? How can you quantify the harm? You make a very compelling point here. The books3 torrent has made so much money by doing what you’ve described. Quantifying the harm is easy, simply take all of the money from books3 (the file archive)’s ill-gotten-gains checking account and give it to its victims.
      - pavlov 3y ago
        Excellent comparison because the implementation details of free speech are far from obvious and have been constantly tuned and tweaked over the centuries. (“Liberty” isn’t any specific right.) It’s easy to pick an example that you consider benevolent and harmless. But have you considered that authors simply may not want their work included in a Simpsons meme generator? Samuel Beckett dictated the set design for his plays to every last detail. Because of long copyright, you still can’t perform your own version of “En attendant Godot” today. I don’t think that’s necessarily a good thing. But I have to respect Beckett’s obsessive creative vision and that the law doesn’t let me (or Disney) make our own versions of Godot yet. Re: your edit — You wouldn’t have written the books I’m selling on Kindle in your name. So by the same logic that there’s no harm to authors when Meta and Google and OpenAI sell products derived from pirated books, there’s no harm to you when I use your name for my own ventures. Sillysaurusx created a database whose only appeal is that it contains works by famous authors. This is not fundamentally different from me writing a book in your name.
- sillysaurusx 3y ago
  Who are authors to decide that their work should be forbidden from being fed into ML models? Everyone does it. And until the laws change, people will continue to. You could make the same argument about LAION (and people do). The difference is that people have to scrape LAION themselves. Google itself would be copyright infringement by that standard. They spider massive amounts of copyrighted content, and serve ads while doing so. My goal was to make it possible for people like you and me to train a ChatGPT competitor.
  - KirillPanov 3y ago
    > Who are authors to decide that their work should be forbidden from being fed into ML models? That's how copyright works. To create a derivative work, you need a license. > They spider massive amounts of copyrighted content ...while obeying `robots.txt`, which is the machine-readable version of "all rights reserved". They also accept DMCA notices.
    - sillysaurusx 3y ago
      https://spawning.ai https://spawning.ai seems to have the right idea. They propose creating an ai.txt standard. I’m also working with them to make an opt out for books3, for whatever that’s worth. Which given the current climate of “sue or jail anyone who disagrees” might only stoke the flames.
    - mellosouls 3y ago
      That's how copyright works. To create a derivative work, you need a license. It's no more derivative than the works the authors produce, publicize and sell on their open source software platforms. Are they also going to be charged for every book released that has in some way contributed to their cultural hinterland? We're talking about building a cultural context en-masse - not taking individual books to enjoy for free and plagiarise. That single vs mass use of the books makes a significant ethical difference.
    - deleted 3y ago
      [deleted]
- hnfong 3y ago
  Assuming you believe technological advancements are a good thing, there's a balance to be struck between depriving authors of revenue they wouldn't have had in the first place (when you write a book in 2010, do you actually do so with the intent that you'd get paid by companies developing an AI with that data?), and stifling technological progress because there's new money to be made and we haven't figured how to divide the pie. (Yes, I believe they should be paid, but I don't think authors should morally be allowed to hold their works for ransom in the 'a million dollars per book, take it or leave it' sense. There should be some more reasonable thing here.) We're all members of society, it's not like you want to live in an authoritarian environment where only Big Brother has the rights to make that decision. Mind you, according to your logic, even the authors of the books themselves would not have the right to make a general decision, only that of their own works. But they wouldn't have any insight at all to how allowing the data be trained on models will advance our understanding of AI tech.
- hgomersall 3y ago
  Copyright is a deal with society. One might just as well ask why an author has any say over transactions between two independent third parties. The answer is because it has been deemed useful for the encouragement of creative works to provide for that say. It's not immutable and the rules can easily be changed if society thinks they should be.
specproc 3y ago
It must be stressful being told you belong in prison, I hope you're doing OK. FWIW, I think you -- and anyone else who collects, organises and makes this sort of stuff available -- are doing God's work. Archivists are heroes. It's the people who try to ruin the lives of those providing public goods that belong in prison. sillysaurusx, I salute you.
- sillysaurusx 3y ago
  Thanks. It is really stressful. I can’t even mention our newborn at this point without being told I’m a horrible father and that I only talk about her to manipulate public opinion. Meanwhile we just pulled up to the ER at 1am to find out why she’s not eating enough, and we have an echocardiogram in 7 hours to try to diagnose why her heart isn’t pumping enough. The Silverman folks served me papers a few days ago. https://x.com/theshawwn/status/1704559992135717238?s=61&t=jQbmCk1JqL7depzFWJNuPA https://x.com/theshawwn/status/1704559992135717238?s=61&t=jQ... I’ve made negative dollars from books3. Thus far it’s been an impressive exercise in self harm. At this point I’m just living each day to the fullest. Whatever happens will happen. I’ve been making a thread of firsts to capture the magic moments. https://x.com/theshawwn/status/1706375699756384318?s=61&t=jQbmCk1JqL7depzFWJNuPA https://x.com/theshawwn/status/1706375699756384318?s=61&t=jQ... I take solace in that people like you can benefit from my work. It was always intended for hackers.
  - all2 3y ago
    I'll be praying for you, man. If there's anything else I or anyone can do, please let us know.
  - vasco 3y ago
    Those papers are only so you don't delete anything. They are asking you for a soft "legal hold" in case they'd subpoena you later. You can let them know everything was lost in a backup mishap last year and have nothing to hold for them and they can stick it where it doesn't shine.
    - sillysaurusx 3y ago
baq 3y ago
Nobody cares until there’s someone’s making a billion dollars out of it. Truth is your only mistake was that you made yourself traceable; someone would’ve done it anyway. Should’ve put the dataset on a magnet link and host via vpn.
srhtftw 3y ago
> I love ML for the sake of ML. I wanted to prove that individual researchers can contribute in a way that matters. Sounds like someone so focused on "if they could" but not giving much thought about "if they should". I don't want you or any responsible ML researcher to go to prison. Responsible researchers make mistakes, but responsible researchers also consider the consequences of their actions. How much consideration was given to those? And given everything you know now, where would you draw the line? At what point do the consequences of this sort of research become indistinguishable from theft? Suppose the research enabled the piracy of a high fidelity copy of a single author's work without their permission? Would that be wrong? What about ~2e5 low fidelity copies of who knows how many man years of labor?
- sillysaurusx 3y ago
  I’m not sure. Someone pointed out that I should’ve put a non commercial license on books3. But books3 isn’t my data to license. As for actual impact, books3 made its way into BloombergGPT and LLaMA. I’m just happy it made LLaMA a little better. Would I do it again? Well, given that there’s little benefit other than warm feelings about LLaMA and pride that hackers can make an open source ChatGPT competitor with this, that’s a tough question. As for the line, I invite you to download books3 and try reading one of the books. The experience is positively awful. Notepad isn’t a good book reader; it’s missing images, which renders lots of the technical books less useful; and the markdown format isn’t even modern markdown syntax. Code snippets are indented with 4 spaces instead of surrounded with triple backticks. Meanwhile anyone seriously interested in reading a book will go straight to libgen and get the real book themselves. So I’m skeptical of claims of harm. As for whether it’s wrong, let history be my judge.
  - srhtftw 3y ago
    > As for the line, I invite you to download books3 and try reading one of the books. I'm not going to and I suspect anyone who values their time won't either, especially when my public library meets my needs. But the line I'm asking you to look for isn't the one around a single book - it's the aggregate value previously locked in hundreds of thousands of them now released by the power of research like yours. Even if you don't agree, I hope you appreciate that authors whose works are protected by copyright will feel entitled to a share of that value. Most of them didn't write their works to further ML research. > Would I do it again? ... that’s a tough question. I am reminded of when I was young how accessing an unprotected computer wasn't in itself a crime. People like me hacked into systems mainly out of curiosity. Then the media started making movies about teenagers starting nuclear wars with dial-up modems. Then journalists jumped on the hype train to sell their books and convinced people hackers needed to be punished. Then some went to jail and we got things like the CFAA. In a few short years what might have been wrong but wasn't explicitly illegal became so. You are clearly a smart and intellectually honest person but society often treats such people poorly - especially when confronted by scary new technology - and the tone of your answers are giving me serious Aaron Swartz vibes. There's a lot of money at stake here.
ajl666 3y ago
[dead]
clouddrover 3y ago
> And as far as I can tell, it’s had zero economic impact on authors. That doesn't matter. That's not your call to make without the permission of the copyright holders.

pavlov 3y ago

> “as far as I can tell, it’s had zero economic impact on authors” Why are you the right person in the world to make these decisions on the behalf of thousands of living authors? I can understand distributing old works and abandonware. Copyright protection is far too long. But I don’t understand taking the recent creative work of somebody who’s alive and trying to make a living, and packaging it so that trillion-dollar corporations have a plausible excuse to use the work without permission.

anothernewdude 3y ago
Who are publishers to do so? Because they're the ones that have decided for everyone right now. There will be zero economic impact on authors because they're already being screwed by publishers.
- pavlov 3y ago
  That’s pointless whataboutism. The authors can decide for themselves if they want to be in a contract with a publisher.
  - anothernewdude 3y ago
    Sure they can. Ask any academic how they feel about publishers.
jrflowers 3y ago
> Why are you the right person in the world to make these decisions on the behalf of thousands of living authors? This is a good argument. Why would anyone focus on the fact that no harm has been caused when we could be talking about theoretical harms that we can’t describe or quantify??
- pavlov 3y ago
  Because there is such a thing as moral rights? Can I start writing books in your name and selling them on Kindle using your reputation with made-up quotes from your professional associates? Why not? How can you quantify the harm?
  - jrflowers 3y ago
    > Because there is such a thing as moral rights? The inalienable right to protect us from a computer using statistical math to output the word “embiggens” when given a Simpsons prompt is as obvious as free speech and liberty. Edit >Can I start writing books in your name and selling them on Kindle using your reputation with made-up quotes from your professional associates? Why not? How can you quantify the harm? You make a very compelling point here. The books3 torrent has made so much money by doing what you’ve described. Quantifying the harm is easy, simply take all of the money from books3 (the file archive)’s ill-gotten-gains checking account and give it to its victims.
    - pavlov 3y ago
      Excellent comparison because the implementation details of free speech are far from obvious and have been constantly tuned and tweaked over the centuries. (“Liberty” isn’t any specific right.) It’s easy to pick an example that you consider benevolent and harmless. But have you considered that authors simply may not want their work included in a Simpsons meme generator? Samuel Beckett dictated the set design for his plays to every last detail. Because of long copyright, you still can’t perform your own version of “En attendant Godot” today. I don’t think that’s necessarily a good thing. But I have to respect Beckett’s obsessive creative vision and that the law doesn’t let me (or Disney) make our own versions of Godot yet. Re: your edit — You wouldn’t have written the books I’m selling on Kindle in your name. So by the same logic that there’s no harm to authors when Meta and Google and OpenAI sell products derived from pirated books, there’s no harm to you when I use your name for my own ventures. Sillysaurusx created a database whose only appeal is that it contains works by famous authors. This is not fundamentally different from me writing a book in your name.
sillysaurusx 3y ago
Who are authors to decide that their work should be forbidden from being fed into ML models? Everyone does it. And until the laws change, people will continue to. You could make the same argument about LAION (and people do). The difference is that people have to scrape LAION themselves. Google itself would be copyright infringement by that standard. They spider massive amounts of copyrighted content, and serve ads while doing so. My goal was to make it possible for people like you and me to train a ChatGPT competitor.
- KirillPanov 3y ago
  > Who are authors to decide that their work should be forbidden from being fed into ML models? That's how copyright works. To create a derivative work, you need a license. > They spider massive amounts of copyrighted content ...while obeying `robots.txt`, which is the machine-readable version of "all rights reserved". They also accept DMCA notices.
  - sillysaurusx 3y ago
    https://spawning.ai https://spawning.ai seems to have the right idea. They propose creating an ai.txt standard. I’m also working with them to make an opt out for books3, for whatever that’s worth. Which given the current climate of “sue or jail anyone who disagrees” might only stoke the flames.
  - mellosouls 3y ago
    That's how copyright works. To create a derivative work, you need a license. It's no more derivative than the works the authors produce, publicize and sell on their open source software platforms. Are they also going to be charged for every book released that has in some way contributed to their cultural hinterland? We're talking about building a cultural context en-masse - not taking individual books to enjoy for free and plagiarise. That single vs mass use of the books makes a significant ethical difference.
  - deleted 3y ago
    [deleted]
hnfong 3y ago
Assuming you believe technological advancements are a good thing, there's a balance to be struck between depriving authors of revenue they wouldn't have had in the first place (when you write a book in 2010, do you actually do so with the intent that you'd get paid by companies developing an AI with that data?), and stifling technological progress because there's new money to be made and we haven't figured how to divide the pie. (Yes, I believe they should be paid, but I don't think authors should morally be allowed to hold their works for ransom in the 'a million dollars per book, take it or leave it' sense. There should be some more reasonable thing here.) We're all members of society, it's not like you want to live in an authoritarian environment where only Big Brother has the rights to make that decision. Mind you, according to your logic, even the authors of the books themselves would not have the right to make a general decision, only that of their own works. But they wouldn't have any insight at all to how allowing the data be trained on models will advance our understanding of AI tech.
hgomersall 3y ago
Copyright is a deal with society. One might just as well ask why an author has any say over transactions between two independent third parties. The answer is because it has been deemed useful for the encouragement of creative works to provide for that say. It's not immutable and the rules can easily be changed if society thinks they should be.

specproc 3y ago

It must be stressful being told you belong in prison, I hope you're doing OK. FWIW, I think you -- and anyone else who collects, organises and makes this sort of stuff available -- are doing God's work. Archivists are heroes. It's the people who try to ruin the lives of those providing public goods that belong in prison. sillysaurusx, I salute you.

sillysaurusx 3y ago
Thanks. It is really stressful. I can’t even mention our newborn at this point without being told I’m a horrible father and that I only talk about her to manipulate public opinion. Meanwhile we just pulled up to the ER at 1am to find out why she’s not eating enough, and we have an echocardiogram in 7 hours to try to diagnose why her heart isn’t pumping enough. The Silverman folks served me papers a few days ago. https://x.com/theshawwn/status/1704559992135717238?s=61&t=jQbmCk1JqL7depzFWJNuPA https://x.com/theshawwn/status/1704559992135717238?s=61&t=jQ... I’ve made negative dollars from books3. Thus far it’s been an impressive exercise in self harm. At this point I’m just living each day to the fullest. Whatever happens will happen. I’ve been making a thread of firsts to capture the magic moments. https://x.com/theshawwn/status/1706375699756384318?s=61&t=jQbmCk1JqL7depzFWJNuPA https://x.com/theshawwn/status/1706375699756384318?s=61&t=jQ... I take solace in that people like you can benefit from my work. It was always intended for hackers.
- all2 3y ago
  I'll be praying for you, man. If there's anything else I or anyone can do, please let us know.
- vasco 3y ago
  Those papers are only so you don't delete anything. They are asking you for a soft "legal hold" in case they'd subpoena you later. You can let them know everything was lost in a backup mishap last year and have nothing to hold for them and they can stick it where it doesn't shine.
  - sillysaurusx 3y ago

baq 3y ago

Nobody cares until there’s someone’s making a billion dollars out of it. Truth is your only mistake was that you made yourself traceable; someone would’ve done it anyway. Should’ve put the dataset on a magnet link and host via vpn.

srhtftw 3y ago

> I love ML for the sake of ML. I wanted to prove that individual researchers can contribute in a way that matters. Sounds like someone so focused on "if they could" but not giving much thought about "if they should". I don't want you or any responsible ML researcher to go to prison. Responsible researchers make mistakes, but responsible researchers also consider the consequences of their actions. How much consideration was given to those? And given everything you know now, where would you draw the line? At what point do the consequences of this sort of research become indistinguishable from theft? Suppose the research enabled the piracy of a high fidelity copy of a single author's work without their permission? Would that be wrong? What about ~2e5 low fidelity copies of who knows how many man years of labor?

sillysaurusx 3y ago
I’m not sure. Someone pointed out that I should’ve put a non commercial license on books3. But books3 isn’t my data to license. As for actual impact, books3 made its way into BloombergGPT and LLaMA. I’m just happy it made LLaMA a little better. Would I do it again? Well, given that there’s little benefit other than warm feelings about LLaMA and pride that hackers can make an open source ChatGPT competitor with this, that’s a tough question. As for the line, I invite you to download books3 and try reading one of the books. The experience is positively awful. Notepad isn’t a good book reader; it’s missing images, which renders lots of the technical books less useful; and the markdown format isn’t even modern markdown syntax. Code snippets are indented with 4 spaces instead of surrounded with triple backticks. Meanwhile anyone seriously interested in reading a book will go straight to libgen and get the real book themselves. So I’m skeptical of claims of harm. As for whether it’s wrong, let history be my judge.
- srhtftw 3y ago
  > As for the line, I invite you to download books3 and try reading one of the books. I'm not going to and I suspect anyone who values their time won't either, especially when my public library meets my needs. But the line I'm asking you to look for isn't the one around a single book - it's the aggregate value previously locked in hundreds of thousands of them now released by the power of research like yours. Even if you don't agree, I hope you appreciate that authors whose works are protected by copyright will feel entitled to a share of that value. Most of them didn't write their works to further ML research. > Would I do it again? ... that’s a tough question. I am reminded of when I was young how accessing an unprotected computer wasn't in itself a crime. People like me hacked into systems mainly out of curiosity. Then the media started making movies about teenagers starting nuclear wars with dial-up modems. Then journalists jumped on the hype train to sell their books and convinced people hackers needed to be punished. Then some went to jail and we got things like the CFAA. In a few short years what might have been wrong but wasn't explicitly illegal became so. You are clearly a smart and intellectually honest person but society often treats such people poorly - especially when confronted by scary new technology - and the tone of your answers are giving me serious Aaron Swartz vibes. There's a lot of money at stake here.

ajl666 3y ago

[dead]

clouddrover 3y ago

> And as far as I can tell, it’s had zero economic impact on authors. That doesn't matter. That's not your call to make without the permission of the copyright holders.