4 ms·
> And for us in the AI community, it was something like the hand of God reaching down to bless us with a guardian angel. I’m disagreeing as a member of the com
by 6gvONxR4sf7o 6y ago
> And for us in the AI community, it was something like the hand of God reaching down to bless us with a guardian angel.
I’m disagreeing as a member of the community. I think back to the original books corpus. It wasn’t Stephen King, but it was books from authors who gave away their content more permissively than King. It sucked how unreproducible it was, but that lack of reproducibility came from compliance with the authors wishes. If you compare it with the recent books3, the real difference is that the creator just ignored the authors.
> Ultimately, their actions are legal.
I worked on a project recently that dealt with the question of whether willingness to do a takedown was sufficient, and our legal team warned us that it isn’t. We threw away lots of work, which was a hard choice I’m proud of. The tl;dr was analogous to this: if your site just hosts dvd rips of all of the big studios’ movies, then Disney comes along and sends you a DMCA request which you comply with, you aren’t protected to hold everybody else’s movies just because they haven’t sent a DMCA request yet. That only protects you when it’s mostly stuff that uploader actually has the authority to upload in the first place. (It was much more legalese, of course, and I don’t understand it perfectly, but the gist is that DMCA compliance is necessary but not sufficient.) i.e. the Pirate Bay or old
school Napster can’t just start honoring DMCA requests and be in the clear.
Putting the onus on creators to request a takedown is telling them to play whack-a-mole. As someone with a pipe dream of leaving ML and becoming a novelist, our community’s popular approach to IP embarrasses me. “Reproducibility is hard because the owners of the data don’t want us to pass it around” doesn’t mean to find a way to ignore the owners. Disregarding data owners’ wishes in return for better models is how the world has gotten into this privacy hell.
So many of us act as if, because copying something is essentially free, the thing should be. It’s embarrassing because our own output is in that form, so we of all people should recognize that the cost of copying it is the least important factor. Or we just call out fair use, as if that applies to everything under the sun. The eye does more, but for me their approach still makes me feel embarrassed that my own community applauds them.
- 6gvONxR4sf7o 6y agoEdit too late to edit: Reading through the The Pile paper they define public data as such: > Public data is data which is freely and readily available on the internet. This primarily excludes data which is pay-walled (regardless of how easy that paywall is to bypass) and data which cannot be easily obtained but can be obtained, e.g. through a torrent or on the dark web. This should disqualify books3, but they use it to justify books3. They need to extent that definition transitively. Maybe that’s my frustration. If I collect data from torrents and make it more freely and readily available, then it meets their definition of public data, which is basically what books3 is. If they included it themselves, it wouldn’t meet their definition but because someone else redistributed it first, it’s more okay for them to redistribute?
- nixtaken 6y agoThey've also scraped HackerNews posts. Since I posted blog links to HackerNews, does that mean they stole all of my blog posts? That represents three years of work and the chapters of 3 books that I intend to publish. They just took it and will start delivering it to their users to help them write more interesting content? Not okay.
- stellaathena 6y agoIf I Google “bibliotik download” the first page of results has a link to download. Calling that “not easy to obtain” is absurd. And yes, your last sentence is correct. That’s exactly the position being taken.
- 6gvONxR4sf7o 6y ago> If I Google “bibliotik download” the first page of results has a link to download. Calling that “not easy to obtain” is absurd. You're right. Then why do you exclude so much else that meets the same criteria? If I Google “[insert dataset here] download” the first page of results has a link to download. Calling that “not easy to obtain” is absurd. In your definitions, you exclude vast swaths of things meeting that criteria in a seemingly arbitrary manner, which is why it's weird.
- dlkf 6y agoIt's always telling when you see a lot of down votes and no counterarguments.