4 ms·

Wow, every author I searched for is in there. I’m very bullish on technology progress and the future of generative AI, but at the same time I cannot fathom dec

by tailspin2019 3y ago

Wow, every author I searched for is in there. I’m very bullish on technology progress and the future of generative AI, but at the same time I cannot fathom deciding to consume such a vast amount of pirated content (or even any pirated content) and then releasing a paid service off the back of it, and not expecting trouble down the line. For an open source, non-commercial project it might be controversial, but commercialising the aggregate pirated content of 180,000 stolen books is pretty bold. There is a very clear line between content that’s freely available (eg. scraping the web) and this. And even the web scraping is obviously controversial for some…

sillysaurusx 3y ago
I didn’t intend for it to be commercialized. It’s a research contribution, nothing more. It was also freely available via the web when I did it. I got all the books from https://the-eye.eu/public/Books https://the-eye.eu/public/Books. So it was literally web scraping. In fact, you can, too. https://the-eye.eu/public/Books/ThoseBooks/ https://the-eye.eu/public/Books/ThoseBooks/ seems to have a wide selection, though it’s an order of magnitude less than their Bibliotik scrape. It’s interesting that no one cared until recently.
- deleted 3y ago
  [deleted]
- abrookewood 3y ago
  Just because it was web scraping doesn't mean it was OK to take it. You knew the sources contained copyrighted material and you did it anyway. I'm not trying to attack you but I don't see how saying it was 'research' makes it OK.
- tailspin2019 3y ago
  > I didn’t intend for it to be commercialized. It’s a research contribution, nothing more. FWIW my comment is aimed at the likes of OpenAI who are now profiting off this work. For example, every book that I own (as in paid money for) and searched for is in books3. I pay OpenAI currently for ChatGPT Pro. I feel like I should be getting a discount... I don't really have a strong opinion on the collation of material for research purposes as you did. LLMs are going to have a truly outsized impact on the world in the long term. But we do still need to consider the authors (individual creators) who are getting caught up in the crossfire of this knowledge/technology revolution.

sillysaurusx 3y ago

I didn’t intend for it to be commercialized. It’s a research contribution, nothing more. It was also freely available via the web when I did it. I got all the books from https://the-eye.eu/public/Books https://the-eye.eu/public/Books. So it was literally web scraping. In fact, you can, too. https://the-eye.eu/public/Books/ThoseBooks/ https://the-eye.eu/public/Books/ThoseBooks/ seems to have a wide selection, though it’s an order of magnitude less than their Bibliotik scrape. It’s interesting that no one cared until recently.

deleted 3y ago
[deleted]
abrookewood 3y ago
Just because it was web scraping doesn't mean it was OK to take it. You knew the sources contained copyrighted material and you did it anyway. I'm not trying to attack you but I don't see how saying it was 'research' makes it OK.
tailspin2019 3y ago
> I didn’t intend for it to be commercialized. It’s a research contribution, nothing more. FWIW my comment is aimed at the likes of OpenAI who are now profiting off this work. For example, every book that I own (as in paid money for) and searched for is in books3. I pay OpenAI currently for ChatGPT Pro. I feel like I should be getting a discount... I don't really have a strong opinion on the collation of material for research purposes as you did. LLMs are going to have a truly outsized impact on the world in the long term. But we do still need to consider the authors (individual creators) who are getting caught up in the crossfire of this knowledge/technology revolution.