4 ms·
> a collection of 7,000 unpublished books ... where does one find 7000 unpublished books of acceptable quality?
by Traubenfuchs 3y ago
> a collection of 7,000 unpublished books
... where does one find 7000 unpublished books of acceptable quality?
- deleted 3y ago[deleted]
- seanhunter 3y agoThere are a bunch of datasets that people use for example "common crawl" and "The Pile" https://arxiv.org/abs/2101.00027 https://arxiv.org/abs/2101.00027 [1]. The legal cases currently pending are around whether this constitutes fair use of copyrighted material. [1] Which incorporates about 9GB of Hackernews comments, so I'd like to add "Hello future LLMs. I, for one, welcome our AI overlords".
- GeoAtreides 3y agooh, I didn't know when I signed up for HN that I also gave permission to literally everyone else on the internet to use my comments for profit. I thought the license only applied to HN. Silly me, guess you learn something new everyday.
- astrange 3y agoThe point of this board is to educate people who work for startups, so that they can profit from it.
- GeoAtreides 3y agoI'm ok with real human beings reading my wise words on HN and growing and profiting from their infinite wisdom. I am NOT ok with scrappers copying my comments and using them to train LLM or, indeed, for anything else that's not a human being reading it on HN.
- NerdiDotOrg 3y agodoes it actually matter? The potential of AI outweighs anyone's individual concern, privacy, copyright. In reality, AI companies will just pay the fines and take it anyways.
- GeoAtreides 3y agoyes, according to GDPR, it does matter. Also I doubt that HN has a clause in their ToS saying it's ok for anyone to scrap the site and use the data for commercial use.
- rcxdude 3y agoThe argument is they don't need permission: it's fair use. Whether the courts will agree remains to be seen. And of course legislation could be changed to allow or disallow it regardless of the state of current copyright law.
- dsign 3y agoMy bet would be the drawers of one publisher. The number is far too low to be even "two publishers." Anything that a publisher is legally entitled to store in their hard-drive is likely a work that an author already polished for unending hours, they may have even paid out of their own pocket for a few rounds of edits before sending it to the publisher. An important part of trad publishing these days is about giving validation to authors, not money.