6 ms·
I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow li
by TFNA 6mo ago
I’m a researcher who for years has been scanning my library’s holdings on my particular discipline for my own use, but also uploading the books to the shadow libraries for everyone else’s benefit. The revelation that LLMs are training on the shadow libraries has made me put a lot more effort into ensuring my scans are well-OCRed. The idea that I could eventually ask ChatGPT or whatever about obscure things in my field, and get useful output (of the "trust but verify" sort), is exciting.
- BrenBarn 6mo agoHow about the idea that you might have to eventually pay an AI company a large amount of money to ask ChatGPT such a question, while the library itself has lost funding?
- TFNA 6mo agoSome people might have to pay a large amount of money to ask a commercial LLM, but advances in this space mean that if I have the data myself on my own computer, or can download it from a shadow library, I might eventually be able to ask everything locally for free. > while the library itself has lost funding Libraries are inherent parts of universities. While their precise role evolves, do you think that they will just be done away with? Already a substantial amount of scholarship in disciplines other than my own has moved online (legally), and the library is still there.
- BugsJustFindMe 6mo agoLibrary funding is a political stance that has only imaginary connection to whether people pay to ask things of ChatGPT. People can pay to talk to an AI and also government can fund libraries.
- bakugo 6mo agoDo you believe it makes sense for the government to fund libraries that almost nobody uses because they'd rather ask ChatGPT?
- indigo945 6mo agoPeople are already not using libraries because they'd rather rot their brains on TikTok than read a book. (Also, for information lookup, the internet and search engines exist, and have for a while now.) This has no actual causal relation.
- snaking0776 6mo agoPeople is a broad term. Outside of major cities (where I live) libraries serve a very essential service for parents and their children and as a free communal space for the broader community. Our libraries are always full and a large part of the health of our area.
- breezybottom 6mo agoWeird that my local library is always full.
- Aurornis 6mo ago> People are already not using libraries because they'd rather rot their brains on TikTok than read a book I rotate through the libraries near me with my kids. They are every bit as busy now as I remember them being when I was a kid.
- stingraycharles 6mo agoIf people prefer to pay ChatGPT, rather than going to the library for free, and ChatGPT sources content from libraries, then sure that makes sense, especially if the information contained is of cultural relevance to the government. It’s the same as asking “should you release open source software knowing that AI companies are training on them”. I could absolutely not care less, that’s not the point why I release my software to the public at all.
- saagarjha 6mo agoChatGPT would of course want a library to exist so they can train on it. However, they are not incentivized to let other people have access to it.
- soco 6mo agoThe government can then soon "optimize" and fund exactly one library.
- locknitpicker 6mo ago> How about the idea that you might have to eventually pay an AI company a large amount of money to ask ChatGPT such a question, while the library itself has lost funding? There are plenty of free models with RAG support. Why do you believe everything starts and ends with a major corporation charging a subscription?
- BrenBarn 6mo agoNot everything starts that way but these days it sure seems like everything ends that way.
- spoaceman7777 6mo agoFree, downloadable AI models have consistently caught up to ChatGPT within 3 months, for almost a year now. I highly encourage you to go and update your priors.
- roygbiv2 6mo agoAnd how much does the hardware cost to run said models?
- dboreham 6mo agoYou can run them slowly on any machine that has enough memory.
- JKCalhoun 6mo agoAnd, to bolster your comment, you can still use this machine as your daily driver. I'm always going to have a machine anyway—might as well max out the RAM when I purchase another. (And so too I jumped on the Mac mini bandwagon a month or two back—64 GB. I'm enjoying pulling down the new models and putting them through my paces.)
- fragmede 6mo agoHow good do you want it to be? For a close to ChatGPT today (April, 2026), you're still looking at a system with 7xH200+chassis, which will run you $300, or a GB200 NV72, which is $2-3 million. OTOH, a Qwen3.6 quantized model can be run on $10,000 (high end Mac) or $1,000 (Mac mini) worth of hardware. Even a Pixel 10 Pro cellphone ($1,000) can run useful models locally.
- dzink 6mo agoGo to Open Router, ask your own in investigative prompt that meets your needs to all the top open models. See how they do. Then notice if you can run any of those locally. Repeat at least once a month.
- deleted 6mo ago[deleted]
- protocolture 6mo agoHow about the idea that one day you might be paying a subscription to use a service while non sequitur.
- woctordho 6mo agoA digital library needs almost no funding. With today's decentralized networking infrastructure such as BitTorrent and IPFS I bet it just exists forever.
- tardedmeme 6mo agoHow much of Anna's Archive are you seeding?
- woctordho 6mo agoAbout 4 TB at hand
- x-complexity 6mo ago> A digital library needs almost no funding. Clarification: To maintain the library still requires resources & effort to do so. It only appears to need no funding because the donators of said (disk space / bandwidth / dev effort) are subsidizing it in aid of a goal they believe in (i.e. the church model).
- Tangurena2 6mo agoThe way public libraries currently "lend" digital books is that they can only lend titles a certain amount of time before the library has to repurchase the title (or remove it from circulation).
- roenxi 6mo ago1. Being offered a service you would pay a lot of money for is a step forward. When people pay a large amount of money for something that means they wanted the thing more than the money. The link between ChatGPT and libraries being under threat seems a bit weak too. 2. The Chinese have been investing a lot into free models, they're perfectly good and keep improving; despite the best efforts of the US. They're even ramping into making their own hardware. Gemma 4 is pretty snappy too. It doesn't seem like there is much of a moat to this, my guess is there will be perfectly good local models if you want to avoid AI companies.
- cheschire 6mo agoWhen people pay a large amount of money for something that means they wanted the thing more another thing. Money just provides the method to defer value transfer. When the person paying the money is rich, the other thing they are foregoing is typically not a life necessity. When the person is poor, however, it typically is.
- Papazsazsa 6mo ago[flagged]
- nullsanity 6mo ago[dead]
- deleted 6mo ago[deleted]
- TFNA 6mo agoOf course not, and many authors are already long dead. But if you knew anything about academic publishing, the authors almost invariably are happy to see their work out there freely available. It’s not as if they make any money from it, and the more eyes on their work, the better their chances of getting cited and thereby furthering their careers. It is some publishers who would object on copyright grounds. But I get the sense that some publishers are already becoming resigned to the fact that most of their new ebook releases are ending up on the shadow libraries within only a few weeks, and Anna’s Archive has become the first place to look (even before one looks at whether one’s own institutional library has the book) for researchers around the world.
- Papazsazsa 6mo ago[flagged]
- ddtaylor 6mo agoWhy assume people lock knowledge in a box and charge for access?
- red75prime 6mo agoThe ridiculously long "70 years after the author's death" makes it highly problematic in many cases.
- altmanaltman 6mo agoHow is any of that legal? Can you just take books from the library and then scan and upload digital copies? How do you deal with the ethics of this personally, stealing to make it easier for AI to steal so AI gets better? Does calling yourself a "researcher" make you feel like its actually something worthwhile you're doing?
- TFNA 6mo agoAs a researcher, the main worthwhile thing that I am doing is publishing research, but having all this prior scholarship at hand 24/7 definitely makes it easier to produce said publications. And if I have created a scan, why not help out my colleagues, too? "Deal with the ethics", seriously? You might want to learn about how heavily shadow libraries are used across academia now. It’s no longer just disadvantaged scholars in the developing world relying on pirated scans because they don’t have good libraries. It’s increasingly everyone everywhere, because today’s shadow libraries can be faster and more convenient than even one’s own institution’s holdings. At conferences, if the presenter mentions a particularly interesting publication, you can sometimes watch several people in the room immediately open LibGen or Anna’s Archive on their laptop to download it right there and then.
- SomaticPirate 6mo ago[flagged]
- reacweb 6mo agoI think the current intellectual property system is flawed. Books are knowledge, and we shouldn't be able to limit the spread of knowledge. I imagine that books could be sold at the cost of printing, and there could be a QR code inside so that readers could freely donate money to the author if they enjoyed the book. Strangely enough, I imagine that with such a system, authors would be better paid.
- breezybottom 6mo ago
- emsign 6mo agoThat's a slave mentality. You are aware that OpenAI charges money for other people's work and intelligence, right? Your own and that of other volunteer pirates and of the original authors as well. I don't get people like you at all.
- TFNA 6mo agoI’ve already posted in this thread about how even if OpenAI charges money for its LLM trained on the literature, that doesn’t change the fact that the literature remains available to everyone through the shadow libraries, and advances in AI mean that one can increasingly work with it locally on one’s own computer.
- __alexs 6mo agoOpen weight models exist and are critical to us avoiding a future where you have to pay sama a slice of every engineers salary.
- wallst07 6mo ago>I don't get people like you at all. Because you don't try, which says more about you than OP. It's a major problem with society.
- lelanthran 6mo ago> The idea that I could eventually ask ChatGPT or whatever about obscure things in my field, and get useful output (of the "trust but verify" sort), is exciting. That's your idea, not the one they are going with. Their idea is that you pay a fee to access any information that was freely available. Your idea is tearing down of fences, their idea is gatekeeping. The two ideas are incompatible.
- baq 6mo agoTheir idea is being able to get answers to questions which were difficult to answer before[0]. Of course they want to get paid for it. The information wasn’t available easily and not always[1] freely. [0] among other things… [1] more like ‘often not at all’
- entrox 6mo ago> Of course they want to get paid for it. So should the original authors, no? That is, getting a share of that payment. Something akin to the German GEMA could work, an entity that levies a usage fee on behalf of all copyright holders and re-distributes to its members, but on a global scale.
- inetknght 6mo ago> So should the original authors, no? That is, getting a share of that payment. Should they? Yes. Will they? Well, do LLM model builders pay for any copyrighted work so far?
- entrox 6mo agoWell, not yet. It's a matter of organization, regulation and litigation. I was thinking along the lines of concepts that already exist, such as the private copying levy [0]. It basically forces a blanket tax on a certain class of products, which then gets redistributed to members of a collecting society such as GEMA [1]. This way, you would force LLM model builders to effectively pay a tax by law. Since these models do not work at all without underlying content, make it proportionate. Let's say 50-70% to make it fair. [0] https://en.wikipedia.org/wiki/Private_copying_levy https://en.wikipedia.org/wiki/Private_copying_levy [1] https://en.wikipedia.org/wiki/GEMA_(German_organization) https://en.wikipedia.org/wiki/GEMA_(German_organization)
- tokai 6mo agoHasn't that been scanned by Google already? Their model should be trained on most of those texts already.