3 ms·
The title got me really excited that they were doing full text search. Boy that would be an awesome project. Zlib and Google Books do it, but it would be great
by droopyEyelids 3y ago
The title got me really excited that they were doing full text search. Boy that would be an awesome project. Zlib and Google Books do it, but it would be great to have a open source version that everyone could contribute to, and provided access to full texts
- raybb 3y agoOpenLibrary does provide search access to full texts. For example: https://openlibrary.org/search/inside?q=%22institutional+thickness%22&mode=everything https://openlibrary.org/search/inside?q=%22institutional+thi... It is open source and they're always looking for contributors. I think they'd especially welcome help improving search! https://github.com/internetarchive/openlibrary/ https://github.com/internetarchive/openlibrary/
- mellutussa 3y agoI think a distributed OCR project is needed. Problem is that a lot of books are PDF scans and missing raw text. OcrMyPdf does a pretty good job of is but it's cpu intensive.
- greggsy 3y agoI'd wager that there are several players in the AI market who have already scraped and OCR'd every book and magazine on zlib and libgen to feed into training models. Google are almost certainly piped everything they have in Google Books into their models, before some future legal case says they can't. Won't take long before the open community starts doing the same.