8 ms·

Google with default settings is useless and using more sophisticated queries quickly walls off the user with increasingly annoying captcha. The "world's knowle

Google with default settings is useless and using more sophisticated queries quickly walls off the user with increasingly annoying captcha. The "world's knowledge under you fingertips" motto is still valid and brilliant though. My personal solution is library of OCR-ed PDFs with most established books from various domains, git repository for each domain. Greppable in miliseconds, locally. Hijack this, SEO experts!

handsomechad 7y ago
do you have a link to this solution
- durnygbur 7y ago
  ImageMagick and Tesseract for OCR-ing each page of a PDF into a separate text file (through TIFF image format, disregard the huge TIFFs afterwards), private git repos for hosting, then ag/grep for searching. Not as easy to find the phrase back in PDF as with eg. Google Books, but then GB with copyright related content restrictions is useless most of the time.
Nextgrid 7y ago
How did you acquire the books? Are they in the public domain, or did you have to buy them? In either case is there a place to acquire/buy these books massively or did you do it one by one manually?
- durnygbur 7y ago
  Whichever most convenient way to obtain a full restriction-free PDF of a book. Fetched one by one through various channels in my case. Very few are in the public domain, if any. BTW one can do the same with academic publications, device manuals, or whatever else content available in PDF.
FiatLuxDave 7y ago
Your personal solution sounds interesting. I'd be interested to know more details about it, and how it works for you. If it is how I imagine it, it could also be one of those things that could be built into a tool that could rival Wikipedia or search engines. Based on your reply to handsomechad, you may think that it's easiest for people to just build one themselves. But there may be a business opportunity in providing a pre-packaged solution for the masses. In the same way that Dropbox provided a tool that was "trivial" to experts, but was difficult for non-experts (see the infamous comment here: https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224), if you have a tool that is essentially a rival to Google Books for reference texts, that is interesting.
- DeathArrow 7y ago
  There's b-ok.org

handsomechad 7y ago

do you have a link to this solution

durnygbur 7y ago
ImageMagick and Tesseract for OCR-ing each page of a PDF into a separate text file (through TIFF image format, disregard the huge TIFFs afterwards), private git repos for hosting, then ag/grep for searching. Not as easy to find the phrase back in PDF as with eg. Google Books, but then GB with copyright related content restrictions is useless most of the time.

Nextgrid 7y ago

How did you acquire the books? Are they in the public domain, or did you have to buy them? In either case is there a place to acquire/buy these books massively or did you do it one by one manually?

durnygbur 7y ago
Whichever most convenient way to obtain a full restriction-free PDF of a book. Fetched one by one through various channels in my case. Very few are in the public domain, if any. BTW one can do the same with academic publications, device manuals, or whatever else content available in PDF.

FiatLuxDave 7y ago

Your personal solution sounds interesting. I'd be interested to know more details about it, and how it works for you. If it is how I imagine it, it could also be one of those things that could be built into a tool that could rival Wikipedia or search engines. Based on your reply to handsomechad, you may think that it's easiest for people to just build one themselves. But there may be a business opportunity in providing a pre-packaged solution for the masses. In the same way that Dropbox provided a tool that was "trivial" to experts, but was difficult for non-experts (see the infamous comment here: https://news.ycombinator.com/item?id=9224 https://news.ycombinator.com/item?id=9224), if you have a tool that is essentially a rival to Google Books for reference texts, that is interesting.

DeathArrow 7y ago
There's b-ok.org