4 ms·

ocrmypdf and friends. I've built an archival system based around Tesseract and PostgreSQL. It takes Images/PDFs, either scanned or generated, and rebuilds them

by asguy 6y ago

ocrmypdf and friends. I've built an archival system based around Tesseract and PostgreSQL. It takes Images/PDFs, either scanned or generated, and rebuilds them as searchable PDFs before being extracted and inserted into Postgres' full-text search. I keep all of the original media because disk is cheap. Originally I used Tesseract directly. But I found that ocrmypdf did a better job than my home-grown pipeline, so I switched.

thibautg 6y ago
Cool I've done exactly the same with ocrmypdf! With a Django web app to search through the scanned documents (around 30k documents and 200k pages).
crocodiletears 6y ago
Very nice. This is quite similar to what I've been building for myself.
Ologn 6y ago
Tesseract is the best FOSS one I found when I looked a bit back. I don't conceive of a superior FOSS one any time soon unless a commercial one open sources, or some one utilizing deep learning comes out.
- kranner 6y ago
  Tesseract 4 has switched to using a deep learning engine by default.
tuddman 6y ago
I also built a system that extracted structured and unstructured text from images/pdfs. For the generated pdfs, I found pdftotext could pull with 100% fidelity, and so that was 'option #1'. for scanned-images-saved-as-pdfs, then tesseract could sometimes extract with 90+% accuracy. But never 100%. Combining pdftotext (with the right flags set) with some of the other associated pdf-tools, we were able to achieve what we were after: Building a searchable DB and auto-informing corpus of information derived entirely from various pdf sources. All in-house. No sending off to 3rd parties.
- undebuggable 6y ago
  > For the generated pdfs, I found pdftotext could pull with 100% fidelity, and so that was 'option #1'. for scanned-images-saved-as-pdfs, then tesseract could sometimes extract with 90+% accuracy. Arrived to a similar conclusion although never have bothered with DB or any web interface running locally. Simply grepping the text files works flawlessly for me.

by asguy 6y ago

thibautg 6y ago

Cool I've done exactly the same with ocrmypdf! With a Django web app to search through the scanned documents (around 30k documents and 200k pages).

crocodiletears 6y ago

Very nice. This is quite similar to what I've been building for myself.

Ologn 6y ago

Tesseract is the best FOSS one I found when I looked a bit back. I don't conceive of a superior FOSS one any time soon unless a commercial one open sources, or some one utilizing deep learning comes out.

kranner 6y ago
Tesseract 4 has switched to using a deep learning engine by default.

tuddman 6y ago

I also built a system that extracted structured and unstructured text from images/pdfs. For the generated pdfs, I found pdftotext could pull with 100% fidelity, and so that was 'option #1'. for scanned-images-saved-as-pdfs, then tesseract could sometimes extract with 90+% accuracy. But never 100%. Combining pdftotext (with the right flags set) with some of the other associated pdf-tools, we were able to achieve what we were after: Building a searchable DB and auto-informing corpus of information derived entirely from various pdf sources. All in-house. No sending off to 3rd parties.

undebuggable 6y ago
> For the generated pdfs, I found pdftotext could pull with 100% fidelity, and so that was 'option #1'. for scanned-images-saved-as-pdfs, then tesseract could sometimes extract with 90+% accuracy. Arrived to a similar conclusion although never have bothered with DB or any web interface running locally. Simply grepping the text files works flawlessly for me.