5 ms·
- "That is, you shouldn’t expect it to work with scanned PDFs" It's surprisingly easy to extend this type of workflow to scanned pdfs (as opposed to software-g
by perihelions 3y ago
- "That is, you shouldn’t expect it to work with scanned PDFs"
It's surprisingly easy to extend this type of workflow to scanned pdfs (as opposed to software-generated, text-containing ones). tesseract(1) makes short work of ToC pages with --psm set to 6 (an OCR setting that tends to collapse convoluted text layouts into a regular, software-parseable output).
It should also be straightforward, but I don't know of an out-of-the-box solution, to automate that example of extracting "text that looks like a header"–based on page layout/relative positioning, or font weight. (I'm working on an adjacent problem, an automatic re-layout of raster documents to squeeze out whitespace and make them slightly nicer on small e-ink devices. Text islands are trivial to identify. I don't know how to quantify font weight, or things like that. I'm "wasting" a lot of time diving into lots of mathematics rabbit holes, but I don't know in advance which ones will be productive or not).
- felipefar 3y agotesseract is fine for basic use cases, but it fails when the image is tilted (and thus the text isn't laid out horizontally), which can happen several times with scanned books. Compared to how well the Google OCR engine works, tesseract should be much better than it is. I wonder how difficult it is to develop a better OCR engine than tesseract.
- perihelions 3y agoAm I overlooking something, or is automating page rotation no more work than just a 2d FFT?
- notyoutube 3y agoMind ELI5ing this? it seems neat
- perihelions 3y agoThe Fourier transforms map plane waves to points. Blocks of regularly-spaced text have a periodic character, with the period length of their line spacing; their Fourier transform (I think??) would, in 2d frequency space, have amplitude peaks on vectors that have the same angle as the rotation of the lines.
- notyoutube 3y agothanks!
- aidenn0 3y agoI think it's more typical to low-pass (i.e. blur) the image and then use a line-detection algorithm like the Hough transform. Properly deskewed text should have prominent horizontal white lines.
- perihelions 3y ago- "Hough transform" Oh, that one has much nicer properties—thank you!
- HeatrayEnjoyer 3y agoTesseract is last gen. Multimodal is SOTA, and can handle even heavily distorted or destroyed text.
- aidenn0 3y agoYou are supposed to deskew (and de-warp if the image isn't flat) images before running through tesseract. There are other tools for doing that.
- bsharper 3y agoI've found EasyOCR to work much better at pulling text out of irregular or unknown images. Requires more resources than tesseract but gets much better results in my projects.
- aidenn0 3y agoIt seems to be not significantly better than tesseract for non-mixed images though, and it takes about 5 orders of magnitude longer to process a page on my machine; I can literally read a book 100 times faster than EasyOCR can process a book on my Ryzen 7 2700.
- aidenn0 3y agoTo give an idea, I started EasyOCR on a single page of a book at ~200dpi before I posted the above comment. It is still running over 3 hours later.
- chazeon 3y agoI have thought about using tessaract, using it to OCR the TOC and generate something like this. But there are just so many edge cases that make the whole process fail. For example, how do you handle it if the title breaks into two lines? What if the page number is not recognized correctly? For example, 10 can be 1o What if there are dots? Maybe you can use GPT to clean the extracted text. In the end, I found ChatGPT-4's multimodal capability can recognize text + page number pairs well if I feed screenshots of TOC into it, and I have settled on that.