4 ms·

As University libraries have moved online, one reads many poorly scanned journal articles. I often wonder about taking the time to clean them up. What replaces

As University libraries have moved online, one reads many poorly scanned journal articles. I often wonder about taking the time to clean them up. What replaces temporal information here is the same characters appearing over and over. So of course I read this article hoping to learn about an off-the-shelf tool that would do a great job of scanned text reconstruction. Alas, the best candidates were "no code available."

throwawayciv651 5y ago
Not off the shelf but here are some tools. I have no experience with them. Wolf binarization - I think it makes the text more clear before OCR. https://github.com/chriswolfvision/local_adaptive_binarization https://github.com/chriswolfvision/local_adaptive_binarizati... This thing OCRs the pdf using Tesseract OCR https://github.com/ocrmypdf/OCRmyPDF/ https://github.com/ocrmypdf/OCRmyPDF/ Two other pdf tools https://github.com/qpdf/qpdf https://github.com/qpdf/qpdf https://github.com/pikepdf/pikepdf https://github.com/pikepdf/pikepdf
- Syzygies 5y ago
  I'll play with the Wolf filter, thanks. Math typesetting is too messy for current OCR tools. It would be nice to reverse-engineer the LaTeX source for a math paper, but not likely soon. OCR for the language would help in mind-mapping a web connecting my saved papers, but I wouldn't use it for reading. I want everything to look like a 600dpi scan mixed down, as I would make, rather than what the libraries thought would be acceptable. For the pure joy of reading. The easiest approach that might work would be language agnostic, understanding only what clean scans of characters look like. Can we back-solve a clean scan from a lower resolution mess, matching up similar characters in the text without identifying the characters? Somehow I imagine this is a giant singular value problem. I'm ok if it takes a day to run per paper, I have spare machines.
  - throwawayciv651 5y ago
    Found these 2 for math LaTeX OCR: https://github.com/lukas-blecher/LaTeX-OCR https://github.com/lukas-blecher/LaTeX-OCR https://github.com/harvardnlp/im2markup https://github.com/harvardnlp/im2markup Also some LaTeX editors: LyX https://en.wikipedia.org/wiki/LyX https://en.wikipedia.org/wiki/LyX TeXstudio https://en.wikipedia.org/wiki/TeXstudio https://en.wikipedia.org/wiki/TeXstudio GNU TeXmacs https://en.wikipedia.org/wiki/GNU_TeXmacs https://en.wikipedia.org/wiki/GNU_TeXmacs
  - rasz 5y ago
    I imagine you could go really far with a dumb approach or matching against pre existing font database.

throwawayciv651 5y ago

Not off the shelf but here are some tools. I have no experience with them. Wolf binarization - I think it makes the text more clear before OCR. https://github.com/chriswolfvision/local_adaptive_binarization https://github.com/chriswolfvision/local_adaptive_binarizati... This thing OCRs the pdf using Tesseract OCR https://github.com/ocrmypdf/OCRmyPDF/ https://github.com/ocrmypdf/OCRmyPDF/ Two other pdf tools https://github.com/qpdf/qpdf https://github.com/qpdf/qpdf https://github.com/pikepdf/pikepdf https://github.com/pikepdf/pikepdf

Syzygies 5y ago
I'll play with the Wolf filter, thanks. Math typesetting is too messy for current OCR tools. It would be nice to reverse-engineer the LaTeX source for a math paper, but not likely soon. OCR for the language would help in mind-mapping a web connecting my saved papers, but I wouldn't use it for reading. I want everything to look like a 600dpi scan mixed down, as I would make, rather than what the libraries thought would be acceptable. For the pure joy of reading. The easiest approach that might work would be language agnostic, understanding only what clean scans of characters look like. Can we back-solve a clean scan from a lower resolution mess, matching up similar characters in the text without identifying the characters? Somehow I imagine this is a giant singular value problem. I'm ok if it takes a day to run per paper, I have spare machines.
- throwawayciv651 5y ago
  Found these 2 for math LaTeX OCR: https://github.com/lukas-blecher/LaTeX-OCR https://github.com/lukas-blecher/LaTeX-OCR https://github.com/harvardnlp/im2markup https://github.com/harvardnlp/im2markup Also some LaTeX editors: LyX https://en.wikipedia.org/wiki/LyX https://en.wikipedia.org/wiki/LyX TeXstudio https://en.wikipedia.org/wiki/TeXstudio https://en.wikipedia.org/wiki/TeXstudio GNU TeXmacs https://en.wikipedia.org/wiki/GNU_TeXmacs https://en.wikipedia.org/wiki/GNU_TeXmacs
- rasz 5y ago
  I imagine you could go really far with a dumb approach or matching against pre existing font database.