3 ms·

Is there a technology that would help ocr-ing standardized forms? I’ve got a couple thousand pages of historical train schedules that i would like to digitize (

by ant6n 7y ago

Is there a technology that would help ocr-ing standardized forms? I’ve got a couple thousand pages of historical train schedules that i would like to digitize (tabulated, printed data, including symbols/icons), but I’m not sure how to automatically recognize the structured data.

GijsjanB 7y ago
I don't know about out-of-the-box ready software. Depending on how much time you have you'd either train your own model with your own ground truth or you can use an open source model. Libraries and archives sometimes make theirs available (I know the Dutch National Archives do). You can train the model and/or HTR the data with software like Transcribus or Kraken (open source).
jasoncabot 7y ago
(disclaimer MS employee but nothing to do with this product) You could check out the Azure Forms Recognizer. It's designed to do exactly this and make even training a custom model pretty easy. It's also pretty cheap (free up to 500 pages) so you can always do a quick experiment and see if it does what you need it to https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/overview https://docs.microsoft.com/en-us/azure/cognitive-services/fo...
martyvis 7y ago
I remember getting some reference software and methods from NIST maybe 20 years ago on handwriting recognition. Never put it into practice, but maybe they have some updated software available
froindt 7y ago
I'm curious, what's your interest in the train schedules? One day at an antique shop, I came across a book from ~1910 which had hundreds of pages of annual reports from railroads with many metrics we'd expect to see in the 10-K reports public companies file. The book was published annually, but had much of its data in tables with grouped headers and cells, which could make automated OCR-ing with a good (useful) end result challenging. I think it'd be interesting to map out the Railroad consolidation, track all their financial metrics over time, and do some level of forensic accounting to see if/which companies probably had funny business going on.
- ant6n 7y ago
  I’ve got European rail schedules, which gives an indication of historical travel patterns. Specifically looking into the development of overnight travel.

by ant6n 7y ago

GijsjanB 7y ago

I don't know about out-of-the-box ready software. Depending on how much time you have you'd either train your own model with your own ground truth or you can use an open source model. Libraries and archives sometimes make theirs available (I know the Dutch National Archives do). You can train the model and/or HTR the data with software like Transcribus or Kraken (open source).

jasoncabot 7y ago

(disclaimer MS employee but nothing to do with this product) You could check out the Azure Forms Recognizer. It's designed to do exactly this and make even training a custom model pretty easy. It's also pretty cheap (free up to 500 pages) so you can always do a quick experiment and see if it does what you need it to https://docs.microsoft.com/en-us/azure/cognitive-services/form-recognizer/overview https://docs.microsoft.com/en-us/azure/cognitive-services/fo...

martyvis 7y ago

I remember getting some reference software and methods from NIST maybe 20 years ago on handwriting recognition. Never put it into practice, but maybe they have some updated software available

froindt 7y ago

I'm curious, what's your interest in the train schedules? One day at an antique shop, I came across a book from ~1910 which had hundreds of pages of annual reports from railroads with many metrics we'd expect to see in the 10-K reports public companies file. The book was published annually, but had much of its data in tables with grouped headers and cells, which could make automated OCR-ing with a good (useful) end result challenging. I think it'd be interesting to map out the Railroad consolidation, track all their financial metrics over time, and do some level of forensic accounting to see if/which companies probably had funny business going on.

ant6n 7y ago
I’ve got European rail schedules, which gives an indication of historical travel patterns. Specifically looking into the development of overnight travel.