5 ms·
This is really cool. Use of GPT-3 to augment OCR is an amazing (and, retrospectively, obvious) insight and a great immediate use case for these language models.
by georgecmu 5y ago
This is really cool. Use of GPT-3 to augment OCR is an amazing (and, retrospectively, obvious) insight and a great immediate use case for these language models.
I wish Remarkable took this idea -- they really oversold their OCR capabilities[1]. It works great in their support and promo videos, but I found the actual performance to be absolutely terrible.
[1] https://support.remarkable.com/hc/en-us/articles/360002661437-Convert-handwritten-notes-into-text https://support.remarkable.com/hc/en-us/articles/36000266143...
- visarga 5y agoIt's an old idea, using a language model on top of character level OCR. Works well for general text but doesn't solve random sequences of digits and letters. So you can't use it to correct your invoices where you have lots of out-of-dictionary tokens.
- einpoklum 5y ago> using a language model on top of character level OCR But if you know you're going to use a language model after the OCR, then you don't OCR to a single character, but rather to a distribution of character similarity (e.g. the 90% least similar or clipping at a certain similarity threshold). Then the language model should have more to work with (although TBH its work becomes more complicated).
- georgecmu 5y agoIf a dictionary satisfies your definition of a language model, yes, with predictably poor results[1]. If I understand correctly, Google Books approach[2] represented a major improvement in accuracy of automated OCR (and this is for printed text!), but I would venture to say that implementing a language model like this would be far beyond the scope of a 'tiny project'. [1] https://tesseract-ocr.github.io/docs/Limits_on_the_Application_of_Frequency-based_Language_Models_to_OCR.pdf https://tesseract-ocr.github.io/docs/Limits_on_the_Applicati... [2] https://tesseract-ocr.github.io/docs/Improving_Book_OCR_by_Adaptive_Language_and_Image_Models.pdf https://tesseract-ocr.github.io/docs/Improving_Book_OCR_by_A...
- bloak 5y agoI've always found it somehow ironic that a human can correctly recognise printed characters even if parts of them are missing and the word is misspelt or in a language the human does not know at all, but computers have to resort to language models because an exact comparison of part of the image with other parts of the image (where the same letter is printed in the same font) for some reason is not feasible?
- throwawayboise 5y agoBrains do patten recognition much better than computers (albeit slower)
- Andrew_nenakhov 5y agoFor now.
- Super_Jambo 5y agoHumans must be using a language model for image recognition when reading though. Otherwise things like failing to spot the the is a duplicate word wouldn't happen so often.
- Retric 5y agoI think people use font models rather than just language models. The post office is perhaps the best example where despite a known list of addresses and intended format they still backup OCR with people.
- AussieWog93 5y ago> The post office is perhaps the best example where despite a known list of addresses and intended format they still backup OCR with people. To be fair, the amount of people who write their own address incorrectly is staggering. I'm in eCommerce and easily 70%+ of addresses have some sort of minor error in them. Around 5-10% are just plain bizarre, with things like two suburbs being included or the street name not including the Rd/Av/Dr etc. I'd suspect this is one of those problems that seems easy in the lab but quickly degenerates when you consider the human aspect of it.
- Swenrekcah 5y agoI actually just now bought their subscription with the particular idea to use it with my reMarkable. Then I only now realised I don’t actually want my notes public, I hope there is some form of access control built into this! :D
- ineedasername 5y agosome form of access control built into this! It would also be great if you could delete a page by taking a picture of the notebook entry completely scribbled out, or a video of tossing it into the fire. We need a product roadmap for this!
- kragen 5y agoThis is cool, but what's GPT-3's pricing model and roadmap?
- twic 5y agoWhat could possibly go wrong? https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres...
- ronenlh 5y agoCrazy: “ The error does occur because image segments, that are considered as identical by the pattern matching engine of the Xerox scan copiers, are only saved once and getting reused across the page. If the pattern matching engine works not accurately, image segments get replaced by other segments that are not identical at all, e.g. a 6 gets replaced by an 8.”
- tdehaene 5y agoYou can also do something similar but with seq2seq models instead of decoder-only models: https://blog.ml6.eu/ocr-correction-with-byt5-5994d1217c07 https://blog.ml6.eu/ocr-correction-with-byt5-5994d1217c07