4 ms·

I don’t really understand how OCRing a PDF could be more accurate than reading the text nodes. Another thing to consider is OCR works well for English but not

by 4pkjai 2mo ago

I don’t really understand how OCRing a PDF could be more accurate than reading the text nodes. Another thing to consider is OCR works well for English but not so well for other languages.

defrost 2mo ago
As the article stated, in this specific case the bank used an "invisible" character (a minus sign) to assist in alignment .. which resulted in errors when parsing the text nodes by layout. OCR of a digital image generated from those PDF draw commands only operates on the visible portions that are rendered. Another case to be considered .. FOI requests for documents from "pain in the arse" government departments or law firms can result in "digital documents" that are deliberately generated to be difficult to process (go figure) .. eg: filled with circular and spaghetti draw commands that "print correctly" but are a nightmare horror show to parse. ( "technically compliant" is a legally compliant "fuck you" )
- TZubiri 2mo ago
  I've had a non technical user ask me how to convert a document to PDF so that it's not editable or copypasteable by someone else. And also ask me how to edit a pdf. PDF is used as DRM by obfuscation by non technical users, it's a sort of emergent phenomenon that occurs hundreds of times per day. And it turns out it's a quite effective tool at that, if anything it associates a cost to editing/copying/automatic processing. In terms of technically compliant, a PDF is as much of a sensible requirement as USB or String types. It checks the box, but it's likely that there's a lot of additional work into integrating two systems that input and ouput these envelope protocols/encoding, because they are purposefully too broad.
TZubiri 2mo ago
I didn't mention anything about accuracy, I said that it would be simpler to implement, which might or might not be correlated with accuracy. However in this case, we see a scenario where OCR would be more accurate indeed. So just reread the article and you might understand how OCR can be more accurate. It's because in case of disparity between the printed optical 2D matrix and some interpretation of the text/print encoding, the optical matrix is correct. PDF is a printing encoding, not a machine consumption, there's thousands of ways to print a document, it's not feasible or practical to understand all of them to build a document parser, you need to focus on the end result, not the process, like we do when analyzing a document. If you are analyzing the Mona Lisa, maybe it's useful to understand how it was painted. But if you are analyzing your electricity bill, it's likely that going into what order the numbers were printed and what ink was used, is going to be more of a distraction than anything useful. >Another thing to consider is OCR works well for English but not so well for other languages. Maybe, not terribly in scope for the case in the OP, as long as arabic numerals are used, that's the core of the requirements.

by 4pkjai 2mo ago

I don’t really understand how OCRing a PDF could be more accurate than reading the text nodes. Another thing to consider is OCR works well for English but not so well for other languages.

defrost 2mo ago

As the article stated, in this specific case the bank used an "invisible" character (a minus sign) to assist in alignment .. which resulted in errors when parsing the text nodes by layout. OCR of a digital image generated from those PDF draw commands only operates on the visible portions that are rendered. Another case to be considered .. FOI requests for documents from "pain in the arse" government departments or law firms can result in "digital documents" that are deliberately generated to be difficult to process (go figure) .. eg: filled with circular and spaghetti draw commands that "print correctly" but are a nightmare horror show to parse. ( "technically compliant" is a legally compliant "fuck you" )

TZubiri 2mo ago
I've had a non technical user ask me how to convert a document to PDF so that it's not editable or copypasteable by someone else. And also ask me how to edit a pdf. PDF is used as DRM by obfuscation by non technical users, it's a sort of emergent phenomenon that occurs hundreds of times per day. And it turns out it's a quite effective tool at that, if anything it associates a cost to editing/copying/automatic processing. In terms of technically compliant, a PDF is as much of a sensible requirement as USB or String types. It checks the box, but it's likely that there's a lot of additional work into integrating two systems that input and ouput these envelope protocols/encoding, because they are purposefully too broad.

TZubiri 2mo ago

I didn't mention anything about accuracy, I said that it would be simpler to implement, which might or might not be correlated with accuracy. However in this case, we see a scenario where OCR would be more accurate indeed. So just reread the article and you might understand how OCR can be more accurate. It's because in case of disparity between the printed optical 2D matrix and some interpretation of the text/print encoding, the optical matrix is correct. PDF is a printing encoding, not a machine consumption, there's thousands of ways to print a document, it's not feasible or practical to understand all of them to build a document parser, you need to focus on the end result, not the process, like we do when analyzing a document. If you are analyzing the Mona Lisa, maybe it's useful to understand how it was painted. But if you are analyzing your electricity bill, it's likely that going into what order the numbers were printed and what ink was used, is going to be more of a distraction than anything useful. >Another thing to consider is OCR works well for English but not so well for other languages. Maybe, not terribly in scope for the case in the OP, as long as arabic numerals are used, that's the core of the requirements.