The open source project I work on [0] returns the letters, their positions and other associated information. We provide support for retrieving words as well as

The open source project I work on [0] returns the letters, their positions and other associated information. We provide support for retrieving words as well as a bunch of different algorithms for document layout analysis [1]. But like the other commenters here mention, it's an extremely difficult problem which doesn't have an easy or general solution. I was trying to build a custom library on top of the open-source library that did a bit more processing, multi-column analysis, statistical analysis of whitespace size, etc. But building something that works for the general case is difficult enough to be functionally impossible. Despite that I think the PDF format is well suited to what it is for and there are very few "implementation mistakes" in the spec itself (no up-front length for inline image data is the main one, plus accessibility obviously). It's ultimately become too successful and as a result developers are stuck handling cases where it's being used for entirely the wrong purpose but I can't see a way to another format gaining purchase for the correct purpose (perhaps it's like JavaScript in that way, it has huge adoption because it was first, not because it does all jobs well). Perhaps a content-first format which also handles presentation well could gain a foothold if it came with a shim for PDF viewers and software to use but I dread to think how much effort that would be. [0]:https://github.com/UglyToad/PdfPig https://github.com/UglyToad/PdfPig [1]:https://github.com/UglyToad/PdfPig/wiki/Document-Layout-Analysis https://github.com/UglyToad/PdfPig/wiki/Document-Layout-Anal...