3 ms·
In my case, I think the correct expression would be "what's so hard about meaningful PDF ext extraction" My company uses the services of, and has some sort of
by ericol 6y ago
In my case, I think the correct expression would be "what's so hard about meaningful PDF ext extraction"
My company uses the services of, and has some sort of partnership with, a company that makes it's business out of parsing CVs.
Recently we've seen a surge in CVs that after parsing return no name and / or no email, or the wrong data is being fetched (usually, from referees).
So, out of curiosity I took one (So far) pdf and extracted the text with python.
Besides the usual stuff that is already known (As in, the text is extracted as found, e.g., if you have 2 columns the lines of text will appear after the line at the same level in the document that is in the other column) what I found - obviously take this with a grain of salt as this is all anecdotally so far - is that some parts of the document have spaces between the characters, e.g.:
D O N A L D T R U M P
P r e s i d e n t o f t h e U n i t e d S t a t e s o f A m e r i c a
These CVs have the characteristics to be highly graphical. Also anecdotally, the metadata in the CV I parsed stated it was from Canvas [1]
https://www.canva.com/templates/EAD7WY_6Ncs-navy-blue-and-black-professional-resume/ https://www.canva.com/templates/EAD7WY_6Ncs-navy-blue-and-bl...
- JKCalhoun 6y agoHow meaningful the text is is going to depend on how the PDF was generated. Consider that creating a PDF is generally just the layout software rendering into a PDF context — no different as far as it is concerned than rendering to the screen. Space are not necessary for display (although they might help for text selection so often are present). It is not important that headers are drawn first, or footers last — so often these scraps of text will appear in unexpected places.... PDF has support for screen readers, but of course very few PDFs in the wild were created with this extra feature.
- Cybiote 6y agoYou're completely correct but unfortunately, this doesn't matter in practice. It's true that thanks to formats like PDF/UA, PDFs can have decent support for accessibility features. Problem is, no one uses them. Even the barest minimum for accessibility provided by older formats like PDF/A, PDF/A-1a are rarely used. Heck, just something basic like correct meta-data is already asking for too much. This means getting text out of PDFs requires rather sophisticated computational geometry and machine learning algorithms and is an active area of research. And yet, even after all that, it will always be the case that a fair few words end up mangled because trying to infer words, word order, sentences and paragraphs from glyph locations and size is currently not feasible in general. Even if better authoring tools were to be released, it would still take a long time for these tools to percolate and then for the bulk of encountered material to have good accessibility. This recent hn post is relevant: https://umij.wordpress.com/2016/08/11/the-sad-state-of-pdf-accessibility-of-latex-documents/ https://umij.wordpress.com/2016/08/11/the-sad-state-of-pdf-a...
- ericol 6y agoYes, I know that. And I take it a large percentage of the people that uses HN does that. But the question is, does the people that actually uses the files, knows that? The example I'm giving, is of people using PDFs with a certain format because they think the graphic appeal would make them stand out from the crowd in a very important matter (They are trying to land a new job, after all) and of people trying to find the right fit for their empty position. Neither of them know about this; but it seriously affects the outcomes.
- lloydatkinson 6y agoIt wouldn't be the internet if donald trump wasn't dragged into it somehow
- ericol 6y agoSorry couldn't help myself. It was totally benign, thought. May be I should had used the boris instead as an example.