3 ms·

It sounds like you either know a lot more than me or a lot less than me. The PDFs I've dealt with don't store text as strings, they store it as individual chara

It sounds like you either know a lot more than me or a lot less than me. The PDFs I've dealt with don't store text as strings, they store it as individual characters. This left me having to write a heuristic based algorithm to group the characters into words, words into lines, lines into paragraphs, paragraphs into columns. Again, as far as I know, there are no heuristics good enough to get that right for all values of PDF.

belval 6y ago
He knows a lot less than you probably because there is absolutely no requirements for PDFs to be in text format and most aren't. The "text" he is editing could render to completely different characters depending on how the PDF document was created. The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a").
- yjftsjthsd-h 6y ago
  > The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a"). What? Why!? I've heard of doing that as a form of DRM, but I can't imagine Darwin defaulting to doing that.
  - belval 6y ago
    I never dug deeper into it, so I am not aware of why it does that or if it's a specific version or whatnot, but take a PDF from which you can extract the text (with pdftotext/pdfbox for example). Open it in the document viewer and "print" it to PDF. If you extract the text again it is not readable anymore. This wouldn't be an issue if it was a conscious choice, but when I parsed a lot of born-digital PDFs we ended up with a lot that were like that from various source. Try explaining that...
    - colejohnson66 6y ago
      Could it be “compacting” the fonts? So if U+0000 to U+0007F aren’t used at all, remove those glyphs and set U+0000’s glyph to be what was U+0080? Yes, I know NULL doesn’t have a glyph, but I hope that gets the idea across.
      - belval 6y ago
        That would be my personal guess yes, as there are other ways to "protect" PDF text like curve-based rendering.
- Someone 6y ago
  Where did I say PDFs have to be in text format? I said every PDF has an equivalent text-only representation. The format, like postscript and eps started as text-only. Compression, making the files binary was added as an afterthought, to make files smaller (much smaller). If it were binary from the start, its designers would not have made the table of contents at the end waste bytes by writing offsets in ascii. See http://blog.idrsolutions.com/?s=%22Make+your+own+PDF+file http://blog.idrsolutions.com/?s=%22Make+your+own+PDF+file for more info on how to write PDF by hand (also shows why I said you have to be stubborn to do that)
  - belval 6y ago
    "OK, they added various ways of data compression, but PDF is, basically, a text-based format." You are trying to move the goal post here. The above statement is simply untrue, PDF is not a text based format and it's that simple.
mkl 6y ago
More. Go here and download the PDF spec.: https://www.adobe.com/devnet/pdf/pdf_reference.html https://www.adobe.com/devnet/pdf/pdf_reference.html Look at Chapter 3, Syntax. The code is all text based. We are not talking about the visible characters in a PDF viewer, but the code of the PDF file itself.
- deleted 6y ago
  [deleted]
- roflc0ptic 6y ago
  Oh, true. I misread, and also didn't know that. Thanks for the info!
  - hnick 6y ago
    Going by your original premise though, just so you know the reference does indeed have examples of text as strings e.g. BT /F13 12 Tf 288 720 Td (ABC) Tj ET This can be extended to include spaces so you can essentially mark up entire lines of text at one time. What it can't do is cohesive paragraphs and flow/wrap, you need to use the relative positions to work out what text is in one block (and usually I'd defer to something like pdftotext for simple cases). Laying out individual characters is common though. It's probably due to kerning concerns.

belval 6y ago

He knows a lot less than you probably because there is absolutely no requirements for PDFs to be in text format and most aren't. The "text" he is editing could render to completely different characters depending on how the PDF document was created. The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a").

yjftsjthsd-h 6y ago
> The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a"). What? Why!? I've heard of doing that as a form of DRM, but I can't imagine Darwin defaulting to doing that.
- belval 6y ago
  I never dug deeper into it, so I am not aware of why it does that or if it's a specific version or whatnot, but take a PDF from which you can extract the text (with pdftotext/pdfbox for example). Open it in the document viewer and "print" it to PDF. If you extract the text again it is not readable anymore. This wouldn't be an issue if it was a conscious choice, but when I parsed a lot of born-digital PDFs we ended up with a lot that were like that from various source. Try explaining that...
  - colejohnson66 6y ago
    Could it be “compacting” the fonts? So if U+0000 to U+0007F aren’t used at all, remove those glyphs and set U+0000’s glyph to be what was U+0080? Yes, I know NULL doesn’t have a glyph, but I hope that gets the idea across.
    - belval 6y ago
      That would be my personal guess yes, as there are other ways to "protect" PDF text like curve-based rendering.
Someone 6y ago
Where did I say PDFs have to be in text format? I said every PDF has an equivalent text-only representation. The format, like postscript and eps started as text-only. Compression, making the files binary was added as an afterthought, to make files smaller (much smaller). If it were binary from the start, its designers would not have made the table of contents at the end waste bytes by writing offsets in ascii. See http://blog.idrsolutions.com/?s=%22Make+your+own+PDF+file http://blog.idrsolutions.com/?s=%22Make+your+own+PDF+file for more info on how to write PDF by hand (also shows why I said you have to be stubborn to do that)
- belval 6y ago
  "OK, they added various ways of data compression, but PDF is, basically, a text-based format." You are trying to move the goal post here. The above statement is simply untrue, PDF is not a text based format and it's that simple.

mkl 6y ago

More. Go here and download the PDF spec.: https://www.adobe.com/devnet/pdf/pdf_reference.html https://www.adobe.com/devnet/pdf/pdf_reference.html Look at Chapter 3, Syntax. The code is all text based. We are not talking about the visible characters in a PDF viewer, but the code of the PDF file itself.

deleted 6y ago
[deleted]
roflc0ptic 6y ago
Oh, true. I misread, and also didn't know that. Thanks for the info!
- hnick 6y ago
  Going by your original premise though, just so you know the reference does indeed have examples of text as strings e.g. BT /F13 12 Tf 288 720 Td (ABC) Tj ET This can be extended to include spaces so you can essentially mark up entire lines of text at one time. What it can't do is cohesive paragraphs and flow/wrap, you need to use the relative positions to work out what text is in one block (and usually I'd defer to something like pdftotext for simple cases). Laying out individual characters is common though. It's probably due to kerning concerns.