5 ms·

PDF is a bad format unless you are printing. I worked for the US Federal Government where we had millions of stored PDFs. At one point in the Federal Judiciary

by tdhz77 4y ago

PDF is a bad format unless you are printing. I worked for the US Federal Government where we had millions of stored PDFs. At one point in the Federal Judiciary we had one of the largest databases in the world. Why? PDFs. We pushed hard for a true digital format like html with a printable format, but the powers that be want a 1:1 replica for Search Warrants and Judges Orders. We can do better for sure, but as a result I was knee deep in pdf’s. It’s tiresome painful little spec. Maybe this go library can solve so many inconsistencies in the pdf world..

auggierose 4y ago
I don't see how PDF is not a true digital format. I guess the powers that be understand something that you don't.
- tashbarg 4y ago
  I guess parent is focusing on the point, that PDFs can render as perfectly human-readable documents, but can be completely non-machine readable at the same time.
- jimjimjim 4y ago
  PDF is a true digital format. In the same way as a zip file is. A pdf page can be made a many many different ways. It depends on what use you are targeting. You want 1:1 digital replica of a page? scan the page as a tiff and add it to a page as an image. Or you could just add the text to the page and the font. Or if you want to mess with people or you are a cad application you draw text as thousands of little lines.
- viraptor 4y ago
  PDFs are not useful for any processing. They represent things you want to print, but not search, understand, analyse, etc. Even those with text actually attached / extractable have no structure. "Selecting blocks of text" involves guessing which order the lines go in, depending on their location / distance from other lines. Compare to having for example "<recipient-address>...</...>" from which you can still generate the printed version.
  - xwolfi 4y ago
    Well usually the pdf processing I did always assumed I had a paper of x by y cm, and a mask I d move around "make a 10 by 20 cm rectangle at position (100,200), what s in that rectangle" basically. There's a structure, just not tag-based but position-based ? Ofc, if humans shit around and change it, you're fucked with versionning your masks, but usually they print from form templates themselves. As I used to say to my colleagues bemoaning this inconvenient analogue bridge: "if you can read it coherently as a human, we can parse it". We have to accept that administrations communicate via geometry and not semantic, and adapt while we also try to convince them to give structured tagging a chance. But they need a critical mass of their documentation pipeline to be machine-read before they even accept to discuss it.
    - 411111111111111 4y ago
      So your point is that pdf is a shit format that needs a custom parser for each document type before it can be used for anything but print...? I'm really confused here, it seems we all agree that pdf is a bad format?
      - kaba0 4y ago
        I don’t know, it seems to be a format that does its intended job just fine. What’s next, will we hate txt as that’s not a good for spreadsheets?
      - xwolfi 4y ago
        It's as shit a format as non-UTF strings, yet it's everywhere and we must adapt, is my point. We can adapt to scanned documents before all documents are semantically tagged, just like we have to adapt to non standard ascii extensions in non English countries, is my point.
    - viraptor 4y ago
      And by "if humans shit around and change it" you mean things that regularly happen and need to be accounted for like moving the physical location to a place with the address one line longer, or getting a new partner which changes the letterhead, or adding extra information required by legal, or ...
- lxgr 4y ago
  In that sense, a blurry, warped TIFF is a true digital format as well.
franga2000 4y ago
> We pushed hard for a true digital format like html with a printable format, but the powers that be want a 1:1 replica for Search Warrants and Judges Orders. Is there a reason you can't have both? Presumably you have structured data at some point, before it's laid out on the page and saved as PDF. Why not just save that alongside the PDFs? You could also serialize it and include it in a PDF metadata field, so it can be extracted from the files even if the database is lost.

by tdhz77 4y ago

auggierose 4y ago

I don't see how PDF is not a true digital format. I guess the powers that be understand something that you don't.

tashbarg 4y ago
I guess parent is focusing on the point, that PDFs can render as perfectly human-readable documents, but can be completely non-machine readable at the same time.
jimjimjim 4y ago
PDF is a true digital format. In the same way as a zip file is. A pdf page can be made a many many different ways. It depends on what use you are targeting. You want 1:1 digital replica of a page? scan the page as a tiff and add it to a page as an image. Or you could just add the text to the page and the font. Or if you want to mess with people or you are a cad application you draw text as thousands of little lines.
viraptor 4y ago
PDFs are not useful for any processing. They represent things you want to print, but not search, understand, analyse, etc. Even those with text actually attached / extractable have no structure. "Selecting blocks of text" involves guessing which order the lines go in, depending on their location / distance from other lines. Compare to having for example "<recipient-address>...</...>" from which you can still generate the printed version.
- xwolfi 4y ago
  Well usually the pdf processing I did always assumed I had a paper of x by y cm, and a mask I d move around "make a 10 by 20 cm rectangle at position (100,200), what s in that rectangle" basically. There's a structure, just not tag-based but position-based ? Ofc, if humans shit around and change it, you're fucked with versionning your masks, but usually they print from form templates themselves. As I used to say to my colleagues bemoaning this inconvenient analogue bridge: "if you can read it coherently as a human, we can parse it". We have to accept that administrations communicate via geometry and not semantic, and adapt while we also try to convince them to give structured tagging a chance. But they need a critical mass of their documentation pipeline to be machine-read before they even accept to discuss it.
  - 411111111111111 4y ago
    So your point is that pdf is a shit format that needs a custom parser for each document type before it can be used for anything but print...? I'm really confused here, it seems we all agree that pdf is a bad format?
    - kaba0 4y ago
      I don’t know, it seems to be a format that does its intended job just fine. What’s next, will we hate txt as that’s not a good for spreadsheets?
    - xwolfi 4y ago
      It's as shit a format as non-UTF strings, yet it's everywhere and we must adapt, is my point. We can adapt to scanned documents before all documents are semantically tagged, just like we have to adapt to non standard ascii extensions in non English countries, is my point.
  - viraptor 4y ago
    And by "if humans shit around and change it" you mean things that regularly happen and need to be accounted for like moving the physical location to a place with the address one line longer, or getting a new partner which changes the letterhead, or adding extra information required by legal, or ...
lxgr 4y ago
In that sense, a blurry, warped TIFF is a true digital format as well.

franga2000 4y ago

> We pushed hard for a true digital format like html with a printable format, but the powers that be want a 1:1 replica for Search Warrants and Judges Orders. Is there a reason you can't have both? Presumably you have structured data at some point, before it's laid out on the page and saved as PDF. Why not just save that alongside the PDFs? You could also serialize it and include it in a PDF metadata field, so it can be extracted from the files even if the database is lost.