24 ms·
PDF Is the World's Most Important File Format
- revskill 8y agoArchiving without extracting is kind of one-way to me. I prefer a long JSON file, which i can read from, or write down it to file format i want, to a hard-to-extract format such as PDF.
- owenmarshall 8y ago> a hard-to-extract format such as PDF What's hard about it? It's an open standard with libraries in every language known to man. If you use it as a dumb wrapper for scanned images, that's going to suck. But as a way to store and faithfully reproduce nicely typeset documents with images - ie, to make an archival copy - I don't think it can possibly be beat.
- krastanov 8y agoI think the comment was about bulk extraction of data. For instance, it is pretty difficult to do bibliographic studies if all the articles you have are provided only as PDFs (even if it is text pdf). For starters, I do not think PDFs have a notion resembling a "paragraph of text", because every symbol is placed separately with its own unique coordinates. Extracting tables of numbers from PDFs is also a pain.
- maxxxxx 8y agoExactly. I have written software for legal document search. Most of what they get is in PDF and it’s a major PITA to get data out of them. Forget about tables. Just try to extract text without some garbled characters and you will lose your mind.
- maxxxxx 8y agoAfter spending time extracting data and text from PDF I would also say it’s the worst file format. We have perfectly fine structured documents, convert them to PDF to lose half of the information and then we spend insane effort to get the data back somehow. It also doesn’t render well on different resolutions. PDF is a perfect case study how inferior solutions can become standards.
- cgtyoder 8y agoIt's important to differentiate between the file format and software that renders that file format. Admittedly there are better and worse PDF viewers, but that shouldn't be the final (or even most important) determiner of how important it is.
- disqard 8y agoYes, I often use SumatraPDF, but there's always those super-complicated Turing-complete documents which use web-based-aunthenticated-signed God-knows-what, that won't open in anything other than Acrobat.
- SigmundA 8y agoPDF is based on Postscript, it designed for print perfect output, its essentially a bunch of rendering commands for a printer its not supposed to be a word processing format that can be edited later its the final output from a word processor or whatever.
- mattnewton 8y agoRight, but somehow it gets used for all sorts of things that are never printed in offices everywhere.
- SigmundA 8y agoIt gets printed to the screen for reading and still retains the text for searching. The ability to easily edit can actually be liability in many situations. Its literally a replacement for paper documents, except you can search and view them on a computer and still hit print and get a great paper reproduction. Word processing/editing has so much more going on, PDF actually removes features from postscript for security purposes, nobody wants to be sending word docs around just for viewing.
- irrational 8y agoI don't know. I personally think plain text is a more important file format. It's read and writeable by a million programs. Any first year programming student can easily build their own program to read and write plain text. It is fairly easily parseable. Etc. I just opened a random PDF on my computer in a text editor and it starts off with "xÕ\ko‹∆˝Œ_1@ÉbD 9|ÌËáƒZçÌDJÇ¢)UZ[nıÚÆσA˛Pˇeœπ"
- SllX 8y agoArguably it actually is plain text, but I think you can build a case for at least a dozen file formats that they’re the most important file format in the world, so arguably, none of them are.
- SigmundA 8y agoIt also very inefficient especially if you want things like images embedded in it. Its not like you can read text without the right program, its still binary, there just happens to be a mostly agreed upon standard and a lot of programs that can decode that standard and render to screen. PDF viewers are built into most browsers now and allows rich page perfect print ready results. If I had it my way we would settle on a binary data serialization format, I don't care MsgPack, Protobuf, heck maybe even sqlite and then everyone could have a viewer to snoop around in it. You would still have to understand whats being encoded but you could always "view source" so to speak.
- hieronymusN 8y agoFor more info on the PDF format: [VIDEO] Programming data for display, the PDF Story by Chas Emerick - https://www.youtube.com/watch?v=MAki8C6qFHY&list=PLGRqfvsPiRShRY94F1p_3TT1HMoipcc3c&index=6&t=0s https://www.youtube.com/watch?v=MAki8C6qFHY&list=PLGRqfvsPiR...
- ashelmire 8y agoEven among non-technical people this isn't true. Those people use excel and word far more than pdf. For devs, we're all about various text formats.
- jerzyt 8y agoMy gripe with PDF is that I don't understand why a standard format which is almost 30 years old, requires seemingly weekly updates of the Acrobat Reader, which in turn requires reboot of my work laptop. I upgrade the reader far more often than I actually use it.
- cromulent 8y agoSounds like your gripe is with Adobe, not PDF.
- jerzyt 8y agoTechnically, you are correct, but in a corporate environment of managed software, I can't separate the format from the company. When an update is scheduled I have no choice but install and reboot.
- justadudeama 8y agoCouldn't you open it in a browser? Or does it want to install regardless of if you open Adobe?
- SigmundA 8y agoNow days you don't need acrobat reader typically, Chrome, Edge and Firefox can all view/print PDF's. If your on a Mac/iOS the viewers are built as the OS uses display postscript internally (quartz) which matches up pretty closely with PDF.
- bluedino 8y agoChrome/Firefox don't handle complex PDF's well. They render slowly and sometimes inaccurately It's not uncommon to print things from Chrome/Firefox and have the margins/cropping be wrong. Then again you're using a web browser to print a pdf file, you get what you deserve
- 8y ago
- levesque 8y agoPossibly also the most hated file format. Just the fact that no two PDF editors behave the same way goes to show how bad of a format it is.
- timw4mail 8y agoPDFs for reading is one thing. Fillable PDFs are evil incarnate. Apparently there are multiple ways to do it, and only Acrobat Reader is capable of making them all work. Then you have abominations like embedded flash... As a standalone file representing the format of a book, PDFs are a good format. But then PDFs can unfortunately (sometimes) store much more, and then can be a security minefield.
- kxyvr 8y agoI used to have good luck using pdftk for filling out forms and the IRS seemed to encode them well, but it looks like pdftk is mostly dead. Nowadays, evince seems to work OK, sometimes, for filling out forms. And, yes, Acrobat seems to be the only piece of software that does all of them. That said, my personal favorite for a complete screw you to the format comes from Texas and form 2382: https://hhs.texas.gov/laws-regulations/forms/2000-2999/form-2382-assisted-living-facilities-checklist https://hhs.texas.gov/laws-regulations/forms/2000-2999/form-... If you decompress it, you'll find that they encoded an HTML webpage into the PDF and, as far as I can tell, this can only be viewed on Windows with Acrobat. It's almost like Texas doesn't want people to apply for medicaid funds.
- phonon 8y agoThey used Livecycle (XFA) forms. It's not part of the latest PDF standard. https://en.wikipedia.org/wiki/XFA https://en.wikipedia.org/wiki/XFA Here is a plain Acroform version :-) https://send.firefox.com/download/3d2da3ccd62d4ce9/#CJRCa5XOhZyW2mWMctlkdg https://send.firefox.com/download/3d2da3ccd62d4ce9/#CJRCa5XO...
- kxyvr 8y agoHa! Finally an answer to what these forms are. Thanks! Speaking of which, what software did you use to make the conversion? It might be nice to pass this information along to the agency to see if they'd be willing to convert all their forms.
- 8y ago
- anon1253 8y agoIt's also actively harming medical and scientific progress as I allude to here in this talk https://www.youtube.com/watch?v=EM61rn9Gxl4&list=PLjzcwcP9P2Lfcz38lDtMF_XupRnecKokz&index=12 https://www.youtube.com/watch?v=EM61rn9Gxl4&list=PLjzcwcP9P2...
- mkl 8y agoCan you summarise your point about that here?
- nathan_f77 8y agoI run a PDF generation service [1], so this is nice to read. When I first launched the service, I was worried that PDFs and paper forms might become obsolete in the near future, when everyone starts to go paperless and digital. Now that I'm more familiar with the market, this is no longer a concern. (I have banks who are using the software to modernize their operations.) I also realized that there might be some pressure to turn into the next TurboTax, where the company eventually lobbies against improvements just so we can stay in business. I made a resolution that I'll never do anything like that. But I guess the founders of TurboTax never intended to do that either. [1] https://formapi.io https://formapi.io
- nojvek 8y agoI love little profitable businesses like this. Seems like you’re living the dream
- lo_fye 8y agoNot moreso than .txt
- crazygringo 8y agoI didn't "get" PDF's until I started doing academic research. But the ability to collect papers, books, documents, etc. all in a single format that I can read on any device, and mark up with highlights and notes, has been a game-changer. Yes it's a lowest-common-denominator format. And it's designed for human reading and manual office tasks, not computer processing of data. But it works. And it's supported everywhere. Doesn't matter if I'm on the Apple or Google or Microsoft or Adobe stack. Doesn't matter if the PDF is 20 years old. It just works.
- whenchamenia 8y agoExcept for many, it doesen't. Why glorify a closed format that we spent decades trying to aviod just because its common now? The inherent issues persist.
- boshomi 8y agosince ISO 32000-2 there is no proprietary technology included[1] [1] https://www.iso.org/standard/63534.html https://www.iso.org/standard/63534.html
- throwaway2048 8y agoNot all PDFs are standards compliant, plenty are filled with acrobat only JS, embedded flash and form filling crap.
- semi-extrinsic 8y agoAs e.g. the fine men and women of Int. J. POC||GTFO have demonstrated, it's possible to embed all sorts of weird and wonderful things in a PDF. I think a format where it's possible to include as much as possible is better. Just look at HTML. Would you rather have it be impossible to extend beyond text, images and stylesheets?
- na85 8y ago
- bla3 8y agoThis would've been a lot more convincing before mobile happened a decade ago. Reading PDFs on my phone is a pain, and it hasn't improved at all in the last ten years. Looking at the web on mobile used to be painful too, but it's ok now.
- casefields 8y agoWhat phone? I'm still using an iPhone 6S and they are seemless. Some textbooks that are like 300+ mb start having hiccups though. Probably due to my oldish phone.
- bla3 8y agoThey're fast, but I have to scroll and zoom around a lot.
- dana321 8y agohttps://www.youtube.com/watch?v=-cFOsAzigyQ https://www.youtube.com/watch?v=-cFOsAzigyQ
- bluedino 8y agoI love PDFs. The first time I used one was Acrobat 2.0. I bought a shitty SAMS computer book that came with a bonus shitty SAMS computer book on the CD. As a PDF of course. It was cumbersome on a Pentium 75MHz with 640x480x8 graphics. I'd be blown away 20 years later viewing those same files on a Macbook Pro with Retina display. However, it was easily the best way to distribute printed documents EXACTLY the way they were meant to be seen. They weren't meant to be edited, modified, data extracted from...And adobe went from a quick, minimal viwer to a bloated security nightmare by adding 'features. Luckily, 3rd party and open-source projects came to the rescue.
- djhworld 8y agoI sometimes read papers but I find them an absolute pain to read them on my phone because the two aren't really designed for each other. So I end up with this frustrating experience of zooming in/out to read this bits I'm interested in - which is impossible to do one handed! I know there's that arxiv vanity thing, which is cool, but most of the time I get the "sorry we can't render this" error message.
- interlocutor 8y agoPDF is based on PostScript, which is a Page Description Language (PDL) also from Adobe. The first PostScript printer, the Apple LaserWriter, launched the desktop publishing revolution. The interesting thing about PostScript is that it is a full programming language, with loops and conditionals and so on. When Adobe designed PDF, they kept the same imaging model as PostScript, but stripped out the programming language. Thus they ended up with a dynamic page description language (PostScript) for media (printers) that cannot fully take advantage of a dynamic PDL, and a static PDL (PDF) for media (i.e., computer screens) that could have benefited from a full programming language!
- srean 8y agoPostscript is a pretty interesting stack based language on its own. Its size is intimidating though. If I remember correctly there was a toy web-server written in postscript just to show that it is possible.
- agumonkey 8y agosomeone made a submission to ICFP contest which was a self drawing post script solution
- username223 8y ago> When Adobe designed PDF, they kept the same imaging model as PostScript, but stripped out the programming language. And then they added JavaScript to it: http://www.adobe.com/devnet/acrobat/javascript.html The small subset that displays text and images is fine, but PDF is a nightmare.
- UglyToad 8y agoI've been working on an open source PDF library for C# [1] and its given me an immense appreciation for the PDF format. Sure it's horribly long, complex, comes with vulnerabilities and different consumers have different behaviour. But given the constraints of machines at the time it was created and the wide range of requirements and usages it's pretty damn good and has stood the test of time. [1]: https://github.com/UglyToad/PdfPig https://github.com/UglyToad/PdfPig
- walterbell 8y agoCould this be used to remove potentially-malicious content from a PDF, e.g. anything executable?
- UglyToad 8y agoUnfortunately not at the moment sorry, the editing and creation capabilities are extremely limited at present. The library is mainly focused on text extraction but editing is on the road map.
- snailmailman 8y agoIt’s a different software entirely, but QubesOS has an interesting way of removing malicious content from a pdf. It basically opens the document in a throwaway VM, then the host takes a high resolution screenshot and makes a new pdf from scratch. The digital equivalent of physically printing and re-scanning the document.
- 781 8y agoIf they said "Most Important DOCUMENT File Format", maybe, through even then you have stuff like JPEG. But arguably executable file formats (.exe, .so, ...) are more important overall.
- c0vfefe 8y agoHaving some executable file is a prerequisite for opening any other!
- alexhutcheson 8y agoAuthor must not realize how many businesses, government organizations, etc. have operations that are completely dependent on .xls files that someone first created in the late 90s.
- lixtra 8y agoWhat happened to HTML? Nobody could read nor discuss about PDF here without it.
- SigmundA 8y agoLet me know when browser support the full CSS print spec...no Prince doesn't count.
- lixtra 8y agoTell me when PDF can reasonably reflow in all environments/devices.
- SigmundA 8y agoPDF are not meant for reflow, layout has already been done by the program that made them, pagination is complete. Again I would be happy with HTML if the major browsers actually implemented full CSS print spec that exists to cover what PDF does with print layout.
- deleted 8y ago[deleted]
- creatornator 8y agoThe whole point of a pdf is to display _exactly_ the same across all devices. For something like a scientific research paper, you want all figures, etc., to remain in the same place to avoid formatting oddities.
- lixtra 8y agoThat’s an important feature, just like reflow. But does the former make PDF World’s most important file format?
- speedplane 8y ago>> The whole point of a pdf is to display _exactly_ the same across all devices. > That’s an important feature, just like reflow. But does the former make PDF World’s most important file format? It's not just that PDFs are displayed the same across devices, it's that it's displayed the same over time. I can open a PDF generated in 1995 and it will look identical today as it did then. The same can't be said about HTML or Word documents generated in 1995.
- shadowtree 8y ago.xls runs half of the world economy.
- boromi 8y agoAdobe acrobat reader has been atrocious. I'm so glad I switched to sumatra this year.
- nyfresh 8y agoPDF is the perfect tool for maintaining document formatting but the worst tool for maintaining document data. The format is not concerned the the actual data of document. Programatically extracting data from the simplest PDF is an exercise in patience. I find it kind of odd that this was chosen as the standard all things considered
- jyriand 8y agoPDF is fine if you are using a laptop/desktop/tablet. But it doesn’t fit well into phone screens. Usually I have to turn my phone horizontally and then zoom in little bit. And when I reach the end of the page I have trobles turning it. And sometimes page turning messes up my perfect zoom fit etc. Responsive PDF is what I need...
- shanhaiguan 8y agoThat’s not a PDF issue though: it’s just an issue of most content being the same dimensions as real world paper sizes. No idea how responsive PDF would work but I agree that would be cool.
- colejohnson66 8y agoPeople in these comments keep mentioning a responsive PDF, but PDF is supposed to be a presentation format. Wouldn’t HTML and CSS serve the purpose fine?
- akvadrako 8y agoYou can't represent vector graphics, complex math, or anything non-standard in HTML+CSS. Even if you could, you couldn't rely on the user seeing it correctly.
- colejohnson66 8y agoAren’t SVG and MathML standard in HTML5?
- akvadrako 8y agoInteresting; I'm not sure if it counts as "in HTML5" but I see they are referenced in the spec. It's still not something you can rely upon.
- vdm 8y agohttps://github.com/koreader/koreader https://github.com/koreader/koreader
- AtlasBarfed 8y agoWorse is better strikes again? I'm not talking about SGML
- jdblair 8y agoI remember when PDF came out, I didn't get it. I thought the problem was already solved by compressed postscript files! I was already used to downloading and sometimes printing paper documentation in this format. It was natural to view Postscript files on NeXT and UNIX machines, and Ghostscript was already thing. What could be better than just using the "native" language of the printer? I didn't realize that this was not a common view, or even possible for most personal computers at the time. I was also misinformed for quite some time about the internal format of PDF, assuming it was just PS wrapped up in a container. In a sense, this is true, but there's a lot more (embedded fonts, transparency, forms are just a few that come to mind).
- mkl 8y ago> I was also misinformed for quite some time about the internal format of PDF, assuming it was just PS wrapped up in a container. In a sense, this is true, but there's a lot more (embedded fonts, transparency, forms are just a few that come to mind). There's also a lot less, as PDF is not a full programming language like PS.
- wyld_one 8y agoIt is not open data. You still can have proprietary blocks of info inside the file. The lack of open source tools to manipulate the format is a major hindrance IMHO. It is also a very space wasting as well when people only do a bitmap dump into the file for scans. Forms are also an area that is not open source either. It has so many hacks and kludges, it would be better if it was trashed and we start with postscript again.
- qrbLPHiKpiux 8y agoTo me, a pdf is only a digital version of a paper version. Just to look at, present... nothing more. Data duplication, extraction always had to be done manually. I could never extract text correctly. So I don’t anymore. I just accepted it. As I did with a lot of other things.
- ris 8y agoThe thing I find craziest about PDF/A is it isn't really a format in itself, just a vague promise not to use certain features in the ensuing file. Whether any reader holds the file to that promise is something I'm quite doubtful of. Instead I suspect most readers will do their best to display anything they're handed, happily passing it through any of the hundred-or-so, possibly legacy-code-powered sub-format decoders the file author wishes - leading to a massive attack surface. From a developer's point of view, when trying to enforce that submitted files are strictly in PDF/A format, from what I can tell there isn't much more you can do than dissect the file looking for umpteen disallowed features. Is there an ISO-compliance validator to anyone's knowledge?
- bondolo 8y agoPDFs are an accessibility nightmare and most production pipelines are terrible at preserving the semantic structure of the document or, in many cases, even preserving the text. A PDF is in many cases as series of page images which aren't usable for anything other than human viewing or printing. "Export as PDF" generally produces much better results than print to PDF though not from every application. Many days I wish there was an alternative solution based on SVG. While not perfect it certainly would avoid many of the problems of PDF while having all of the important capabilities.
- deleted 8y ago[deleted]
- thatoneuser 8y agoI guess if you wanna claim that, but it's a shit format that causes way more pain than convenience for most people. Just today I had to upload sensitive docs to a server to edit it because I can't locally and refuse to pay to do so. Fuck pdf.
- sbuk 8y agoIt’s not designed to be edited.
- thatoneuser 8y agoOh so all those fillable pdfs are just hacks then right?
- stcredzero 8y agoAre there any people going around and harvesting data of all the PDFs in the world, using Machine Learning to clean collate, and localize the data?
- johnvega 8y agoMicrosoft tried to compete with XPS, xml paper specification around mid 2000's but did not succeed.
- aktuel 8y agoI couldn't disagree more. I almost never use PDFs. I read a lot of books on my mobile phone and PDFs are basically unusable there due to their fixed layout. To say in 2019 that a file format without mobile support is the most important is moronic.
- xvilka 8y agoAnd at the same time PDF Forms[1], animation[2] and 3D extensions are poorly supported in FOSS implementations - poppler, mupdf, etc. [1] https://gitlab.freedesktop.org/poppler/poppler/issues/463 https://gitlab.freedesktop.org/poppler/poppler/issues/463 [2] https://gitlab.freedesktop.org/poppler/poppler/issues/683 https://gitlab.freedesktop.org/poppler/poppler/issues/683
- KingMachiavelli 8y agoI really hate PDF for many reasons. It seems it's only a partially open format, a lot of features are implementation dependent (how can a file be 'locked' to prevent printing or editing?). There are very few free and FOSS clients that handle forms, highlighting, etc. Some clients do highlighting and annotations but don't save it to the PDF itself. The failure of epub and other HTLM based formats in this use case, IMO, is that their focus on reflowing to support any display and device makes them inconsistent and therefore impractical for replacing PDF based content.
- topicseed 8y agoSurely database file formats are at least as important to hold whatever data the World needs to go round.
- Adamantcheese 8y agoPDF is great! For human reading. But it's garbage for everything else, and, as I found out from a a bunch of frustrating job application efforts, absolute shit for resumés. Literally not parseable, even with OCR apparently. I don't even know WHY any job application even accepts a PDF for a resumé. It shouldn't be allowed to if there's any sort of post-processing done to it to extract information. A bunch of applications didn't even let me see what it extracted, which, depending on the PDF to text application they're using, may spit out absolutely nothing from an entire page of text. That's right, a whole page converted to a single empty line. Marvelous. Truly a technological marvel that's helping the "it's hard to get a job" feeling go away.
- forrestthewoods 8y agoPDF is the one and only format any resume should ever be submitted in. It's the only way to reasonably guarantee someone will be able to open and read a resume in its intended form.
- Adamantcheese 8y agoOnly from my view, once it's automatically parsed, nobody is going to look at it, because either it'll automatically throw it out from the parsed data (in which case a blank resume is an obvious throw away), or it'll do some keyword parsing and determine you haven't copy/pasted enough. The second one is a problem with all of those systems though. After that point it'll pass it on to the hiring manager, at which point they'll look at it regardless of format. PDF then is simply not the best choice if it gives the highest chance of failure at the first step. A Word document is a much better choice.
- forrestthewoods 8y ago> A Word document is a much better choice. It really truly isn't. Anyone on any device can reliably open a PDF. That is not true for Word files. Funny enough, this thread made me want to look at my old resumes. I have a .doc resume from 2009. You know what happens when I double-click it? Nothing. I don't have anything that can view it! Windows 10 doesn't come with a preview tool for doc files. Chrome/Firefox can't preview it either.
- fxfan 8y agoI will leave an old subthread here https://news.ycombinator.com/item?id=19785487 https://news.ycombinator.com/item?id=19785487 which discusses how PDF has a major shortcoming which makes it non-portable for any kind of internationalization