13 ms·
I looked into coding PDFs once. Then I closed my MacBook (Pro) and went for a long walk into the ocean. I think I almost got to America, but then I turned and s
by kebman 6y ago
I looked into coding PDFs once. Then I closed my MacBook (Pro) and went for a long walk into the ocean. I think I almost got to America, but then I turned and swam back again. Turnd out I had just fallen asleep and had a nightmare. I was actually just working with regular text files, and everything was fine.
- belval 6y agoAs someone still working with PDF processing, I can confirm that it doesn't get easier.
- divbzero 6y agoI had to handle action buttons for a PDF once. I swam out from America a long long ways before turning back. Might have spotted you middle of the ocean.
- meshaneian 6y agoLegit username warns of one PDF peril.
- rvense 6y agoMy favourite PDF fact is that it doesn't have to start at the beginning or end at the end of a file. Any sea of bytes that contains a PDF file is an acceptable PDF file...
- TeMPOraL 6y agoDid anyone try to pluck out PDFs from /dev/urandom? How about from radiotelescope feed? Maybe the first evidence of extraterrestrial life will be some poor alien's tax form?
- willis77 6y agoThe digits of pi contain every pdf that ever could and ever will exist.
- mtzet 6y agoWell maybe. We don't know if pi is a normal number.
- SteveGoob 6y ago> We don't know if pi is a normal number. Sure we do. There are plenty of proofs out there that pi is an irrational number.
- andreareina 6y agoIrrational does not imply normal. For example, 1.01001000100001... is irrational but it's certainly not normal.
- x3c 6y agoTechnically, 1.01001000100001... can be normal depending on what ... stands for. :)
- coddle-hark 6y agoOr what 1 or 0 or . stands for.
- bscphil 6y agoWell, obviously. But presumably the ... is meant to imply that this is the summation of 1/(10^(x(x+3)/2)).
- landryl 6y agoActually I'd argue the example you provided is normal, as long as you authorise a particular encoding where every number n you're looking for is encoded as a string of n zeros. It's then trivial to see that every number you can think of is encoded in there, and therefore any data, piece of music or movie that ever existed. (I'm not sure we're allowed to fiddle with the encoding, but since we allow ourselves to represent a piece of music into a number, we're already talking about encoding anyway, so it doesn't seem like cheating to me...)
- sp332 6y agoOn the other hand, this allows for some incredible polyglot files, like some of the tricks with PoC||GtfO issues where the file is a readable PDF but also a game cartridge and also a zip file with the proof-of-concept code in the issue. And the front cover has the MD5 hash of the whole file printed on it... but that's another trick entirely!
- rvense 6y agoYeah, next time I need a CV it'll be a single-file Ruby web server and PDF that's also an archive of its own sources.
- toomanybeersies 6y agoI'm currently looking for a job as a Rails Developer. I might just do that. Probably won't send it to any recruiters, but it will be a funny anecdote for interviews.
- maddyboo 6y agoCan a PDF file contain a PDF file, and if so can that PDF file contain a PDF file?
- betatim 6y agoYes. Because the PDF standard specifies a mechanism that lets you "attach" files to a PDF :)
- brailsafe 6y agoI worked for PDFTron on their WebViewer product earlier this year, and primarily spent time implementing this feature in JS. Understanding the spec on this was tricky, because standard PDF viewers need to be able to uncompress the stuff you jam in there. It kind of blew my mind that you can literally jam any arbitrary file into a PDF.
- layoutIfNeeded 6y agoZIP files can: https://research.swtch.com/zip https://research.swtch.com/zip
- brailsafe 6y agoYes.
- folmar 6y agoMy stupid bank sends encrypted attachments as an encrypted PDF with HTML file attached.
- cgb223 6y agoMy favorite pdf fact is that the security flags for things like copy protection and passwords are on the viewer to implement so you can just turn them off and all the security is gone
- Drdrdrq 6y agoMy somewhat less favorite pdf fact is that if you do that, you are still breaking protection, legally speaking.
- maxerickson 6y agoSeems to be a reasonable analogy with trespass, where you are violating the law when you cross an invisible line. The need for marking the line varies considerably. And even places with strong roaming rights tend place limits on well marked land.
- smegger001 6y agoSo what if you open it in a postscript viewer instead of a PDF viewer? Because they are compatible formats except for some edge cases like security flags.
- mkl 6y agoPostscript and PDF are definitely not compatible formats. The drawing model is similar, but the structure and code are completely different.
- toomanybeersies 6y agoYou can also circumvent copy protection on PDFs by taking a screenshot, or taking a photo of the screen with your phone.
- r00fus 6y agoThis is not entirely true, you can encrypt PDFs [1] since v1.3 of the spec but the cypher is often so weak (RC4 until v1.6) they can be bruteforced in reasonable amounts of time. [1] https://www.pdflib.com/pdf-knowledge-base/pdf-password-security/encryption/ https://www.pdflib.com/pdf-knowledge-base/pdf-password-secur...
- nostoc 6y agoIt'll depend on the pdf reader you're using, but I'm pretty sure the PDF header needs to start in the first 1K of the file.
- mkl 6y agoSome readers won't need a header at all, I think. Near the end (usually!) of the file there's an index of objects (page data etc.) with byte offsets, which can point to anywhere in the file.
- alexott 6y agoYou can imagine the pain when you need to reliably detect PDF mime type on web proxy, or something like...
- agumonkey 6y agoI can never find the PDF hack talk where author explains all 100 ways to embed things in pdf or pdf into things
- wffurr 6y agoIt's hidden in a PDF in the digits of Pi.
- chuinard 6y agoI never understood that Google security blog post on how they could make 2 different PDFs with different content have the same SHA but now that you mention you can stuff bytes in a file unrelated to the PDF, it makes sense...
- mettamage 6y agoWhat's your favorite PDF feature that causes a brain meltdown? I've read a few comments on HN how PDF is, well, not developer-friendly. If people are interested in providing some more examples here, I'd be curious to know!
- amelius 6y agoAt least it's probably better than MS-Word's internal format ... (?)
- liversage 6y agoMicrosoft Word stores XML documents inside a zip archive. There is a detailed specification of the format available: https://docs.microsoft.com/en-us/openspecs/office_standards/ms-offstandlp/d5784a8b-7070-466b-befa-b7bf3724c6f0 https://docs.microsoft.com/en-us/openspecs/office_standards/...
- xaldir 6y agoI think he was talking about the classic .doc format which was a clusterfuck and not the open XML.
- akie 6y agoIf I remember correctly the XML format was just an XML-encoded version of the binary counterpart. Including all or most of the bugs and weird hacks.
- amaccuish 6y agowith the previous format being essentially a memory dump, i'd say that's progress
- alexott 6y agoThat’s correct - I worked with MS team that documented old formats, and they said that sometimes they don’t have people left who knew what specific struct was intended for - although that was mostly for people PowerPoint and Visio, excel and word was better documented
- Someone 6y agoOK, they added various ways of data compression, but PDF is, basically, a text-based format. As far as I know, any PDF can be losslessly converted to an equivalent PDF that can be edited in any text editor, even Notepad. And yes, you could fill in the forms there, too (if you were stubborn enough)
- roflc0ptic 6y agoIt sounds like you either know a lot more than me or a lot less than me. The PDFs I've dealt with don't store text as strings, they store it as individual characters. This left me having to write a heuristic based algorithm to group the characters into words, words into lines, lines into paragraphs, paragraphs into columns. Again, as far as I know, there are no heuristics good enough to get that right for all values of PDF.
- belval 6y agoHe knows a lot less than you probably because there is absolutely no requirements for PDFs to be in text format and most aren't. The "text" he is editing could render to completely different characters depending on how the PDF document was created. The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a").
- yjftsjthsd-h 6y ago> The default MacOS PDF printer will actually remap the font cmap making born-digital PDFs where the "text" is something else entirely (say "$" maps to "a"). What? Why!? I've heard of doing that as a form of DRM, but I can't imagine Darwin defaulting to doing that.
- belval 6y agoI never dug deeper into it, so I am not aware of why it does that or if it's a specific version or whatnot, but take a PDF from which you can extract the text (with pdftotext/pdfbox for example). Open it in the document viewer and "print" it to PDF. If you extract the text again it is not readable anymore. This wouldn't be an issue if it was a conscious choice, but when I parsed a lot of born-digital PDFs we ended up with a lot that were like that from various source. Try explaining that...
- gorgoiler 6y agoOnly Forward ...a wondeful novel along these lines. They only get shot and kidnapped though. Nothing so bad as PDFs.
- moultano 6y agoIt's kinda crazy that this is the format we've standardized on to carry all of the output of academia into the future.
- mjcohen 6y agoA lot of the input is LaTeX, so that's ok.
- MayeulC 6y agoAnd arxiv asks for the original latex source when submitting. Well, at least, pdf is probably better than printed paper for that purpose.
- larrybud 6y agoActually, it’s not. The archiving standard is PDF/A, which is (as I understand it) much more structured and standardized.
- schoolornot 6y agoI thought I found a nifty trick by using OpenOffice to create a form with a pre-filled value. I decoded the PDF using pdktk or one of the free tools, and then modified the value. Nope, that caused some kind of cascading/checksum error. Ended up just making the app generate HTML before calling wkhtmltopdf. The PDF spec is insane! But like all things what you get out of Word/OpenOffice is 100x more complex than if you wrote it yourself, which is indeed doable.
- phpdave11 6y agoIt's really not that difficult if you read and understand the PDF specification. As a learning exercise, I created a simple PDF generator library that creates ASCII PDF documents (you can open them in Notepad) and includes comments about what each drawing instruction does. https://github.com/phpdave11/davepdf https://github.com/phpdave11/davepdf
- recursive 6y agoI'm sure generating PDFs is much easier than reading them, such that "it just works" with any kind of PDF.
- phpdave11 6y agoReading them is also easy. I wrote a library that reads PDFs and imports page(s) from an existing PDF into a new PDF as a Form XObject. https://github.com/phpdave11/gofpdi https://github.com/phpdave11/gofpdi
- 1vuio0pswjnm7 6y agoDid you look at this: https://github.com/catseye/pdf.lua https://github.com/catseye/pdf.lua
- thehappypm 6y agoI am actually interested in doing pure JS pdf processing. All of the web interfaces for PDF processing are server side — which means it’s tough to process large files. The dream is a purely JavaScript solution that never leaves the local computer. I’ve got a few client-side success stories that do fairly significant image generation through the canvas. So far PDF seems reasonably manageable through manipulating the text, but it’s not the best format.
- anoopelias 6y ago> The dream is a purely JavaScript solution that never leaves the local computer. Mozilla's pdfjs[1] project is a pure HTML/JavaScript solution for PDF rendering. This is the same code that ships in Firefox browser as well. This is standalone, AFAIK, it doesn't talk to a mothership. [1]https://mozilla.github.io/pdf.js/ https://mozilla.github.io/pdf.js/
- thehappypm 6y agoIt’s not quite what I’m looking for—more of a viewer.
- emiliosic 6y agoYears ago, I worked on a project that required generating PDF invoices. Used the FPDF library and I was shocked how small the files are (including a properly sized logo) when generated compared to most other PDFs 'rendered' from word or print processors.
- saagarjha 6y agoI just had to do tech support for someone whose 3 MB PowerPoint of text and some shapes became a 70 MB PDF that they couldn’t send through email anymore :/
- ajsnigrutin 6y agoThe last time i wanted to do that, i noped the fuck out, generated latex, and pdflatex-ed the thing, to get the pdf (we're talking generic reports, text, table, graph, email to customer on a schedule).
- godelmachine 6y agoI didn’t get it. Are you implying that coding PDF is an onerous task?
- maushu 6y agoCheck out the specs file for PDF and you will understand. I will be extremely surprised if anyone (besides Adobe) has implemented 100% of it.
- saagarjha 6y agoI’d be surprised if Adobe has implemented 100% of it. With a format this complex, there’s bound to be discrepancies between the spec and the code they have.
- enriquto 6y agoReading PDF files is certainly a nightmare. But you can easily produce a valid and simple pdf file just by printf-ing whatever needs to be printf-ed. There's an ugly header and the rest is essentially your text.
- bjoli 6y agoI had a look through the XPS standard and had a similar feeling. I complained to a friend that had been involved in one of the bigger pdf libraries. He then made me compare it to his version of the pdf 2.0 (iirc) standard. That is truly nightmare material. Especially considering a non-trivial percentage of pdfs circulating are non-conformant and people still expect them to render...