3 ms·

The PDF file format is anachronous. When the format was created, computers only had a few KBs of RAM. Yet the format should be capable of editing documents wit

The PDF file format is anachronous. When the format was created, computers only had a few KBs of RAM. Yet the format should be capable of editing documents with thousand of pages. The format solves this issue by delegating the memory management to the user. Also, the file was made with the assumption it was suppose to be printed, not shared. It is easier to hide parts of the document instead of removing the data. A funny trivia. The PDF is suppose to be read from the end of file. That's why some documents need to fully downloaded before they can display the first page. Of course, nowadays most PDF are linearized and load, at least, the first page right away. Over the years specification got so complex it became very hard to implement a minimal editor, viewer, parser or generator. If the format was simpler, it would be possible to make "save as PDF" more accessible. I've other issues with the typesetting and the way color is handled (it has a printer first approach), but I think this post got too long already. I just want to point out the spec supports so many pointless features such drawing in 3D space, movies, audio, HTML support, etc. Finally, I don't understand why most people are against a revision on the PDF format despite clearly having very little knowledge on how it works. I think multi person edition of the same entry with some version control can be useful. By the way, the format kinda let many people edit the document at once, as long as they are not working in the same part.

kccqzy 7y ago
> When the format was created, computers only had a few KBs of RAM. Yet the format should be capable of editing documents with thousand of pages. The format solves this issue by delegating the memory management to the user. That's a good decision. Make the file format versatile and powerful. Don't constrain it by the limitations of contemporary hardware. > Also, the file was made with the assumption it was supposed to be printed, not shared. It is easier to hide parts of the document instead of removing the data. I agree it's made with the assumption of being printed, but that's part of the appeal—preserving visual fidelity of how the document looks. You can't send people a docx and expect them to see the exact same thing on their screen down to every detail. And no it's not difficult to remove data. If you know exactly what to remove, it is quite easy to remove things. To remove text, find the Tj or TJ operators, remove them and their arguments. To remove an image, find the Do operator (occasionally BI, ID, EI) and remove it. You might have to perform decompression before doing that. For images, you might have to run another pass to delete the referenced object. But all these are all very easily automated. > Over the years specification got so complex it became very hard to implement a minimal editor, viewer, parser or generator. If the format was simpler, it would be possible to make "save as PDF" more accessible. The reason "save to PDF" is difficult to implement from scratch is not because of its complicated specification. Indeed parsers are quite easy to write. The real reason "save to PDF" is difficult to implement is because PDF wants visual fidelity; that comes at the price of specifying where exactly text should be placed, all the way from how paragraphs are flowed to how kerning of the letter is to be handled. Most applications do not care about these details. Most developers hardly have any interest in understanding line-breaking algorithms or interpreting font files to produce the right offsets and glyphs (think ligatures). These things are, rightfully, way beyond the business domain of typical applications and beyond the knowledge of typical developers.
- maest 7y ago
  > If you know exactly what to remove, it is quite easy to remove things. I think you are severly underselling how difficult it is to do these things. 1. You need to be a specialist in how the PDF format works. 2. From experience, it's not trivial to have logic that correctly handles all possible formatting cases in a PDF. 2.
- piadodjanho 7y ago
  Given the constraints of the time when the format was conceived, the pdf format has a great design. I think, embedded designers should have a quick look on the PDF format to learn some tricks on how reduce unneeded memory accesses -- it basically implement a directory inside a file. With the entry removal example, I was trying to show the format was not meant not to be shared. I know it is possible to remove data in other ways, and that probably every modern editor removes the data correctly. But it was not how the format itself deals with it. Of course, hiding entries with the flag had others uses such only print only the pages you currently working on without having to rescan the whole file. I agree most devs don't have interest in learning how to do typesetting. But also, typesetting is quite complex by itself, specially when dealing with non western language. Luckily, projects such Harfbuzz (nowadays, hb is used even my emacs) makes it a lot easier. Like I said in my original post, the format is anachronous. I don't think the format is intrinsically bad, I just think the format is not right for your time. I think we can do better nowadays. PS: I've been thinking, it would be pretty cool to talk with the engineering team that worked on the first spec, and actually know what they were thinking back them and what they would change in it nowadays.
gpvos 7y ago
The main reason making a "save as PDF" is hard is the font handling. The rest is fairly straightforward.
tonyedgecombe 7y ago
Microsoft's XPS format solved many of the issues with PDF. Unfortunately it was too late and came from the wrong people so didn't succeed. As it's a zip file it also needs to be read from the end although it can be linearised as well.
- Mikhail_Edoshin 7y ago
  Yep, XPS is a very clean format. And it seems to be easy to add support for it because of this simplicity. E.g. I normally use Sumatra PDF viewer on PC and MuPDF on Android and they both view XPS just fine.
rusk 7y ago
> computers only had a few KBs of RAM. I don't think this is right. Postscript maybe ... but PDF in my experience came about in the 90s, when computers typically had between 4 and 16MB of RAM ...
- piadodjanho 7y ago
  You are probably right. But my point still stand, the format was made to deal with files much larger then the memory.
  - rusk 7y ago
    Probably? No, exactly.

kccqzy 7y ago

> When the format was created, computers only had a few KBs of RAM. Yet the format should be capable of editing documents with thousand of pages. The format solves this issue by delegating the memory management to the user. That's a good decision. Make the file format versatile and powerful. Don't constrain it by the limitations of contemporary hardware. > Also, the file was made with the assumption it was supposed to be printed, not shared. It is easier to hide parts of the document instead of removing the data. I agree it's made with the assumption of being printed, but that's part of the appeal—preserving visual fidelity of how the document looks. You can't send people a docx and expect them to see the exact same thing on their screen down to every detail. And no it's not difficult to remove data. If you know exactly what to remove, it is quite easy to remove things. To remove text, find the Tj or TJ operators, remove them and their arguments. To remove an image, find the Do operator (occasionally BI, ID, EI) and remove it. You might have to perform decompression before doing that. For images, you might have to run another pass to delete the referenced object. But all these are all very easily automated. > Over the years specification got so complex it became very hard to implement a minimal editor, viewer, parser or generator. If the format was simpler, it would be possible to make "save as PDF" more accessible. The reason "save to PDF" is difficult to implement from scratch is not because of its complicated specification. Indeed parsers are quite easy to write. The real reason "save to PDF" is difficult to implement is because PDF wants visual fidelity; that comes at the price of specifying where exactly text should be placed, all the way from how paragraphs are flowed to how kerning of the letter is to be handled. Most applications do not care about these details. Most developers hardly have any interest in understanding line-breaking algorithms or interpreting font files to produce the right offsets and glyphs (think ligatures). These things are, rightfully, way beyond the business domain of typical applications and beyond the knowledge of typical developers.

maest 7y ago
> If you know exactly what to remove, it is quite easy to remove things. I think you are severly underselling how difficult it is to do these things. 1. You need to be a specialist in how the PDF format works. 2. From experience, it's not trivial to have logic that correctly handles all possible formatting cases in a PDF. 2.
piadodjanho 7y ago
Given the constraints of the time when the format was conceived, the pdf format has a great design. I think, embedded designers should have a quick look on the PDF format to learn some tricks on how reduce unneeded memory accesses -- it basically implement a directory inside a file. With the entry removal example, I was trying to show the format was not meant not to be shared. I know it is possible to remove data in other ways, and that probably every modern editor removes the data correctly. But it was not how the format itself deals with it. Of course, hiding entries with the flag had others uses such only print only the pages you currently working on without having to rescan the whole file. I agree most devs don't have interest in learning how to do typesetting. But also, typesetting is quite complex by itself, specially when dealing with non western language. Luckily, projects such Harfbuzz (nowadays, hb is used even my emacs) makes it a lot easier. Like I said in my original post, the format is anachronous. I don't think the format is intrinsically bad, I just think the format is not right for your time. I think we can do better nowadays. PS: I've been thinking, it would be pretty cool to talk with the engineering team that worked on the first spec, and actually know what they were thinking back them and what they would change in it nowadays.

gpvos 7y ago

The main reason making a "save as PDF" is hard is the font handling. The rest is fairly straightforward.

tonyedgecombe 7y ago

Microsoft's XPS format solved many of the issues with PDF. Unfortunately it was too late and came from the wrong people so didn't succeed. As it's a zip file it also needs to be read from the end although it can be linearised as well.

Mikhail_Edoshin 7y ago
Yep, XPS is a very clean format. And it seems to be easy to add support for it because of this simplicity. E.g. I normally use Sumatra PDF viewer on PC and MuPDF on Android and they both view XPS just fine.

rusk 7y ago

> computers only had a few KBs of RAM. I don't think this is right. Postscript maybe ... but PDF in my experience came about in the 90s, when computers typically had between 4 and 16MB of RAM ...

piadodjanho 7y ago
You are probably right. But my point still stand, the format was made to deal with files much larger then the memory.
- rusk 7y ago
  Probably? No, exactly.