3 ms·

Each character needs to be individually placed for the best possible typesetting (e.g with microtype). Nevertheless, copying text from a PDF works in many case

Each character needs to be individually placed for the best possible typesetting (e.g with microtype). Nevertheless, copying text from a PDF works in many cases, so there has to be some support for that, no? Or is everything done in the reader?

aidos 9y ago
Text is a funny thing in PDF. Sometimes, there's real text broken into lines and then you're pretty good. Often it's a subset font where the codes used in the script don't correspond to their visual glyphs. When that happens there might be a unicode map that lets you know which internal char maps to which external char (used for copying text from the PDF by viewers). Sometimes that's missing and you can rebuild it from other encoding information attached to the font. Other times you can't get the relationship and all you're left with is the randomly selected codes for each character — at that point it's not too dissimilar from a simple Enigma machine problem I guess. I sometimes see documents in which the same font has been subset again and again, once for each word. If you have a unicode map for each one, that's fine, if not it's not going to be much fun. In that case every word is going to look like random characters in the PDF and those character-glyph relationships are going to change from word to word. Other times the glyphs are rendered as vector paths at write time and you're down to trying to find the character from the outline it the shape. I deal with this a lot and there are common patterns but normally each glyph will be broken into several bits itself, so you have to find which bits go with which glyphs before you even start. Does anyone have a technical paper to hand, we could crack it open and take a look for fun. EDIT If you're trying to reliably convert everything, in a way you're better off catering for the lowest common denominator so you can do it consistently. Here, that means assuming you don't have any raw text data to work with and you have to do everything with image recognition. Either way, it's a fun problem.

aidos 9y ago

Text is a funny thing in PDF. Sometimes, there's real text broken into lines and then you're pretty good. Often it's a subset font where the codes used in the script don't correspond to their visual glyphs. When that happens there might be a unicode map that lets you know which internal char maps to which external char (used for copying text from the PDF by viewers). Sometimes that's missing and you can rebuild it from other encoding information attached to the font. Other times you can't get the relationship and all you're left with is the randomly selected codes for each character — at that point it's not too dissimilar from a simple Enigma machine problem I guess. I sometimes see documents in which the same font has been subset again and again, once for each word. If you have a unicode map for each one, that's fine, if not it's not going to be much fun. In that case every word is going to look like random characters in the PDF and those character-glyph relationships are going to change from word to word. Other times the glyphs are rendered as vector paths at write time and you're down to trying to find the character from the outline it the shape. I deal with this a lot and there are common patterns but normally each glyph will be broken into several bits itself, so you have to find which bits go with which glyphs before you even start. Does anyone have a technical paper to hand, we could crack it open and take a look for fun. EDIT If you're trying to reliably convert everything, in a way you're better off catering for the lowest common denominator so you can do it consistently. Here, that means assuming you don't have any raw text data to work with and you have to do everything with image recognition. Either way, it's a fun problem.