8 ms·

This is why iPhone didn't initially ship with double-tap to zoom for PDF paragraphs (like it had for blocks on web pages). I know because I was assigned the fea

This is why iPhone didn't initially ship with double-tap to zoom for PDF paragraphs (like it had for blocks on web pages). I know because I was assigned the feature, and I went over to the PDF guy to ask how I would determine on an arbitrary PDF what was probably a "block" (paragraph), and I got a huge explanation on how hard it would be. I relayed this to my manager and the bug was punted. Edit: To add a little more color, given that none of us was (or at least certainly I wasn't) an expert on the PDF format, we had so far treated the bug like a bug of probably at-most moderate complexity (just have to read up on PDF and figure out what the base unit is or whatever). After discovering what this article talks about, it became evident that any solution we cobbled together in the time we had left would really just be signing up for an endless stream of it-doesn't-work-quite-right bugs. So, a feature that would become a bug emitter. I remember in particular considering one of the main use cases: scientific articles that are usually in two columns, AND also used justified text. A lot of times the spaces between words could be as large as the spaces between columns, so the statistical "grouping" of characters to try to identify the "macro rectangle" shape could get tricky without severely special-casing for this. All this being said, as the story should make clear, I put about one day of thought into this before the decision was made to avoid it for 1.0, so far all I know there are actually really good solutions to this. Even writing this now I am starting to think of fun ways to deal with this, but at the time, it was one of a huge list of things that needed to get done and had been underestimated in complexity.

Alex3917 7y ago
> I know because I was assigned the feature, and I went over to the PDF guy to ask how I would determine on an arbitrary PDF what was probably a "block" (paragraph), and I got a huge explanation on how hard it would be. The funny thing is that creating a universal algorithm to convert PDFs and/or HTML to plaintext is probably comparable in difficulty to building level 5 self-driving cars, and would accrue at least as much profit to any company that can solve it. But there are hundreds of billions of dollars going into self-driving cars, and like zero dollars going into this problem.
- jimmaswell 7y ago
  Rendering the document to an image and using OCR on it would bypass a lot of trouble trying to make sense of the source, wouldn't it?
  - Alex3917 7y ago
    Not really, it's just a different set of challenges. The original article sums it up well, in terms of a lack of text-order hints. I haven't really tried incorporating OCR approaches at all, but I suspect they could probably be used to detect hidden text. The basic issue imho is that NLP algorithms are very inaccurate even with perfect input. E.g. even with perfect input, they're maybe only 75% accurate. And even an a text-processing algorithm that's like 99.9% accurate will yield input to your NLP algorithms that's like 50% accurate, so any results will be mostly unusable.
    - tastyminerals 7y ago
      NLP algorithms are just fine. It is the combination of regexes, NLP and deep learning that allows you to achieve good extraction results. So, basically OCR / pdf parser -> jpeg/xml/json -> regexes + NLP / DL extractor.
- tolmasky 7y ago
  What are the groups that would benefit most from the PDF-to-HTML conversion? Who are the customers that would drive this profit? I tried to make those sentences not sound contentious but unfortunately they do, but I am genuinely curious about this space and who is feeling the lack of this technology most.
  - Alex3917 7y ago
    So I have a lot of experience with basically the same problem just from working on this: https://www.prettyfwd.com https://www.prettyfwd.com. As an example of the opportunity size just in the email domain, the amount of personal non-spam email sent every day is like 100x the total size of Wikipedia, but nothing is really done with any of this information because of this challenge. Basically applications are things like: - Better search engine results - Identifying experts within a company - Better machine translation - Finding accounting fraud - Automating legal processes For context, the reason why Facebook is the most successful social network is that they're able to turn behavioral residue into content. If you can get better at taking garbage data and repackaging it into something useful, it stands to reason that there are lots of other companies the size of Facebook that can be created.
calf 7y ago
I don't know if PDF has truly evolved from its desktop publishing origins, but it is a terrible format because it no longer contains the higher level source information that you would have in an InDesign or a LaTeX file. PDF/Postscript were meant to represent optical fidelity and thus are too low-level abstractions for a lot of end-user, word processing tasks (such as detecting layout features), and thus trying to reverse engineer the "design intent" from them feels like doing work that is unecessarily tedious. But that's the way it seems to be given the popularity of the format.
rbobby 7y ago
> double-tap to zoom Why wouldn't you just zoom with the center point being where the tap occurred?
- abiogenesis 7y ago
  That's probably what they did for v1.0 after they saw it is not that easy to zoom such that the whole paragraph always fits into the visible area.
- tolmasky 7y ago
  Where to center to is only one vector, the other is how much to zoom: ideally it’s such that the text block fits on the screen. But again, that requires knowing the bounds of the text block. Zooming by a constant wherever you tap is a much less useful feature for text (vs. a map for instance), but I think it’s what we defaulted to (can’t remember if it was that of just nothing).

Alex3917 7y ago

> I know because I was assigned the feature, and I went over to the PDF guy to ask how I would determine on an arbitrary PDF what was probably a "block" (paragraph), and I got a huge explanation on how hard it would be. The funny thing is that creating a universal algorithm to convert PDFs and/or HTML to plaintext is probably comparable in difficulty to building level 5 self-driving cars, and would accrue at least as much profit to any company that can solve it. But there are hundreds of billions of dollars going into self-driving cars, and like zero dollars going into this problem.

jimmaswell 7y ago
Rendering the document to an image and using OCR on it would bypass a lot of trouble trying to make sense of the source, wouldn't it?
- Alex3917 7y ago
  Not really, it's just a different set of challenges. The original article sums it up well, in terms of a lack of text-order hints. I haven't really tried incorporating OCR approaches at all, but I suspect they could probably be used to detect hidden text. The basic issue imho is that NLP algorithms are very inaccurate even with perfect input. E.g. even with perfect input, they're maybe only 75% accurate. And even an a text-processing algorithm that's like 99.9% accurate will yield input to your NLP algorithms that's like 50% accurate, so any results will be mostly unusable.
  - tastyminerals 7y ago
    NLP algorithms are just fine. It is the combination of regexes, NLP and deep learning that allows you to achieve good extraction results. So, basically OCR / pdf parser -> jpeg/xml/json -> regexes + NLP / DL extractor.
tolmasky 7y ago
What are the groups that would benefit most from the PDF-to-HTML conversion? Who are the customers that would drive this profit? I tried to make those sentences not sound contentious but unfortunately they do, but I am genuinely curious about this space and who is feeling the lack of this technology most.
- Alex3917 7y ago
  So I have a lot of experience with basically the same problem just from working on this: https://www.prettyfwd.com https://www.prettyfwd.com. As an example of the opportunity size just in the email domain, the amount of personal non-spam email sent every day is like 100x the total size of Wikipedia, but nothing is really done with any of this information because of this challenge. Basically applications are things like: - Better search engine results - Identifying experts within a company - Better machine translation - Finding accounting fraud - Automating legal processes For context, the reason why Facebook is the most successful social network is that they're able to turn behavioral residue into content. If you can get better at taking garbage data and repackaging it into something useful, it stands to reason that there are lots of other companies the size of Facebook that can be created.

calf 7y ago

I don't know if PDF has truly evolved from its desktop publishing origins, but it is a terrible format because it no longer contains the higher level source information that you would have in an InDesign or a LaTeX file. PDF/Postscript were meant to represent optical fidelity and thus are too low-level abstractions for a lot of end-user, word processing tasks (such as detecting layout features), and thus trying to reverse engineer the "design intent" from them feels like doing work that is unecessarily tedious. But that's the way it seems to be given the popularity of the format.

rbobby 7y ago

> double-tap to zoom Why wouldn't you just zoom with the center point being where the tap occurred?

abiogenesis 7y ago
That's probably what they did for v1.0 after they saw it is not that easy to zoom such that the whole paragraph always fits into the visible area.
tolmasky 7y ago
Where to center to is only one vector, the other is how much to zoom: ideally it’s such that the text block fits on the screen. But again, that requires knowing the bounds of the text block. Zooming by a constant wherever you tap is a much less useful feature for text (vs. a map for instance), but I think it’s what we defaulted to (can’t remember if it was that of just nothing).