4 ms·

My own little moment of fame: WWDC June 8, 2009, Keynote, 21 minutes and 22 seconds in: Bertrand Serlet says (about improvements to Preview): "We have added lot

My own little moment of fame: WWDC June 8, 2009, Keynote, 21 minutes and 22 seconds in: Bertrand Serlet says (about improvements to Preview): "We have added lots of little touches. The one I like...we have added a little bit of AI to actually infer the selection" My colleague and I (we co-developed the "little bit of AI") never saw this comment (and the applause) live: we were still lining up outside of Moscone West. As Apple employees we weren't allowed in until all the guests and paying attendees had entered. Still, we watched the video afterwards and got a huge kick out of the mention. BTW, the AI we added did not use neural nets or deep learning.

blowski 7y ago
That must be an amazing feeling! Is this a feature that's still in macOS today? Can I see it in action?
- herodotus 7y ago
  The feature is there. I assume our code is still used, but it may have been rewritten since I retired. You can use this as an example if you like: https://www.extension.purdue.edu/extmedia/FNR/FNR-269-W.pdf https://www.extension.purdue.edu/extmedia/FNR/FNR-269-W.pdf (which is random, but you need two columns in the PDF). Try select from column 1 near the bottom, across to column 2. PDFs are very diverse. Our algorithm does not always work, alas, but it is significantly better than what existed before. In fact, for some reason, it works well for this document on page 1 and 3 but not so well on page 2! But I no longer have access to the low level tools I used to use to figure out what went wrong.
  - amartya916 7y ago
    This is so neat. Like someone else mentioned already, this feature has been very useful for me countless times. In fact, when it doesn't work, I tend to blame the PDF itself :). Thanks a ton!
- cfitz 7y ago
  Yup, just open Preview for an image (possibly other documents as well e.g., PDF), go into “markup” mode, and begin to draw with the pen or pencil. Preview will try to “perfect” your drawn check mark, X, straight line, curved line, etc.
  - herodotus 7y ago
    That's a nice feature! But it was added after we left Apple. I cannot claim any credit for it.
  - coldtea 7y ago
    That's a different "smart" feature -- the one mentioned has to do with guessing where the selected text continues (when it goes on to a different column, text box, etc). PDFs are more "baked" layouts rather than high level semantic markup, so it's not easy to automatically determine this. The AI is probably a set of heuristics.
telegrammae 7y ago
Very interesting! Also a great reminder that AI is just a program that solves a problem in a certain way that may not need the most talked-about techniques, like deep learning.
fierarul 7y ago
Just to confirm, are you the person the changed the Preview PDF text selection in newer macOS versions? o_0
- herodotus 7y ago
  We created a low-level framework that was used by PDFKit (and hence by Preview) to change the way selection of text in PDF documents is done. We never worked on Preview itself. Since the feature is part of PDFKit, it is also available to third-party apps that use PDFKit.
jakobegger 7y ago
I remember that! It's such a great feature! Unfortunately it doesn't work correctly for the PDFs I get from my accountant. It's one of those features that are really obvious, but probably surprisingly hard to implement.
bigiain 7y ago
> BTW, the AI we added did not use neural nets or deep learning. So proper old-school AI then? Perl and regexes?
- herodotus 7y ago
  We took a rule-based approach, coupled with some heavy-duty statistics.
  - psadri 7y ago
    That’s “AI” alright.
    - psadri 7y ago
      I didn’t mean any disrespect. My comment was meant to be humorous.
      - bigiain 7y ago
        As was mine. (Have you _seen_ the internals of Perl's regex code? I swear it has more inexplicable random interconnectedness than most neural nets...)
Angostura 7y ago
I remember this being introduced, and thought ‘that’s cool’. I use that function in Preview most days so - thank you.
JKCalhoun 7y ago
I know you.
- amatecha 7y ago
  haha, I think I know who you are (though I don't _know you_) .. developer of Glider and Pararena, among others? :) If so, thank you for making such fun and memorable games!
  - JKCalhoun 7y ago
    You are correct.
blub 7y ago
This might explain a bug that's been bugging me for years: text selection in preview tends to fail when working with PDFs printed from Firefox (and others). The selection jumps from the cursor and includes too much text from below. If I select from below it jumps and selects from above. Not just words, but sometimes entire extra sentences or paragraphs. This is quite similar to iOS text selection in practice: mostly works, but when it doesn't you're screwed.
pschastain 7y ago
And then Apple went and borked Preview by rewriting it post-Mavericks. Used to keep a VM just for it. Not sure why they decided to "fix" something that wasn't broken but to this day Preview lacks a lot of functionality it once had. Just a bit of an aside. I'm still somewhat bitter about it :-p

blowski 7y ago

That must be an amazing feeling! Is this a feature that's still in macOS today? Can I see it in action?

herodotus 7y ago
The feature is there. I assume our code is still used, but it may have been rewritten since I retired. You can use this as an example if you like: https://www.extension.purdue.edu/extmedia/FNR/FNR-269-W.pdf https://www.extension.purdue.edu/extmedia/FNR/FNR-269-W.pdf (which is random, but you need two columns in the PDF). Try select from column 1 near the bottom, across to column 2. PDFs are very diverse. Our algorithm does not always work, alas, but it is significantly better than what existed before. In fact, for some reason, it works well for this document on page 1 and 3 but not so well on page 2! But I no longer have access to the low level tools I used to use to figure out what went wrong.
- amartya916 7y ago
  This is so neat. Like someone else mentioned already, this feature has been very useful for me countless times. In fact, when it doesn't work, I tend to blame the PDF itself :). Thanks a ton!
cfitz 7y ago
Yup, just open Preview for an image (possibly other documents as well e.g., PDF), go into “markup” mode, and begin to draw with the pen or pencil. Preview will try to “perfect” your drawn check mark, X, straight line, curved line, etc.
- herodotus 7y ago
  That's a nice feature! But it was added after we left Apple. I cannot claim any credit for it.
- coldtea 7y ago
  That's a different "smart" feature -- the one mentioned has to do with guessing where the selected text continues (when it goes on to a different column, text box, etc). PDFs are more "baked" layouts rather than high level semantic markup, so it's not easy to automatically determine this. The AI is probably a set of heuristics.

telegrammae 7y ago

Very interesting! Also a great reminder that AI is just a program that solves a problem in a certain way that may not need the most talked-about techniques, like deep learning.

fierarul 7y ago

Just to confirm, are you the person the changed the Preview PDF text selection in newer macOS versions? o_0

herodotus 7y ago
We created a low-level framework that was used by PDFKit (and hence by Preview) to change the way selection of text in PDF documents is done. We never worked on Preview itself. Since the feature is part of PDFKit, it is also available to third-party apps that use PDFKit.

jakobegger 7y ago

I remember that! It's such a great feature! Unfortunately it doesn't work correctly for the PDFs I get from my accountant. It's one of those features that are really obvious, but probably surprisingly hard to implement.

bigiain 7y ago

> BTW, the AI we added did not use neural nets or deep learning. So proper old-school AI then? Perl and regexes?

herodotus 7y ago
We took a rule-based approach, coupled with some heavy-duty statistics.
- psadri 7y ago
  That’s “AI” alright.
  - psadri 7y ago
    I didn’t mean any disrespect. My comment was meant to be humorous.
    - bigiain 7y ago
      As was mine. (Have you _seen_ the internals of Perl's regex code? I swear it has more inexplicable random interconnectedness than most neural nets...)

Angostura 7y ago

I remember this being introduced, and thought ‘that’s cool’. I use that function in Preview most days so - thank you.

JKCalhoun 7y ago

I know you.

amatecha 7y ago
haha, I think I know who you are (though I don't _know you_) .. developer of Glider and Pararena, among others? :) If so, thank you for making such fun and memorable games!
- JKCalhoun 7y ago
  You are correct.

blub 7y ago

This might explain a bug that's been bugging me for years: text selection in preview tends to fail when working with PDFs printed from Firefox (and others). The selection jumps from the cursor and includes too much text from below. If I select from below it jumps and selects from above. Not just words, but sometimes entire extra sentences or paragraphs. This is quite similar to iOS text selection in practice: mostly works, but when it doesn't you're screwed.

pschastain 7y ago

And then Apple went and borked Preview by rewriting it post-Mavericks. Used to keep a VM just for it. Not sure why they decided to "fix" something that wasn't broken but to this day Preview lacks a lot of functionality it once had. Just a bit of an aside. I'm still somewhat bitter about it :-p