4 ms·

Don't bookmark. Instead: Print to PDF. I have every interesting article or web reference I've ever read since 1997, sitting in a PDF file alongside tens of th

Don't bookmark. Instead: Print to PDF. I have every interesting article or web reference I've ever read since 1997, sitting in a PDF file alongside tens of thousands of others. It is so convenient to be able to "ls -l | grep subject" and find all the pages related to that subject, and then to mine the data out of the PDF's for further reference. Or, I just open the PDF and gain the knowledge again. This works very well and doesn't require an active Internet connection. Every year I spend a few weeks up in a mountain retreat, going through the collection and getting an idea for the variety of topics I've read about for the year. Its pretty neat to see, also, the changes in my interests over the years - and as well, its pretty wild to go back to the sites after time and realise I've got the only copy of the site - because its gone now. I'd prefer to see more tools for manipulating PDF's become mainstream in the future. I can't recommend highly enough the productivity I've gained by being able to organise things this way. Its like having my own private Internet of 80,000+ pages, tailored to all the things I'm interested in ..

lcampbell 7y ago
I've always thought of PDF as an opaque format. How do you search and/or browse your collection? Does the subject show up in grep [e: without being diligent with filenames]?
- lloydatkinson 7y ago
  I'd be interesting in knowing this too. It sounds like a good idea.
- mrspuratic 7y ago
  I (*nix user) use a script that basically does: pdftotext -layout -eol unix -nopgbrk $PDF | egrep ... Many PDFs have compressed content streams, plain text utilities only see metadata in that case. Cached, compressed text-only output is usually tiny, and can be zgrep-ed. pdfinfo shows document metadata (title, subject, keywords and more), but it's quite uncommon for these to be useful (Adobe and LᴬTᴇX-sourced PDFs tend to have this data). Both come with xpdf.
  - mxuribe 7y ago
    This great; thanks for sharing!
eitland 7y ago
That sounds brilliant, and yet I can't remember having heard or read this idea before! It kind of amazes me that you've been doing this for two decades already.
- fit2rule 7y ago
  I always wonder why nobody else does it, but its a very pleasing experience to have an archive like this - and it has proven very valuable in production crunches and peaceful moments of downtime, alike. Another thing it allows me to do is generalise my reading in real time - I know I can always come back to the subject of the PDF in a few days time, or whenever really, without needing to know where to find the details: I just grep my file tree, and mine the data that way. Key thing is, though, that when saving the PDF, I always check that it is named a decent subject, derived from the <title> tags .. if there isn't one, I add it myself. This is the only place I pay attention to 'tagging an article' - in naming the file of a poorly-named page, but if the page title fits the subject, I don't change a thing. And it means I have a huge set of data also in the filenames, not just the contents themselves .. Periodically, I use 'detox' on my PDFArchive/ directory, also .. this helps with consistent-naming for regexes, and so on. I keep getting the itch to put all this into a DB with proper indexing or whatever, but .. actually using plain ol' shell tools is turning out to be just as productive. Its a big set of data, but plain ol' files and pipes is proving to be all I really need to get the data mined ..
erikbye 7y ago
> Don't bookmark. Instead: Print to PDF. I "print" to markdown instead, even though pdfgrep sort of works I much prefer text files.
- pensatoio 7y ago
  That’s a great idea! How do you go about it?
  - erikbye 7y ago
    I use markdown-clipper for Firefox, it does an OK job but sometimes requires a bit of fixing up. It does not save images, so I have to do this manually; I use VSCode for Markdown, which has an extension for copy/pasting images to Markdown, it inserts a link and saves the image in a subfolder. A little friction, but if that's too much, then the article is not worth keeping. But yes, someone with the time on their hands could contribute to markdown-clipper, saving images locally.
- ScottFree 7y ago
  I'm also interested in how you go about this. Evernote and Notion have good clippers, but I'd rather not rely on them. I tried to use markdownifier[0] for a while, but images are hotlinked and will break if the site goes down. I should really just write my own clipper that does this, but I haven't found the time yet. [0]: http://heckyesmarkdown.com/ http://heckyesmarkdown.com/
mxuribe 7y ago
I've always disliked pdfs - they just seem too, i don't know, obscure or scrambly/messy, maybe? However, i have recently seriously considered beginning to archive instead of bookmarking. I would prefer archiving as plain text (for easier searching and compressing, etc.)...but enough content that i consume (and wish to archive) contains media (mostly imagery, not really much video), so i guess printing to pdf is more convenient for the "capture" phase. The one concern i have is storage...I mean how much storage would bloated pdfs have to use up i wonder (as opposed to, say, leaner plain text)?? Legitimate concern, not trying to be facetious. I've also heard there's some sort of web archive file format - .warc or something? If so, i hope that becomes an established thing - maybe bst of all worlds...at least for this archiving use-case.

lcampbell 7y ago

I've always thought of PDF as an opaque format. How do you search and/or browse your collection? Does the subject show up in grep [e: without being diligent with filenames]?

lloydatkinson 7y ago
I'd be interesting in knowing this too. It sounds like a good idea.
mrspuratic 7y ago
I (*nix user) use a script that basically does: pdftotext -layout -eol unix -nopgbrk $PDF | egrep ... Many PDFs have compressed content streams, plain text utilities only see metadata in that case. Cached, compressed text-only output is usually tiny, and can be zgrep-ed. pdfinfo shows document metadata (title, subject, keywords and more), but it's quite uncommon for these to be useful (Adobe and LᴬTᴇX-sourced PDFs tend to have this data). Both come with xpdf.
- mxuribe 7y ago
  This great; thanks for sharing!

eitland 7y ago

That sounds brilliant, and yet I can't remember having heard or read this idea before! It kind of amazes me that you've been doing this for two decades already.

fit2rule 7y ago
I always wonder why nobody else does it, but its a very pleasing experience to have an archive like this - and it has proven very valuable in production crunches and peaceful moments of downtime, alike. Another thing it allows me to do is generalise my reading in real time - I know I can always come back to the subject of the PDF in a few days time, or whenever really, without needing to know where to find the details: I just grep my file tree, and mine the data that way. Key thing is, though, that when saving the PDF, I always check that it is named a decent subject, derived from the <title> tags .. if there isn't one, I add it myself. This is the only place I pay attention to 'tagging an article' - in naming the file of a poorly-named page, but if the page title fits the subject, I don't change a thing. And it means I have a huge set of data also in the filenames, not just the contents themselves .. Periodically, I use 'detox' on my PDFArchive/ directory, also .. this helps with consistent-naming for regexes, and so on. I keep getting the itch to put all this into a DB with proper indexing or whatever, but .. actually using plain ol' shell tools is turning out to be just as productive. Its a big set of data, but plain ol' files and pipes is proving to be all I really need to get the data mined ..

erikbye 7y ago

> Don't bookmark. Instead: Print to PDF. I "print" to markdown instead, even though pdfgrep sort of works I much prefer text files.

pensatoio 7y ago
That’s a great idea! How do you go about it?
- erikbye 7y ago
  I use markdown-clipper for Firefox, it does an OK job but sometimes requires a bit of fixing up. It does not save images, so I have to do this manually; I use VSCode for Markdown, which has an extension for copy/pasting images to Markdown, it inserts a link and saves the image in a subfolder. A little friction, but if that's too much, then the article is not worth keeping. But yes, someone with the time on their hands could contribute to markdown-clipper, saving images locally.
ScottFree 7y ago
I'm also interested in how you go about this. Evernote and Notion have good clippers, but I'd rather not rely on them. I tried to use markdownifier[0] for a while, but images are hotlinked and will break if the site goes down. I should really just write my own clipper that does this, but I haven't found the time yet. [0]: http://heckyesmarkdown.com/ http://heckyesmarkdown.com/

mxuribe 7y ago

I've always disliked pdfs - they just seem too, i don't know, obscure or scrambly/messy, maybe? However, i have recently seriously considered beginning to archive instead of bookmarking. I would prefer archiving as plain text (for easier searching and compressing, etc.)...but enough content that i consume (and wish to archive) contains media (mostly imagery, not really much video), so i guess printing to pdf is more convenient for the "capture" phase. The one concern i have is storage...I mean how much storage would bloated pdfs have to use up i wonder (as opposed to, say, leaner plain text)?? Legitimate concern, not trying to be facetious. I've also heard there's some sort of web archive file format - .warc or something? If so, i hope that becomes an established thing - maybe bst of all worlds...at least for this archiving use-case.