11 ms·

This is still not as powerful as my one, simple trick to handle all bookmarks, ever: Print to PDF. I've been doing it since last century, and I have 10's of th

This is still not as powerful as my one, simple trick to handle all bookmarks, ever: Print to PDF. I've been doing it since last century, and I have 10's of thousands of PDF's of every single web page I've ever found interesting, sitting right there in a directory on my computer. Its indexable, searchable, grok'able, available off-line, allows me to harvest data without fuss, and gives me access to anything I can remember about the article, almost instantaneously. $ ls -l ~/PDFArchive/ | grep -i "bookmark" | grep -i "manage" | wc -l = I've seen 20 other bookmark management 'solution' articles in 20 years .. nothing beats print-to-PDF. Its just awesome.

imperialdrive 6y ago
Kudos - I'll be practicing this later today :)
danielecook 6y ago
What sorts of web pages do you do this for? What if the pdf version is difficult to read?
- fit2rule 6y ago
  Every web page I'll ever want to refer to, ever again. There are no good reasons for exceptions to this technique, imho. If the PDF version is difficult to read - which it rarely is, by the way - all I need to do is open the PDF and use the links in the page header to go visit the site again - all the details about the page are still there in the PDF, links are still clickable, etc. And if its really important, and I've taken the time, before moving to my PDF Archive, to verify that the site is not readable due to some layout inconsistency in the conversion to PDF (I do sometimes suspect this with the fancier laid out pages), I Print-to-PDF again after enabling Reader mode/view (Safari/Firefox): problem solved. But really, there are very few web pages that don't survive the PDF conversion. And anyway, I mostly pipe the .PDF output through something like pdf2text for further grok/grep'ing...
  - cortesoft 6y ago
    > There are no good reasons for exceptions to this technique, imho Dynamic web sites? A PDF of my bank website isn't going to help me much.
    - fit2rule 6y ago
      If the intention is to save data from your bank website, you're probably going to have to jump through hoops anyway, assuming your bank is doing its job. (Or just remember to use Reader mode first..) However, if the intention is to just save a link to the bank website for future reference, my technique still works since every page in the PDF produced contains a header with the URL - just like a normal bookmark.
- nojito 6y ago
  Most if not all web browsers have a reader mode which trims away nonsense.
cocktailpeanuts 6y ago
you should productize this idea. I would use it. It's a bit hassle to print to PDF every time, but if the barrier is low enough, it would be useful.
- fit2rule 6y ago
  There's nothing to productize. You just use CMD[,Control}-P, select "Print to PDF", save to a relevant folder, and off you go. If that's too many steps, use Automator or whatever the equivalent is on your OS to make a shorter hotkey. (I had a Hammerspoon script for this once, but reverted to just doing it manually, since my muscle memory on the keystrokes is sufficiently well trained that it supplants my desire to find the .lua files somewhere to pass to Hammerspoon..) The entire point is that there is absolutely no need for a third party to get involved in organising your web browsing history or remembering your bookmarks. Use the shell. Very few third-party services will be able to match the power of this tooling, for the reasons I gave above. My history = my data, for my own private purposes.
  - GuiA 6y ago
    That’s the software economy we’re in. Everyone’s thinking in terms of “productization” and “features” and “user journeys”. Only old grumpy hackers care about minimal, composable tools anymore. Sigh.
    - cocktailpeanuts 6y ago
      There are tons of people who would appreciate this feature and I only brought this up because these people will definitely appreciate something like this. Reading through your comment i can already imagine you sitting there probably thinking all these inferior "normies" who don't deserve to enjoy convenience in their lives because they don't know tech. Well, everybody on this site is a "hacker" yet not everybody has time and discipline to devote to doing this all the time. And I don't care for learning every single esoteric automation features on my computer. It's kind of weird giving a positive comment and getting all these negative snarky comments back. Lastly it's stupid to think "Productization" is some evil thing. Productization simply means take some process that's useful and make it easily accessible for other people. It doesn't even mean you sell it for money. If you don't care about improving other people's lives, that's fine, but don't be so condescending about people who genuinely appreciate the idea and just wanted to provide some positive feedback.
gregsadetsky 6y ago
It sounds like a really good idea (in addition to images being part of this single file PDF "archive" and thus won't go missing), but the PDFs being searchable depend on how the PDF is made, no? I printed to PDF this HN thread in Chrome (I assume that the PDF printing was done on the system level by OSX -- EDIT yes, from the file: "/Producer (macOS Version 10.15.2 $Build 19C57$ Quartz PDFContext)"), and none of the page's strings appear as ascii or utf-8 in the document. grep is unable to find any string in that file. Do you have a specific print to PDF setup? Or a PDF-aware grep..? EDIT: Seeing the command-line you're using, the search you do is over the files' names, correct? The PDF/(original web page) text content is not indexed, right? Just to make sure I understand correctly.
- fit2rule 6y ago
  I just use the PDF defaults from whatever browser I'm using at the time. Nothing special involved, just the defaults. I do use 'pdftotext' to do more fine-grained searching if I need to - but for the most part I find that a simple "ls -l | grep <search>" suffices, since this method preserves page title text too .. I did the same thing for this thread and had no issues with this command, whatsoever: $ pdftotext WorldBrain\'s\ Memex:\ Bookmarking\ for\ the\ power\ users\ of\ the\ web\ \|\ Hacker\ News.pdf - | grep -i "Print-to-pdf" Results: ".. nothing beats print-to-PDF. Its just awesome." "fancier laid out pages), I Print-to-PDF again after enabling Reader"
  - gregsadetsky 6y ago
    Got it, makes sense.
    - snazz 6y ago
      > EDIT: Seeing the command-line you're using, the search you do is over the files' names, correct? The PDF/(original web page) text content is not indexed, right? Just to make sure I understand correctly. pdftotext gets the actual text from the PDF. I don't do this, but I'm sure that you could automate the process of generating a text file for each PDF in a directory with pdftotext and then ripgrep the text files when it's time to search the contents. That would be doable with a makefile or a couple of shell scripts.
ropeladder 6y ago
It's too bad browsers don't have an easy way to print to browser-page-sized PDF. Standard 8.5x11/A4 paper sized PDFs of webpages tend to look pretty terrible. I used to use the Scrapbook plugin for Firefox but I realized for the most part just plaintext might be best. So I'm in the process of setting up a workflow that will save article in markdown in one click and sync between my phone and my computer.
- fit2rule 6y ago
  Reader View is the solution to the A4- problem, imho. But I honestly don't mind the rendering issue - this is just a reference repository, after all. If I really need the cleaner page, I either Reader-View it beforehand, or just open it up on the Web again - links are preserved in PDF.
  - jannes 6y ago
    Links are only preserved if you use Chrome's PDF export... This is not true for Firefox. (At least on Windows. I haven't tried Firefox on macOS.)
    - fit2rule 6y ago
      Safari for the win! ;) (Seems like a bug in Firefox to me.. maybe this should be a config option..)
- jannes 6y ago
  You could try the awesome SingleFile extension: https://github.com/gildas-lormeau/SingleFile https://github.com/gildas-lormeau/SingleFile It might be a good compromise between PDF and plain text. It's pretty nice because it essentially serialises a snapshot of the current DOM tree, so it works with all kinds of JS-generated pages. The files should be relatively grep-able, because it's normal HTML. Of course, you might want to strip HTML tags for more sophisticated searching.
  - ropeladder 6y ago
jannes 6y ago
Do you use a Chromium-based browser? Chrome/ium's Print-To-PDF uses quite a different (better) method for generating the PDF compared to the OS-level Print-To-PDF. The OS-level PDF converter can lose a lot of information. Especially hyperlinks are not present in the PDF when it's generated through a print driver. Unfortunately, this is one of the few times when it sucks to be a Firefox user, because it doesn't have a builtin Print-To-PDF.
- fit2rule 6y ago
  I use Safari and Firefox mostly, and haven't yet run into any of the issues you bring up - pdftotext gives me full text search with ease. All the links still work, PDF's are easy to read (assuming the page doesn't do weird layout tricks), and for the worst case, I at least can search my "bookmark" PDF archive and go back to the original live web page if needed.
  - jannes 6y ago
    Looks like Apple's implementation is a bit better then. I just tried on Windows and I definitely don't get any clickable links when printed with the OS print-to-pdf driver. (I do get them with Chrome's save-to-pdf feature which circumvents "printing" and generates the PDF itself.)
    - fit2rule 6y ago
      Good to know for future discussions about this technique. I wonder if there is a way to make Chrome on Windows behave better in this regard .. seems quite shortsighted to me that they'd remove the links, although maybe a security boffin has made the case for it. Either way, haven't used Windows in decades, so its a non-issue, but it is interesting to note that this isn't something I'd be doing if I did switch.
ramraj07 6y ago
This is amazingly simple, but what about in mobile? There's pdf printing here too but it's not as simple as command P! Any ideas?
- fit2rule 6y ago
  If I need to save a page I've read on mobile, I mail myself the link to my desktop and print it there, where the PDF Archive lives. Its muscle memory at this point. Still, would be nice if the Browser vendors would cotton on to how powerful this is, and make the whole thing a bit more seamless for the mobile/desktop bridge, or just make Print-to-PDF work more smoothly for this case on mobile. Either way, I also have a list of every mail I've ever sent myself containing a URL from mobile, which is handy in and of itself at times, hehe ..
  - WanderPanda 6y ago
    On iOS you can easily create a shortcut for the share-sheet to creat pdf prints fyi
    - fit2rule 6y ago
      Yeah, I should probably set that up some time, but I've become reliant on my muscle memory for my forwarding-to-desktop flow ..
  - rakoo 6y ago
    The advantage of doing it on the desktop is that you get to print the desktop version of the site. There probably is an avenue to automate printing from your desktop as soon as it receives the email; who knows, maybe even The inconvenient of your way is that your bookmarks are only available on the Desktop, from your mobile all you can have is the list of URLs you sent yourself. It's still probably better than having some service running all the time you want to query though. Your system is surprisingly "low-tech" and sounds extremely interesting, I'm tempted to start doing it as well. Do you have any kind of organization or viewer for your stack of PDFs, or are they just files amassed on your hard drive ?
pvg 6y ago
nothing beats print-to-PDF What's the advantage over browser's built-in 'save entire page' option? Print to PDF loses formatting and obscures the URL you got the thing from.
- fit2rule 6y ago
  $ pdftotext | grep sometext ^^ first advantage Also: a) Formatting is not lost, it just changes to fit the default paper size I've got selected (A4) but doesn't really make much difference, since its a snapshot, and b) URL is right there in the Header of the PDF, and is clickable, so no - not really an issue. This archive also functions as a bookmark collection as well as an offline copy for future reference .. (Disclaimer: may be that your browser is borking the PDF's. Not the case with Safari, anyway, but ymmv..)
  - pvg 6y ago
    You can grep the saved archives and they often save working copies of local interactive content in a way PDF doesn't. Internal structure and annotation is also preserved. I'm not sure I understand the formatting comment, you seem to be saying formatting is not lost and supporting that with an example of how formatting is lost. Don't get me wrong, it should definitely be easier to save, index and otherwise manipulate web pages. But out of the the trivial methods, 'print to PDF' is one of the poorer methods.
    - fit2rule 6y ago
      It depends on the site - but I haven't found 'lost formatting' to be an issue at all - since, when I want to do a granular search I'm using 'pdftotext' to search on plaintext, and when I find a PDF of interest, I open it and can go directly back to the web page from which it was printed by way of the footer/header which contains a clickable URL. Most of the time though, the formatting isn't an issue. It depends on the site though - some authors produce stuff that doesn't look good as PDF, even if the content is still there. That doesn't bug me much.
      - 6y ago
filoleg 6y ago
I would be careful with using this method and check the generated PDF versions with your eyes before writing them off as "all is good, it is archived now". I recently got bitten by that, when I was trying to print out some page in Chrome, and it was rendering as a bunch of white space surrounded by some elements from the page, but without any actual content I cared about. Turns out, my situation isn't that uncommon for pages that are heavily JS-dependent Note: I am not saying JS=bad. This has nothing to do with JS itself and everything to do with how JS is used to generate/render the page. A lot of pages just don't bother with doing it the right way that doesn't screw up generated PDFs.
- fit2rule 6y ago
  I've since learned in this thread that Chrome and Firefox are not as good as Safari for this technique - it hasn't impacted me much since I only use Chrome/Firefox for development, mostly. And although I do occasionally check the produced PDF's, the layout doesn't matter to me at all since I use a cmd-line grep or combination of 'pdftotext' to find the page, open the PDF, and click the link to go to the original web page if I need to .. haven't found a single dud PDF in the collection in a randomised sampling, but then again in 20,000+ files, there's bound to be one that didn't make it through the rendering pipeline, but so far, hasn't been an issue.
  - filoleg 6y ago
    That clarifies it a lot, thanks for explaining. I actually remember having to eventually go through Safari to print out that one page I mentioned above that was giving me issues in Chrome, so that makes a lot of sense. Glad to find out it wasn't just me somehow being lucky with Safari, and that it is actually a known thing.
  - deleted 6y ago
    [deleted]
wazoox 6y ago
I discovered Zotero for this use. I don't have any use of its bibliographical abilities, but it stores web pages and PDF articles fine, and is searchable, etc.
- nishparadox 6y ago
  True Zotero is a neat tool, especially for files and web pages. I've been using it to read research papers mostly.
- igravious 6y ago
  Yes, because of all the metadata you can preserve with the save. Also, frequently what yo want to save is a link to a book or an article (or a Wikipedia article, say) and Zotero recognizes many of these formats and saves them correctly, it's toolbar icon even changes to let you know. It's a fantastic tool. It's got cloud storage, and it has an API (which I've used -- it's super easy).
throwawaysea 6y ago
Are you manually printing each page to PDF? I would love to have an automated way to do this but haven't figured out how to deal with logging into subscription based site and all that. There is also some degree of messiness even with printing to PDF. For example let's say I want to save an HN or Reddit discussion along with the comments - I would need to make sure I capture all the comments that overflow to "More" on HN or are behind a "load more comments" link on Reddit. Is there any elegant way to traverse all that and capture it?
- nikisweeting 6y ago
  ArchiveBox.io saves to PDF and to screenshot and WARC, and HTML to avoid issues that PDF alone has, but the feature that allows archiving sites behind a login isn't completely finished yet.
- fit2rule 6y ago
  Yes, manually, but as I stated elsewhere: its muscle memory and smooth as an operation for me locally anyway now. I often go through the archive, find the HN comment PDF's I've created, and then automatically update them to get whatever new comments have occurred in the meantime. Haven't figured out how to navigate to the 'next' comment page automatically though - some pages detect they're being printed and use a print-friendly layout, though, which is nice .. would be cool to see more of that.
andrepd 6y ago
How is this "not as powerful"? What is this tool lacking that save to pdf provides? I can see at least one way it is vastly inferior, in that you break formatting by converting to a horrendous paper page-based format.
- fit2rule 6y ago
  Off-line mode, my data is my data, and I can process the data freely and easily using my own local tools without getting anyone else' CPU involved. And the formatting issue isn't really that big of a deal, if I'm honest. The formatting did its work in the initial contact of the web page - beyond that, to me anyway, its superfluous to the later task of finding the reference again. pdftotext don't care about the formatting, either.
  - andrepd 6y ago
    >Off-line mode, my data is my data, and I can process the data freely and easily using my own local tools without getting anyone else' CPU involved. Open the page and you will probably realise that unless you want to pay for sync, this extention works offline.
    - fit2rule 6y ago
      I don't want to pay for sync, because print-to-pdf is my sync, and is free.
alexmarkelenkov 6y ago
Tip for a better use of browser's print to PDF feature: use an ad blocker (uBlock Origin in Chrome works fine) to remove noisy html elements, ads and clutter. I basically cut everything but text, even navigation elements. And this btw is where Chrome comes handy for (e.g.) Firefox users: you can browse and print separately. You always have a ready-to-print setup in Chrome (or else) while browsing somewhere else. I haven't yet looked into another promising feature though, which is (if I'm not mistaken) the availability of a CLI for this . Would be very interested to automate the process, but am concerned about whether the element blocking will work. Ideally I'd like to be able to save printed pages either automatically or via bulk selection (within my browser history) to Evernote or anything like (it seems to be the best solution for me, as it has a great indexing, fuzzy search and relevance, plus the fact that the storage is not limited unless I start to upload more than 10 GB per month which is unlikely). Anyway, would be glad to hear if anyone came up with a similar or a better solution.
bharani_m 6y ago
I am also a big fan of Print to PDF. I've actually built a simple bookmarking service [0] that does just this. EmailThis extracts meaningful content from web pages and sends it to your email inbox. You can also tell it to save a PDF copy of each page, in which case the PDF is sent as an attachment. Print-to-Pdf is done using Headless Chrome (so it works exactly like doing a Ctrl-P). I find that the Print to PDF works best because it gives you a copy of the web page even if the original one disappears. Also, none of the content extraction services (mine included) work in 100% of the cases. Sometimes, they might incorrectly remove images and other meaningful content. So in such cases, having a full PDF snapshot is quite handy. Let me know what you guys think. [0] https://www.emailthis.me https://www.emailthis.me
- fit2rule 6y ago
  Nice way to find out what everyone is reading while gathering addresses of smart people. ;) > Sometimes, they might incorrectly remove images and other meaningful content. So in such cases, having a full PDF snapshot is quite handy. Also interesting is that the context is preserved locally across visits to the site - over 10 years, I have gathered a pretty interesting view of some of the various A/B changes that have gone on, on my favourite 'daily visit' sites .. And, it is often very revealing of my own habits. This highlights the privacy-factor of having a local-file based bookmark/ontology system a little more in my favour.
Galaco 6y ago
To be a little pedantic, while this is a fantastic idea, is it really bookmarking? What you’ve done instead is compiled a personal digital library; akin to a Kindle
- fit2rule 6y ago
  Inasmuch as every PDF still has the URL to the original page, for my uses - I'd say yes, it is bookmarking. Its not like I'd cut the spine off every book in my library and create an index out of the covers ..
  - fauigerzigerk 6y ago
    Apparently only Firefox saves the source URL. In PDFs saved from Safari or Chrome on macOS I can't find the URL anywhere. Maybe I'm missing something.
    - fit2rule 6y ago
      Interesting. For me, its right there on the bottom-left of the footer, always. /tmpᐅ pdftotext Add\ Comment\ -\ Hacker\ News.pdf - |grep http https://news.ycombinator.com/reply?id=23232588&goto=threads%3Fid%3Dfit2rule%2323232588 https://news.ycombinator.com/reply?id=23232588&goto=threads%...
      - fauigerzigerk 6y ago
        Weird. It's not in there when I store it.
        fit2rule 6y ago
        Lets compare versions just so I know this feature isn't going away any time soon. Safari 13.1 here, MacOS 10.14.6...
        cilefen 6y ago
        Try checking "Print headers and footers" in the print dialog.
dependenttypes 6y ago
Wouldn't being able to print to epub be better? epubs are just zipped html files after all so the conversion is more direct and you don't lose any info.
- fit2rule 6y ago
  PDF just has better tooling for search, which supplants my need for proper formatting since by the point I'm searching for things in my history its the content that matters to me anyway, exclusive of the original formatting.
phiresky 6y ago
> Yeah, my computer is fast enough that I can just do "find . -name '*.pdf' -exec pdftotext {} \; | grep -i someSearchTerm" and come back later. You may be interested in my ripgrep-all [1] tool, it should allow you to search those tens of thousands of PDFs in under a second (with hot cache) [1] https://phiresky.github.io/blog/2019/rga--ripgrep-for-zip-targz-docx-odt-epub-jpg/ https://phiresky.github.io/blog/2019/rga--ripgrep-for-zip-ta...
- tomcooks 6y ago
  Really great tool, if there is a way to promote it to the main pacman archives, instead of having to use AUR, let me know where I can vote
- fit2rule 6y ago
  Thanks for that - yeah, silversearch-ag also does a pretty good job.. Its really not a big deal to search so many docs these days, either ..
validuser 6y ago
Many sites have horribly broken/nonexistent print styling. Any tips on how to get good prints matching what you see in the browser?
aiilns 6y ago
I think it all depends on what you are using bookmarks for. I bookmark sites I want to check out again in the near future, but searching in my bookmarks/tagging as memex does has never been an issue for me. If it is something worth searching its text, then it is something worth saving offline, reading and annotating. I use Polar [1] for most and wallabag [2] for its .epub converting ability - especially if there is mainly text that interests me and a lot of it, so I can read it on my ereader. As soon as Polar manages .epubs I shall import all my .epub articles into it. :) [1] https://getpolarized.io/ https://getpolarized.io/ [2] https://wallabag.org/en https://wallabag.org/en
- fit2rule 6y ago
  Print-to-PDF, put the important sites in a Folder on your Desktop .. The URL for every single site is in the PDF.
deleted 6y ago
[deleted]
gildas 6y ago
> nothing beats print-to-PDF. Its just awesome. HTML largely beats PDF for this use case. And if you want something that produces a file in which you can easily extract resources from the saved page, see [1]. There is an option to automatically save the pages you add to your bookmark. There's also an option to make the text of the files indexable without unzipping them. [1] https://github.com/gildas-lormeau/SingleFileZ https://github.com/gildas-lormeau/SingleFileZ
detaro 6y ago
I really hate this kind of response. No, your thing is not strictly better. It's cool, it might be better for you and many others, but it doesn't even do the same thing (archive all visited pages)!
- Abishek_Muthian 6y ago
  True, there are different approaches to solve this problem according to the individual preferences. A user recently said, X would use 'Calendar to remind about bookmarks' in a discussion about this problem[1]. [1]https://needgap.com/problems/57-i-forget-my-web-bookmarks-quite-often.-internet-organizing https://needgap.com/problems/57-i-forget-my-web-bookmarks-qu...
- fit2rule 6y ago
  I really hate this kind of whinging response. This archives every page I'm ever interested in, just fine. Links are preserved just fine, all the data that got me interested in the web page in the first place are just there on my disk, easily accessible any time of day without requiring any further accounts. It is better since it doesn't require any involvement of a third party, is always accessible to me no matter the state of the Internet, and gives me absolute control over all of the data, which I can mine using whatever toolset I want. In fact, I get more data out of this method than the service described in the article.
nuka_coffee 6y ago
Based on your comment I stumbled upon this tool: https://wkhtmltopdf.org/ https://wkhtmltopdf.org/ It preserves hyperlinks and styling. Neat.
choicenotchance 6y ago
And I've been saving them as epub for the last 7.

imperialdrive 6y ago

Kudos - I'll be practicing this later today :)

danielecook 6y ago

What sorts of web pages do you do this for? What if the pdf version is difficult to read?

fit2rule 6y ago
Every web page I'll ever want to refer to, ever again. There are no good reasons for exceptions to this technique, imho. If the PDF version is difficult to read - which it rarely is, by the way - all I need to do is open the PDF and use the links in the page header to go visit the site again - all the details about the page are still there in the PDF, links are still clickable, etc. And if its really important, and I've taken the time, before moving to my PDF Archive, to verify that the site is not readable due to some layout inconsistency in the conversion to PDF (I do sometimes suspect this with the fancier laid out pages), I Print-to-PDF again after enabling Reader mode/view (Safari/Firefox): problem solved. But really, there are very few web pages that don't survive the PDF conversion. And anyway, I mostly pipe the .PDF output through something like pdf2text for further grok/grep'ing...
- cortesoft 6y ago
  > There are no good reasons for exceptions to this technique, imho Dynamic web sites? A PDF of my bank website isn't going to help me much.
  - fit2rule 6y ago
    If the intention is to save data from your bank website, you're probably going to have to jump through hoops anyway, assuming your bank is doing its job. (Or just remember to use Reader mode first..) However, if the intention is to just save a link to the bank website for future reference, my technique still works since every page in the PDF produced contains a header with the URL - just like a normal bookmark.
nojito 6y ago
Most if not all web browsers have a reader mode which trims away nonsense.

cocktailpeanuts 6y ago

you should productize this idea. I would use it. It's a bit hassle to print to PDF every time, but if the barrier is low enough, it would be useful.

fit2rule 6y ago
There's nothing to productize. You just use CMD[,Control}-P, select "Print to PDF", save to a relevant folder, and off you go. If that's too many steps, use Automator or whatever the equivalent is on your OS to make a shorter hotkey. (I had a Hammerspoon script for this once, but reverted to just doing it manually, since my muscle memory on the keystrokes is sufficiently well trained that it supplants my desire to find the .lua files somewhere to pass to Hammerspoon..) The entire point is that there is absolutely no need for a third party to get involved in organising your web browsing history or remembering your bookmarks. Use the shell. Very few third-party services will be able to match the power of this tooling, for the reasons I gave above. My history = my data, for my own private purposes.
- GuiA 6y ago
  That’s the software economy we’re in. Everyone’s thinking in terms of “productization” and “features” and “user journeys”. Only old grumpy hackers care about minimal, composable tools anymore. Sigh.
  - cocktailpeanuts 6y ago
    There are tons of people who would appreciate this feature and I only brought this up because these people will definitely appreciate something like this. Reading through your comment i can already imagine you sitting there probably thinking all these inferior "normies" who don't deserve to enjoy convenience in their lives because they don't know tech. Well, everybody on this site is a "hacker" yet not everybody has time and discipline to devote to doing this all the time. And I don't care for learning every single esoteric automation features on my computer. It's kind of weird giving a positive comment and getting all these negative snarky comments back. Lastly it's stupid to think "Productization" is some evil thing. Productization simply means take some process that's useful and make it easily accessible for other people. It doesn't even mean you sell it for money. If you don't care about improving other people's lives, that's fine, but don't be so condescending about people who genuinely appreciate the idea and just wanted to provide some positive feedback.

gregsadetsky 6y ago

It sounds like a really good idea (in addition to images being part of this single file PDF "archive" and thus won't go missing), but the PDFs being searchable depend on how the PDF is made, no? I printed to PDF this HN thread in Chrome (I assume that the PDF printing was done on the system level by OSX -- EDIT yes, from the file: "/Producer (macOS Version 10.15.2 $Build 19C57$ Quartz PDFContext)"), and none of the page's strings appear as ascii or utf-8 in the document. grep is unable to find any string in that file. Do you have a specific print to PDF setup? Or a PDF-aware grep..? EDIT: Seeing the command-line you're using, the search you do is over the files' names, correct? The PDF/(original web page) text content is not indexed, right? Just to make sure I understand correctly.

fit2rule 6y ago
I just use the PDF defaults from whatever browser I'm using at the time. Nothing special involved, just the defaults. I do use 'pdftotext' to do more fine-grained searching if I need to - but for the most part I find that a simple "ls -l | grep <search>" suffices, since this method preserves page title text too .. I did the same thing for this thread and had no issues with this command, whatsoever: $ pdftotext WorldBrain\'s\ Memex:\ Bookmarking\ for\ the\ power\ users\ of\ the\ web\ \|\ Hacker\ News.pdf - | grep -i "Print-to-pdf" Results: ".. nothing beats print-to-PDF. Its just awesome." "fancier laid out pages), I Print-to-PDF again after enabling Reader"
- gregsadetsky 6y ago
  Got it, makes sense.
  - snazz 6y ago
    > EDIT: Seeing the command-line you're using, the search you do is over the files' names, correct? The PDF/(original web page) text content is not indexed, right? Just to make sure I understand correctly. pdftotext gets the actual text from the PDF. I don't do this, but I'm sure that you could automate the process of generating a text file for each PDF in a directory with pdftotext and then ripgrep the text files when it's time to search the contents. That would be doable with a makefile or a couple of shell scripts.

ropeladder 6y ago

It's too bad browsers don't have an easy way to print to browser-page-sized PDF. Standard 8.5x11/A4 paper sized PDFs of webpages tend to look pretty terrible. I used to use the Scrapbook plugin for Firefox but I realized for the most part just plaintext might be best. So I'm in the process of setting up a workflow that will save article in markdown in one click and sync between my phone and my computer.

fit2rule 6y ago
Reader View is the solution to the A4- problem, imho. But I honestly don't mind the rendering issue - this is just a reference repository, after all. If I really need the cleaner page, I either Reader-View it beforehand, or just open it up on the Web again - links are preserved in PDF.
- jannes 6y ago
  Links are only preserved if you use Chrome's PDF export... This is not true for Firefox. (At least on Windows. I haven't tried Firefox on macOS.)
  - fit2rule 6y ago
    Safari for the win! ;) (Seems like a bug in Firefox to me.. maybe this should be a config option..)
jannes 6y ago
You could try the awesome SingleFile extension: https://github.com/gildas-lormeau/SingleFile https://github.com/gildas-lormeau/SingleFile It might be a good compromise between PDF and plain text. It's pretty nice because it essentially serialises a snapshot of the current DOM tree, so it works with all kinds of JS-generated pages. The files should be relatively grep-able, because it's normal HTML. Of course, you might want to strip HTML tags for more sophisticated searching.
- ropeladder 6y ago

jannes 6y ago

Do you use a Chromium-based browser? Chrome/ium's Print-To-PDF uses quite a different (better) method for generating the PDF compared to the OS-level Print-To-PDF. The OS-level PDF converter can lose a lot of information. Especially hyperlinks are not present in the PDF when it's generated through a print driver. Unfortunately, this is one of the few times when it sucks to be a Firefox user, because it doesn't have a builtin Print-To-PDF.

fit2rule 6y ago
I use Safari and Firefox mostly, and haven't yet run into any of the issues you bring up - pdftotext gives me full text search with ease. All the links still work, PDF's are easy to read (assuming the page doesn't do weird layout tricks), and for the worst case, I at least can search my "bookmark" PDF archive and go back to the original live web page if needed.
- jannes 6y ago
  Looks like Apple's implementation is a bit better then. I just tried on Windows and I definitely don't get any clickable links when printed with the OS print-to-pdf driver. (I do get them with Chrome's save-to-pdf feature which circumvents "printing" and generates the PDF itself.)
  - fit2rule 6y ago
    Good to know for future discussions about this technique. I wonder if there is a way to make Chrome on Windows behave better in this regard .. seems quite shortsighted to me that they'd remove the links, although maybe a security boffin has made the case for it. Either way, haven't used Windows in decades, so its a non-issue, but it is interesting to note that this isn't something I'd be doing if I did switch.

ramraj07 6y ago

This is amazingly simple, but what about in mobile? There's pdf printing here too but it's not as simple as command P! Any ideas?

fit2rule 6y ago
If I need to save a page I've read on mobile, I mail myself the link to my desktop and print it there, where the PDF Archive lives. Its muscle memory at this point. Still, would be nice if the Browser vendors would cotton on to how powerful this is, and make the whole thing a bit more seamless for the mobile/desktop bridge, or just make Print-to-PDF work more smoothly for this case on mobile. Either way, I also have a list of every mail I've ever sent myself containing a URL from mobile, which is handy in and of itself at times, hehe ..
- WanderPanda 6y ago
  On iOS you can easily create a shortcut for the share-sheet to creat pdf prints fyi
  - fit2rule 6y ago
    Yeah, I should probably set that up some time, but I've become reliant on my muscle memory for my forwarding-to-desktop flow ..
- rakoo 6y ago
  The advantage of doing it on the desktop is that you get to print the desktop version of the site. There probably is an avenue to automate printing from your desktop as soon as it receives the email; who knows, maybe even The inconvenient of your way is that your bookmarks are only available on the Desktop, from your mobile all you can have is the list of URLs you sent yourself. It's still probably better than having some service running all the time you want to query though. Your system is surprisingly "low-tech" and sounds extremely interesting, I'm tempted to start doing it as well. Do you have any kind of organization or viewer for your stack of PDFs, or are they just files amassed on your hard drive ?

pvg 6y ago

nothing beats print-to-PDF What's the advantage over browser's built-in 'save entire page' option? Print to PDF loses formatting and obscures the URL you got the thing from.

fit2rule 6y ago
$ pdftotext | grep sometext ^^ first advantage Also: a) Formatting is not lost, it just changes to fit the default paper size I've got selected (A4) but doesn't really make much difference, since its a snapshot, and b) URL is right there in the Header of the PDF, and is clickable, so no - not really an issue. This archive also functions as a bookmark collection as well as an offline copy for future reference .. (Disclaimer: may be that your browser is borking the PDF's. Not the case with Safari, anyway, but ymmv..)
- pvg 6y ago
  You can grep the saved archives and they often save working copies of local interactive content in a way PDF doesn't. Internal structure and annotation is also preserved. I'm not sure I understand the formatting comment, you seem to be saying formatting is not lost and supporting that with an example of how formatting is lost. Don't get me wrong, it should definitely be easier to save, index and otherwise manipulate web pages. But out of the the trivial methods, 'print to PDF' is one of the poorer methods.
  - fit2rule 6y ago
    It depends on the site - but I haven't found 'lost formatting' to be an issue at all - since, when I want to do a granular search I'm using 'pdftotext' to search on plaintext, and when I find a PDF of interest, I open it and can go directly back to the web page from which it was printed by way of the footer/header which contains a clickable URL. Most of the time though, the formatting isn't an issue. It depends on the site though - some authors produce stuff that doesn't look good as PDF, even if the content is still there. That doesn't bug me much.
    - 6y ago

filoleg 6y ago

I would be careful with using this method and check the generated PDF versions with your eyes before writing them off as "all is good, it is archived now". I recently got bitten by that, when I was trying to print out some page in Chrome, and it was rendering as a bunch of white space surrounded by some elements from the page, but without any actual content I cared about. Turns out, my situation isn't that uncommon for pages that are heavily JS-dependent Note: I am not saying JS=bad. This has nothing to do with JS itself and everything to do with how JS is used to generate/render the page. A lot of pages just don't bother with doing it the right way that doesn't screw up generated PDFs.

fit2rule 6y ago
I've since learned in this thread that Chrome and Firefox are not as good as Safari for this technique - it hasn't impacted me much since I only use Chrome/Firefox for development, mostly. And although I do occasionally check the produced PDF's, the layout doesn't matter to me at all since I use a cmd-line grep or combination of 'pdftotext' to find the page, open the PDF, and click the link to go to the original web page if I need to .. haven't found a single dud PDF in the collection in a randomised sampling, but then again in 20,000+ files, there's bound to be one that didn't make it through the rendering pipeline, but so far, hasn't been an issue.
- filoleg 6y ago
  That clarifies it a lot, thanks for explaining. I actually remember having to eventually go through Safari to print out that one page I mentioned above that was giving me issues in Chrome, so that makes a lot of sense. Glad to find out it wasn't just me somehow being lucky with Safari, and that it is actually a known thing.
- deleted 6y ago
  [deleted]

wazoox 6y ago

I discovered Zotero for this use. I don't have any use of its bibliographical abilities, but it stores web pages and PDF articles fine, and is searchable, etc.

nishparadox 6y ago
True Zotero is a neat tool, especially for files and web pages. I've been using it to read research papers mostly.
igravious 6y ago
Yes, because of all the metadata you can preserve with the save. Also, frequently what yo want to save is a link to a book or an article (or a Wikipedia article, say) and Zotero recognizes many of these formats and saves them correctly, it's toolbar icon even changes to let you know. It's a fantastic tool. It's got cloud storage, and it has an API (which I've used -- it's super easy).

throwawaysea 6y ago

Are you manually printing each page to PDF? I would love to have an automated way to do this but haven't figured out how to deal with logging into subscription based site and all that. There is also some degree of messiness even with printing to PDF. For example let's say I want to save an HN or Reddit discussion along with the comments - I would need to make sure I capture all the comments that overflow to "More" on HN or are behind a "load more comments" link on Reddit. Is there any elegant way to traverse all that and capture it?

nikisweeting 6y ago
ArchiveBox.io saves to PDF and to screenshot and WARC, and HTML to avoid issues that PDF alone has, but the feature that allows archiving sites behind a login isn't completely finished yet.
fit2rule 6y ago
Yes, manually, but as I stated elsewhere: its muscle memory and smooth as an operation for me locally anyway now. I often go through the archive, find the HN comment PDF's I've created, and then automatically update them to get whatever new comments have occurred in the meantime. Haven't figured out how to navigate to the 'next' comment page automatically though - some pages detect they're being printed and use a print-friendly layout, though, which is nice .. would be cool to see more of that.

andrepd 6y ago

How is this "not as powerful"? What is this tool lacking that save to pdf provides? I can see at least one way it is vastly inferior, in that you break formatting by converting to a horrendous paper page-based format.

fit2rule 6y ago
Off-line mode, my data is my data, and I can process the data freely and easily using my own local tools without getting anyone else' CPU involved. And the formatting issue isn't really that big of a deal, if I'm honest. The formatting did its work in the initial contact of the web page - beyond that, to me anyway, its superfluous to the later task of finding the reference again. pdftotext don't care about the formatting, either.
- andrepd 6y ago
  >Off-line mode, my data is my data, and I can process the data freely and easily using my own local tools without getting anyone else' CPU involved. Open the page and you will probably realise that unless you want to pay for sync, this extention works offline.
  - fit2rule 6y ago
    I don't want to pay for sync, because print-to-pdf is my sync, and is free.

alexmarkelenkov 6y ago

Tip for a better use of browser's print to PDF feature: use an ad blocker (uBlock Origin in Chrome works fine) to remove noisy html elements, ads and clutter. I basically cut everything but text, even navigation elements. And this btw is where Chrome comes handy for (e.g.) Firefox users: you can browse and print separately. You always have a ready-to-print setup in Chrome (or else) while browsing somewhere else. I haven't yet looked into another promising feature though, which is (if I'm not mistaken) the availability of a CLI for this . Would be very interested to automate the process, but am concerned about whether the element blocking will work. Ideally I'd like to be able to save printed pages either automatically or via bulk selection (within my browser history) to Evernote or anything like (it seems to be the best solution for me, as it has a great indexing, fuzzy search and relevance, plus the fact that the storage is not limited unless I start to upload more than 10 GB per month which is unlikely). Anyway, would be glad to hear if anyone came up with a similar or a better solution.

bharani_m 6y ago

I am also a big fan of Print to PDF. I've actually built a simple bookmarking service [0] that does just this. EmailThis extracts meaningful content from web pages and sends it to your email inbox. You can also tell it to save a PDF copy of each page, in which case the PDF is sent as an attachment. Print-to-Pdf is done using Headless Chrome (so it works exactly like doing a Ctrl-P). I find that the Print to PDF works best because it gives you a copy of the web page even if the original one disappears. Also, none of the content extraction services (mine included) work in 100% of the cases. Sometimes, they might incorrectly remove images and other meaningful content. So in such cases, having a full PDF snapshot is quite handy. Let me know what you guys think. [0] https://www.emailthis.me https://www.emailthis.me

fit2rule 6y ago
Nice way to find out what everyone is reading while gathering addresses of smart people. ;) > Sometimes, they might incorrectly remove images and other meaningful content. So in such cases, having a full PDF snapshot is quite handy. Also interesting is that the context is preserved locally across visits to the site - over 10 years, I have gathered a pretty interesting view of some of the various A/B changes that have gone on, on my favourite 'daily visit' sites .. And, it is often very revealing of my own habits. This highlights the privacy-factor of having a local-file based bookmark/ontology system a little more in my favour.

Galaco 6y ago

To be a little pedantic, while this is a fantastic idea, is it really bookmarking? What you’ve done instead is compiled a personal digital library; akin to a Kindle

fit2rule 6y ago
Inasmuch as every PDF still has the URL to the original page, for my uses - I'd say yes, it is bookmarking. Its not like I'd cut the spine off every book in my library and create an index out of the covers ..
- fauigerzigerk 6y ago
  Apparently only Firefox saves the source URL. In PDFs saved from Safari or Chrome on macOS I can't find the URL anywhere. Maybe I'm missing something.
  - fit2rule 6y ago
    Interesting. For me, its right there on the bottom-left of the footer, always. /tmpᐅ pdftotext Add\ Comment\ -\ Hacker\ News.pdf - |grep http https://news.ycombinator.com/reply?id=23232588&goto=threads%3Fid%3Dfit2rule%2323232588 https://news.ycombinator.com/reply?id=23232588&goto=threads%...
    - fauigerzigerk 6y ago
      Weird. It's not in there when I store it.
      - fit2rule 6y ago
        Lets compare versions just so I know this feature isn't going away any time soon. Safari 13.1 here, MacOS 10.14.6...
      - cilefen 6y ago
        Try checking "Print headers and footers" in the print dialog.

dependenttypes 6y ago

Wouldn't being able to print to epub be better? epubs are just zipped html files after all so the conversion is more direct and you don't lose any info.

fit2rule 6y ago
PDF just has better tooling for search, which supplants my need for proper formatting since by the point I'm searching for things in my history its the content that matters to me anyway, exclusive of the original formatting.

phiresky 6y ago

> Yeah, my computer is fast enough that I can just do "find . -name '*.pdf' -exec pdftotext {} \; | grep -i someSearchTerm" and come back later. You may be interested in my ripgrep-all [1] tool, it should allow you to search those tens of thousands of PDFs in under a second (with hot cache) [1] https://phiresky.github.io/blog/2019/rga--ripgrep-for-zip-targz-docx-odt-epub-jpg/ https://phiresky.github.io/blog/2019/rga--ripgrep-for-zip-ta...

tomcooks 6y ago
Really great tool, if there is a way to promote it to the main pacman archives, instead of having to use AUR, let me know where I can vote
fit2rule 6y ago
Thanks for that - yeah, silversearch-ag also does a pretty good job.. Its really not a big deal to search so many docs these days, either ..

validuser 6y ago

Many sites have horribly broken/nonexistent print styling. Any tips on how to get good prints matching what you see in the browser?

aiilns 6y ago

I think it all depends on what you are using bookmarks for. I bookmark sites I want to check out again in the near future, but searching in my bookmarks/tagging as memex does has never been an issue for me. If it is something worth searching its text, then it is something worth saving offline, reading and annotating. I use Polar [1] for most and wallabag [2] for its .epub converting ability - especially if there is mainly text that interests me and a lot of it, so I can read it on my ereader. As soon as Polar manages .epubs I shall import all my .epub articles into it. :) [1] https://getpolarized.io/ https://getpolarized.io/ [2] https://wallabag.org/en https://wallabag.org/en

fit2rule 6y ago
Print-to-PDF, put the important sites in a Folder on your Desktop .. The URL for every single site is in the PDF.

deleted 6y ago

[deleted]

gildas 6y ago

> nothing beats print-to-PDF. Its just awesome. HTML largely beats PDF for this use case. And if you want something that produces a file in which you can easily extract resources from the saved page, see [1]. There is an option to automatically save the pages you add to your bookmark. There's also an option to make the text of the files indexable without unzipping them. [1] https://github.com/gildas-lormeau/SingleFileZ https://github.com/gildas-lormeau/SingleFileZ

detaro 6y ago

I really hate this kind of response. No, your thing is not strictly better. It's cool, it might be better for you and many others, but it doesn't even do the same thing (archive all visited pages)!

Abishek_Muthian 6y ago
True, there are different approaches to solve this problem according to the individual preferences. A user recently said, X would use 'Calendar to remind about bookmarks' in a discussion about this problem[1]. [1]https://needgap.com/problems/57-i-forget-my-web-bookmarks-quite-often.-internet-organizing https://needgap.com/problems/57-i-forget-my-web-bookmarks-qu...
fit2rule 6y ago
I really hate this kind of whinging response. This archives every page I'm ever interested in, just fine. Links are preserved just fine, all the data that got me interested in the web page in the first place are just there on my disk, easily accessible any time of day without requiring any further accounts. It is better since it doesn't require any involvement of a third party, is always accessible to me no matter the state of the Internet, and gives me absolute control over all of the data, which I can mine using whatever toolset I want. In fact, I get more data out of this method than the service described in the article.

nuka_coffee 6y ago

Based on your comment I stumbled upon this tool: https://wkhtmltopdf.org/ https://wkhtmltopdf.org/ It preserves hyperlinks and styling. Neat.

choicenotchance 6y ago

And I've been saving them as epub for the last 7.