5 ms·
Ask HN: Should we be saving our favorite information locally?
Here's a thought, and someone please tell me if I'm wrong, but if you have bookmarks to favorite articles, essays, poems, etc... I recommend you use the browser print function to print them to PDF format and save them locally. Reasons:
1) With the advent of ChatAI people will be googling much less if at all, which will reduce traffic to websites. It may not make economic sense for the websites to stay up, so your favorite essay will go away. It might be saved in the internet archive, or it might not
2) Hard drives are ridiculously cheap. A 10 terabyte hard drive (10,000 gigabytes) is less than $200
3) An AI like ChapGPT is not guaranteed to be trained on your favorite information, so it may be lost to you or hard to find again
4) Soon we will have AI assistants which can be trained on all the PDFs you have saved to produce a highly customized and personal AI tailored to what you like
- chromoblob 3y ago> It might be saved in the internet archive, or it might not Anyone can save the current content of any http:/https: URL in Wayback Machine, so the question is simply whether IA will be around for the time that you care about. https://web.archive.org/save https://web.archive.org/save > It may not make economic sense for the websites to stay up So, no more WWW?
- logicalmonster 3y agoI'm not certain what kind of impact AI will have on the marketplace, but it seems like a good idea to store what you value locally regardless of what happens with AI. Sites die off. Online web archives aren't reliable or completely trustworthy. And censorship of many forms seems to be occurring more and more. With storage so cheap, there's little downside to saving what you like.
- labrador 3y agoIt's a bit of work, more than just bookmarking it, but I've started doing it. My thinking is that we could see websites going away at a much higher rate now that we have LLMs.
- JohnFen 3y agoI have had the habit of saving my favorite stuff locally for a very long time. If it's locally stored, it's always available. If it's online, there's a chance that you'll lose access either temporarily or permanently.
- theandrewbailey 3y agoAbsolutely. The internet has proven itself to be ephemeral. The only part of it that is guaranteed is now. Content can be silently changed. Posts get deleted. Links break and 404. Images get lost. Sites put up paywalls, or go down entirely if the owners go bankrupt. If you find something worth saving, save it! And don't forget to back up your stuff!
- animesh 3y agoYes, had the same realization some months ago. I started building a CLI based tool, smaller in scope, offline first and occasionally online.
- labrador 3y agoI've used HTTrack before. It's handy for downloading entire websites. The resulting pile of html can be converted to text or PDF later for processing https://www.httrack.com/ https://www.httrack.com/
- xp84 3y agoHow do things like these cope with client-side render websites? Man, do I miss back when you could “save as webpage, complete” and generally get a working copy. I saved webpages a lot because after all, you might not always be online in those days!
- animesh 3y agoHTTrack - a name I hadn't heard in 15 years may be. I used it back then. However, I want to clarify that the thing I set out to build is not about entire site backup, but more of site URL and some metadata, but with local-first approach and self-hostable.
- Terretta 3y agoYes and no. Instead of PDF, use Markdownload (on iOS, use a Safari web content to markdown file extension): https://github.com/deathau/markdownload https://github.com/deathau/markdownload And save in a journaled folder like "YYYY-MM-DD - Page Title.md" with a YAML frontmatter of all available metadata. Have this as a folder in your PKM of choice (Obsidian, Foam, whatever). These days, point some text embedding at it, and let it generate your own LLM brain. But you can also static-site-generate that back into your own web knowledge site or base. If you don't need it locally, and depending on the capture you want, consider pinboard.in or historio.us: https://pinboard.in/ https://pinboard.in/ https://historio.us/ https://historio.us/
- labrador 3y agoI figured I'd have to convert PDF or HTML to some format like markdown later, but for some articles I could do as you say. For others with great images inlined I'll stick with PDF until I actually need to convert. I'll check out those sites, but I don't trust that they'll stick around.
- blharr 3y agoCan Markdownload handle things like equations within text?
- Jtsummers 3y agoFrom the 3.1.0 notes: Added support for MathJax -> LaTeX (thanks @LeLocTai) So if done with MathJax, yes.
- sdwr 3y agoOnly if you're a hoarder, or it's career-related and you're meticulous. I find that if I start saving stuff, the drive to collect starts outweighing the value of the collection rapidly. AI search would be the only reason to. If it saves everything automatically and can query references / make inferences seamlessly, then great. Anything less, and my life eats itself like a snake.
- labrador 3y agoI'm not talking about hording, which implies collecting every thing, but just your favorite, quality info that you can train an AI on later to produce a highly customized AI
- quickthrower2 3y agoI agree. Instapaper (phone app) is a good tool for doing this. But pdfs are probably more “open” in that you know the format and can choose where to put the files. Internet archive sometimes saves dead links though.
- bsnnkv 3y agoI have a slightly different take on this: I save the text that I care about, and have some automation set up to archive the source URL of the text to archive.org[1] (which works well enough for me, even if it's not 100% perfect, because I'm only archiving it for the greater context of the highlighted text, which I rarely go back to). I just got myself an Nvidia 4090, and I'm looking into using local LLMs to feed my data into (I think this is called retrieval augmented generation?) for various assistant-type use cases. I'm particularly excited to potentially be able to go through my saved Kindle highlights for multi-novel sci-fi and fantasy series in order to refresh my memory by clarifying key story beats before continuing with the next book. [1]: https://lgug2z.com/articles/notado-07-2023-update/ https://lgug2z.com/articles/notado-07-2023-update/
- puttycat 3y agoYes. Even before the AI risks, I have moved everything locally – photos, files, nothing syncs to iCloud/Dropbox/etc. Every once in a while I prepare and order printed photo albums. The only service I still did not localize is email, secifically Gmail, which I believe Google is imminent to monetize/AI-ize in the very near future.
- HenryBemis 3y agoFor backups I do both local & cloud. I got a WD 4TB Gold to be my "D drive" as my "Library" I then back the "my Docs", and "Portable Apps" folders on Carbonite (my keys, unlimited space). I also dump an Acronis TIB of my C drive to my D drive, so that it is also siphoned to the Carbonite. Using my keys make it harder, in case of a crash I need to build a vanilla machine, download the whole blob, and go from there. If I trusted Microsoft (which I don't) I would have a second machine with some 'handover' (which I haven't studied) so I would leave MS keep my machines synced. But I don't, and I don't :)
- noman-land 3y agoThe answer is a resounding yes. The corporate cloud is fake. A mirage. A timebomb whose chance of going off nears 100% over time. When it goes off your digital belongings are gone. The only way to own things is to have copies offline. In three or more geographically distant locations.
- appleaday1 3y agoI keep my stuff on Dropbox and Github and it has been working for a while.
- mikewarot 3y agoThis is an old idea... when it was first proposed by Bush[1], the media to record on was microfilm, not PDF files. It's never been implemented, and forces that believe in "intellectual property" are aligned against doing so. (One of the main features was the ability to dump a selected "trail" through documents to microfilm for others) You're right, of course. I'd like to see a local proxy that caches everything for at least a month, then automatically keeps stuff referenced or revisited. [1] https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/ https://www.theatlantic.com/magazine/archive/1945/07/as-we-m...
- dinvlad 3y agoThis is such a great idea! I've used Squid like 15 years ago when no suitable AI existed. Looks like it can finally be put to some use :-)