19 ms·
Most "read it later" services save web pages on their servers, which cannot preserve paywalls, ad-free content, pages that require login or registration, and pa
by hamsterbase 3y ago
Most "read it later" services save web pages on their servers, which cannot preserve paywalls, ad-free content, pages that require login or registration, and pages on a local network.
Sharing why I decided to develop my own "read it later" software.
1. I have a habit of saving web pages I like, most of which are in MHTML format, some are saved as single-file HTML, and others are web archives saved on iOS. Altogether, I have accumulated thousands of them.
2. On my computer, I can preview them one by one, but I cannot search through them. So, I developed a Node.js service that parses web pages locally and stores them in an SQLite FTS for full-text search. I deployed the service using Docker on my NAS.
3. To enhance my learning experience, I also developed an annotation feature that allows me to make notes and annotations directly on the offline HTML. For good articles, I save them to read slowly over time.
4. Gradually, the app gained some users. Since they were not familiar with Docker, I wrapped it with Electron and developed a standalone desktop version. The desktop version and Docker version use CRDT for peer-to-peer synchronization.
5. Some users provided feedback that it was inconvenient to annotate after saving, so I developed an open-source browser extension. Users can now annotate web pages directly in the browser, and the annotations and snapshots are saved automatically. When visiting the page again in the future, previous annotations can be restored.
- Dr4kn 3y agoWhat is your program called?
- hamsterbase 3y agohttps://hamsterbase.com https://hamsterbase.com
- whitmank 3y agoI love this! Gonna try it out over the next couple of days.
- wsostt 3y agoGoing to check this out. FYI - your Obsidian and Logseq links on the honmepage are swapped.
- BiteCode_dev 3y agoJust learning it exists and it's something I could never found a good software for. And it supports saving things with the excellent SingleFile. I'm definitely going to give it a try, although as a Firefox user the workflow is going to be a bit less nice. EDIT: just tried it, it's very nice. Right now, the UI makes it a bit better than just using in SingleFiles, but not enough to justify $50. However, if this would become a way to fully manage my Firefox bookmarks, I would pay for this, in the spirit of https://www.bitecode.dev/p/i-checked-if-browsers-could-cache https://www.bitecode.dev/p/i-checked-if-browsers-could-cache. Right now, the UI to manage the bookmarks sucks. Plus there is no way to: - save their content for offline reading - search their content This means it's are very hard to get value out of them.
- hamsterbase 3y agoSinglefile supports batch saving URLs to local storage. Right-click on the webpage > select Singlefile > choose "Batch save URLs". Using this method, I have converted hundreds of my bookmarks into html files. I also think that Hamsterbase is not worth $60, so I offered a free trial. Only after it is released on iOS and Android will I consider charging.
- poisonborz 3y agoLooks cool, but writing "pay for what you need" and no pricing info on the page is really off-putting.
- replygirl 3y agoTried the pricing page?
- hamsterbase 3y ago"Thank you for your reminder. May I ask if you are accessing the website on your mobile phone? The pricing page is located at the top of the official website for desktop version. If you are using the mobile version, you will need to click on the hamburger button. I have updated the wording to: “Free during the beta period, Provide Believer plan, one-time purchase (60$) with lifetime updates.”
- poisonborz 3y agoUh, yes, sorry. I also use a dark theme extension that renders the menu icon almost invisible (maybe try to view the page with some of them, in high enough contrast ratio this shouldn't happen). I thought the footer had all the links, so maybe that should be extended - stating the price on multiple places does not seem necessary otherwise. Price seems to be reasonable, good luck with your project!
- hamsterbase 3y agoYou are right, I added the price link in the footer.
- deleted 3y ago[deleted]
- jacquesm 3y agoHa, I love the 'Believer plan', it's much more honest than most of the vaporware sellers out there (errm... kickstarters) and to outdo them you have at least released. Kudos. I'll consider becoming a believer, since the 'scrapbook' extension got murdered by FF leaving me with years of inaccessible notes I've been looking for a replacement, even if not perfect it will definitely have to be based on open standards and open source.
- asaddhamani 3y agoYesterday I was on a flight so I downloaded some paywalled articles on my phone using safari reading list. Once we took off and I lost connectivity I opened the article and saw - the paywall page. I tried looking for mobile read later apps that save offline and save what I saw not what their servers see, but not much luck. I’ve my own app which can save pages as I saw them and create reader views out of them, but no offline access yet. This is a pretty niche use case but I feel these days more and more decent writing is behind paywalls (Substack and newsletters) so the current read it later apps are losing usefulness and they’re all frozen on features.
- pm3003 3y agoOpera offline save saves locally, it's efficient. I never had the paywall problem.
- simmerup 3y agoHonestly Apple just in general stops working the moment you lose internet connection and it sucks. The whole ecosystem seems to assume you always have internet.
- birdyrooster 3y agoMaps is offline at least finally
- pmontra 3y agoFirefox Android used to save a local copy of pages bookmarked from inside reader mode. Unfortunately that went away years ago, maybe 2018 or 2019.
- spectre3d 3y agoThat’s really rough. Instapaper (free version) and GoodLinks ($5) both work perfectly for me without internet access. Haven’t tried substacks in GoodLinks, but Instapaper grabs them without issue. Nice to hear about hamsterbase, will check it out.
- 3y ago
- wintermutestwin 3y ago>web archives saved on iOS Could you elaborate on how you are doing this?
- hamsterbase 3y agohttps://www.techrepublic.com/article/how-to-archive-websites-as-pdfs-and-web-archives-in-ios-14/ https://www.techrepublic.com/article/how-to-archive-websites... You can take a look at this article. I will sync web archives to my Mac through iCloud, and then import it in bulk into Hamsterbase.
- PaulHoule 3y agoSee https://getupnext.com/ https://getupnext.com/, it can save many paywalled articles and dejunk quite a few pages. Their infrastructure for “read it later” is great and they told me they are working on adding a recommender to it to make it a more complete solution.
- gxqoz 3y agoMy understanding is that Readwise uses the HTML you're seeing on your browser to make the copy and this works with paywalls better than most services I've tried. The downside is you need to give their desktop browser plugin more permissions than is ideal and this doesn't work when saving from mobile.