3 ms·
This is really interesting, is there a way to provide downloadable caches for websites where that would be legal? I could Imagine just pre-downloading wikipedi
by dec0dedab0de 8mo ago
This is really interesting, is there a way to provide downloadable caches for websites where that would be legal? I could Imagine just pre-downloading wikipedia, stack overflow, all kinds of documentation, etc. In a compressed and preorganized format instead of scraping it every time.
- ploum 8mo agoThe cache is pure and straight files-in-folders. This makes it trivial to browse the cache by hand: cd .cache/offpunk/https/news.ycombinator.com/item cat "id=46943752" So it could be trivially shared. The "netcache" tool gives you the cached content or, with --path, returns the path were to find the contentd. The only point is to preserve the file-modification attribute, which serves to know the age of a cached ressource.
- iamnothere 8mo agoKiwix basically provides this. You can use kiwix-serve to serve the downloaded zim files locally.
- ploum 8mo agoI tried to add kiwix support to Offpunk in the early days but kiwix do not support incremental update of the wikipedia database, which makes it a huge pain point.
- iamnothere 8mo agoYeah, that’s the biggest disadvantage of kiwix. It probably makes their hosting bill enormous as well. Zimdiff/zimpatch was implemented a decade ago so I’m not sure what the holdup is.